Libwebex: A Non-discriminating Web Content Extraction Library

摘要

Libwebex is designed to be a powerful and flexible tool for the purpose of web page content extraction in the domain of Data Integration. Utilizing the semi-structured nature of web pages, this library provides users with a skeletal view of the web page content. The data elements that need to be extracted are arranged into a hierarchical tree of related topics and content. In this representation, users are able to build custom applications on top of this framework using the object-oriented design concept, the Visitor design pattern. All a programmer needs to provide is the customized behavior to be executed at each node in the tree. The library provides multiple basic building blocks for additional applications, such as archiving, searching, and printing mechanisms.