MetaNews: An Information Agent for Gathering News Articles on the Web

Abstract

This paper presents MetaNews, an information gathering agent for news articles on the Web. MetaNews reads HTML documents from online news sites and extracts article information from them. in order to recognize and extract articles from an HTML document, MetaNews removes redundant HTML tags, builds a reference string which is a sequence of semantic components of a noise-removed document, and performs pattern matching between the reference string and each of pre-defined information patterns for articles. With a few training inputs from the operator with intermediate-level skills, MetaNews is capable of adding new sites easily and extracting articles in real time. By reducing the complexity of designing and creating wrapper interfaces, the proposed techniques in MetaNews are useful for many information-mediator applications such as meta-search engines, information-push solutions, and comparison-shopping agents.