A survey of Web information extraction systems

Chia Hui Chang, Mohammed Kayed, Moheb Ramzy Girgis, Khaled F. Shaalan

Research output: Contribution to journalArticlepeer-review

582 Scopus citations

Abstract

The Internet presents a huge amount of useful information which is usually formatted for its users, which makes it difficult to extract relevant data from various sources. Therefore, the availability of robust, flexible Information Extraction (IE) systems that transform the Web pages into program-friendly structures such as a relational database will become a great necessity. Although many approaches for data extraction from Web pages have been developed, there has been limited effort to compare such tools. Unfortunately, in only a few cases can the results generated by distinct tools be directly compared since the addressed extraction tasks are different. This paper surveys the major Web data extraction approaches and compares them in three dimensions: the task domain, the automation degree, and the techniques used. The criteria of the first dimension explain why an IE system fails to handle some Web sites of particular structures. The criteria of the second dimension classify IE systems based on the techniques used. The criteria of the third dimension measure the degree of automation for IE systems. We believe these criteria provide qualitatively measures to evaluate various IE approaches.

Original languageEnglish
Article number1683775
Pages (from-to)1411-1428
Number of pages18
JournalIEEE Transactions on Knowledge and Data Engineering
Volume18
Issue number10
DOIs
StatePublished - Oct 2006

Keywords

  • Information extraction
  • Web mining
  • Wrapper
  • Wrapper induction

Fingerprint

Dive into the research topics of 'A survey of Web information extraction systems'. Together they form a unique fingerprint.

Cite this