Automatic information extraction from semi-structured Web pages by pattern discovery

Chia Hui Chang, Chun Nan Hsu, Shao Cheng Lui

Research output: Contribution to journalArticlepeer-review

79 Scopus citations

Abstract

The World Wide Web is now undeniably the richest and most dense source of information; yet, its structure makes it difficult to make use of that information in a systematic way. This paper proposes a pattern discovery approach to the rapid generation of information extractors that can extract structured data from semi-structured Web documents. Previous work in wrapper induction aims at learning extraction rules from user-labeled training examples, which, however, can be expensive in some practical applications. In this paper, we introduce IEPAD (an acronym for Information Extraction based on PAttern Discovery), a system that discovers extraction patterns from Web pages without user-labeled examples. IEPAD applies several pattern discovery techniques, including PAT-trees, multiple string alignments and pattern matching algorithms. Extractors generated by IEPAD can be generalized over unseen pages from the same Web data source. We empirically evaluate the performance of IEPAD on an information extraction task from 14 real Web data sources. Experimental results show that with the extraction rules discovered from a single page, IEPAD achieves 96% average retrieval rate, and with less than five example pages, IEPAD achieves 100% retrieval rate for 10 of the sample Web data sources.

Original languageEnglish
Pages (from-to)129-147
Number of pages19
JournalDecision Support Systems
Volume35
Issue number1
DOIs
StatePublished - Apr 2003

Keywords

  • Information extraction
  • Multiple string alignment
  • PAT trees
  • Semi-structured data
  • Wrapper generation

Fingerprint

Dive into the research topics of 'Automatic information extraction from semi-structured Web pages by pattern discovery'. Together they form a unique fingerprint.

Cite this