GeoWeb Crawler: An extensible and scalable web crawling framework for discovering geospatial web resources

Chih Yuan Huang, Hao Chang

研究成果: 雜誌貢獻期刊論文同行評審

18 引文 斯高帕斯(Scopus)

摘要

With the advance of the World-Wide Web (WWW) technology, people can easily share content on the Web, including geospatial data and web services. Thus, the "big geospatial data management" issues start attracting attention. Among the big geospatial data issues, this research focuses on discovering distributed geospatial resources. As resources are scattered on the WWW, users cannot find resources of their interests efficiently. While the WWW has Web search engines addressing web resource discovery issues, we envision that the geospatial Web (i.e., GeoWeb) also requires GeoWeb search engines. To realize a GeoWeb search engine, one of the first steps is to proactively discover GeoWeb resources on the WWW. Hence, in this study, we propose the GeoWeb Crawler, an extensible Web crawling framework that can find various types of GeoWeb resources, such as Open Geospatial Consortium (OGC) web services, Keyhole Markup Language (KML) and Environmental Systems Research Institute, Inc (ESRI) Shapefiles. In addition, we apply the distributed computing concept to promote the performance of the GeoWeb Crawler. The result shows that for 10 targeted resources types, the GeoWeb Crawler discovered 7351 geospatial services and 194,003 datasets. As a result, the proposed GeoWeb Crawler framework is proven to be extensible and scalable to provide a comprehensive index of GeoWeb.

原文???core.languages.en_GB???
文章編號136
期刊ISPRS International Journal of Geo-Information
5
發行號8
DOIs
出版狀態已出版 - 8月 2016

指紋

深入研究「GeoWeb Crawler: An extensible and scalable web crawling framework for discovering geospatial web resources」主題。共同形成了獨特的指紋。

引用此