An efficient distributed hierarchical-clustering algorithm for large scale data

Cheng Hsien Tang, An Ching Huang, Meng Feng Tsai, Wei Jen Wang

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

3 Scopus citations

Abstract

The data-classification process can possibly involve a huge amount of data in today's cloud computing environment. It could take a long time for processing, and could consume many resources for computation and storage. This study focuses on the problem of using the traditional hierarchical agglomerative clustering algorithm on a distributed environment since hierarchical agglomerative clustering has high applicability and efficiency. A parallel hierarchical agglomerative clustering algorithm is proposed in this study. The proposed algorithm divides the whole computation into several small tasks, distribute the tasks to message-passing processes, and merge the results to form a hierarchical cluster. A threshold is used to reduce the storage requirement during the computation. To evaluate the performance and limitation of our algorithm, this study has conducted several experiments using real astronomical data, the main asteroid belt catalog. The experimental results confirm that the proposed parallel algorithm is efficient.

Original languageEnglish
Title of host publicationICS 2010 - International Computer Symposium
Pages869-874
Number of pages6
DOIs
StatePublished - 2010
Event2010 International Computer Symposium, ICS 2010 - Tainan, Taiwan
Duration: 16 Dec 201018 Dec 2010

Publication series

NameICS 2010 - International Computer Symposium

Conference

Conference2010 International Computer Symposium, ICS 2010
Country/TerritoryTaiwan
CityTainan
Period16/12/1018/12/10

Keywords

  • Hierarchical clustering
  • Parallel computing

Fingerprint

Dive into the research topics of 'An efficient distributed hierarchical-clustering algorithm for large scale data'. Together they form a unique fingerprint.

Cite this