TY - JOUR
T1 - An adaptively multi-attribute index framework for big IoT data
AU - Huang, Chih Yuan
AU - Chang, Yu Jui
N1 - Publisher Copyright:
© 2021
PY - 2021/10
Y1 - 2021/10
N2 - In recent years, the concept of the Internet of Things (IoT) has been attracting attention from various fields as IoT devices can continuously monitor various environmental properties. While the number of IoT devices increases rapidly, managing large volume of IoT data faces a serious scalability issue. To address this issue, many studies have shown that the performance of key-value storages is better than traditional relational databases. However, IoT data have multi-dimensional attributes including spatial, temporal and thematic attributes. How to construct an efficient multi-attribute combined index is an important topic. In this research, we consider four main types of attributes and their corresponding queries, which are spatial, temporal, keyword, and value attributes. While each attribute has its own suitable index method, integrating the indexes into a combined index usually requires a certain sequence of indexes, which significantly decides the query performance. As many literatures directly present their designed combined index, this research proposes an adaptive method to decide the most efficient combined index by estimating the selectivity and query performance of individual query criterion. The main idea is that highly-selective queries should be performed first to reduce the number of intermediate results, which can improve the query performance of following queries. Hence, this research proposes an index framework considering every possible sequence and automatically identifying the most efficient combined index for each query. According to the result, the proposed system has 94–99% chance to save 25 to 51 times response time compared to using a single combined index, and is twice faster than PostGIS on average when querying a one-million-record real-world dataset.
AB - In recent years, the concept of the Internet of Things (IoT) has been attracting attention from various fields as IoT devices can continuously monitor various environmental properties. While the number of IoT devices increases rapidly, managing large volume of IoT data faces a serious scalability issue. To address this issue, many studies have shown that the performance of key-value storages is better than traditional relational databases. However, IoT data have multi-dimensional attributes including spatial, temporal and thematic attributes. How to construct an efficient multi-attribute combined index is an important topic. In this research, we consider four main types of attributes and their corresponding queries, which are spatial, temporal, keyword, and value attributes. While each attribute has its own suitable index method, integrating the indexes into a combined index usually requires a certain sequence of indexes, which significantly decides the query performance. As many literatures directly present their designed combined index, this research proposes an adaptive method to decide the most efficient combined index by estimating the selectivity and query performance of individual query criterion. The main idea is that highly-selective queries should be performed first to reduce the number of intermediate results, which can improve the query performance of following queries. Hence, this research proposes an index framework considering every possible sequence and automatically identifying the most efficient combined index for each query. According to the result, the proposed system has 94–99% chance to save 25 to 51 times response time compared to using a single combined index, and is twice faster than PostGIS on average when querying a one-million-record real-world dataset.
KW - Adaptivity
KW - Data management
KW - Index
KW - Multi-attribute
KW - Selectivity
UR - http://www.scopus.com/inward/record.url?scp=85108385196&partnerID=8YFLogxK
U2 - 10.1016/j.cageo.2021.104841
DO - 10.1016/j.cageo.2021.104841
M3 - 期刊論文
AN - SCOPUS:85108385196
SN - 0098-3004
VL - 155
JO - Computers and Geosciences
JF - Computers and Geosciences
M1 - 104841
ER -