An adaptively multi-attribute index framework for big IoT data

In recent years, the concept of the Internet of Things (IoT) has been attracting attention from various fields as IoT devices can continuously monitor various environmental properties. While the number of IoT devices increases rapidly, managing large volume of IoT data faces a serious scalability issue. To address this issue, many studies have shown that the performance of key-value storages is better than traditional relational databases. However, IoT data have multi-dimensional attributes including spatial, temporal and thematic attributes. How to construct an efficient multi-attribute combined index is an important topic. In this research, we consider four main types of attributes and their corresponding queries, which are spatial, temporal, keyword, and value attributes. While each attribute has its own suitable index method, integrating the indexes into a combined index usually requires a certain sequence of indexes, which significantly decides the query performance. As many literatures directly present their designed combined index, this research proposes an adaptive method to decide the most efficient combined index by estimating the selectivity and query performance of individual query criterion. The main idea is that highly-selective queries should be performed first to reduce the number of intermediate results, which can improve the query performance of following queries. Hence, this research proposes an index framework considering every possible sequence and automatically identifying the most efficient combined index for each query. According to the result, the proposed system has 94–99% chance to save 25 to 51 times response time compared to using a single combined index, and is twice faster than PostGIS on average when querying a one-million-record real-world dataset.