A Memory-Resident Index Structure to Enhancing Read Performance for NoSQL

Abstract

Nowadays, big data has become one of keywords representing the IT market. Big data is a large amount of data. lot of data has been generated in daily life from various places such as words that people are searching for on the Internet and the moving route data obtained through the traffic cards to the public data such as the social network service and the data transmitted and received between the base station and the mobile phone. Big data technology was started from the thought of how massive data produced from company can be used in useful. Big Data technologies fall into DBMS technology for managing data and a file system for storing data. The representative storage technology is Hadoop, and the managing technology for big data is NoSQL database. So, many companies have been analyzing the data and obtaining the useful data based on NoSQL. You can find examples using big data in SAP Match Insight and Google Flu trends. HBase, the Representative NoSQL database, uses Log-Structured Merge tree to optimize to write large amounts of data. The write operation of the tree structure is performed in a way to store the data in the memory first. And the data in the memory stores to HDFS by the batch write method. Therefore, HBase optimizes the write operation. HBase also has the same characteristic features of HDFS that is impossible to update because it uses HDFS. Therefore, many files are stacked as the data is increased. Searching the data is needed many accesses to the file system. For solving this problem, we propose the index structure in HBase for improving the read performance. In this paper, we show that the proposed system yields better performance than the existing read operation of HBase through the experiment.