A Novel Big Data Index Architecture for Programming Environment

Abstract

Abstract With the rapid development of computer information technology and the upgrading of programming software, the types and number of project codes are growing rapidly, showing typical characteristics of large data such as massive, instantaneous, diverse and variable. The distributed column storage database HBase based on the Hadoop big data platform, has the characteristics of high reliability, high performance, column-oriented, and scalability. It also has good scalability, can store more than ten billions of data, and is suitable for large-scale data reading and writing, which takes advantages in processing large-scale unstructured and semi-structured software data on the programming site. So, this paper studies the big data index architecture in the programming field. In view of the low efficiency of HBase non-primary key attribute query, the inverted index is a natural platform for cloud-based big data storage and query in this large development site. Based on this advantage, this paper designs and optimizes the secondary index architecture based on the HBase for programming field big data inverted index.