Advances in Data Science

6Incremental Calculation Framework for Complex Data

In today’s digital world, the data for business analysis are more likely to be collected from diverse sources. This contributes to the complexity of data analysis because of the introduction of many uncommon data types, such as functional data, compositional data, histogram data, and so on. Much attention has been paid to the statistical methods about these new types of data. However, there have been few studies about the incremental calculation for these complex data types, which is necessary in practical applications at present. In this chapter, we develop an incremental calculation framework for complex data, one that can be applied to various data types. We first transform the complex data into basic data and then propose the incremental calculation method based on this type of data. The incremental calculation framework can be implemented to many frequently used statistical models built on the covariance matrix. We take linear regression of functional data and principal component analysis of compositional data as examples to discuss the incremental calculation for complex types of data. The simulation result shows both the efficiency and effectiveness of this framework.

6.1. Introduction

With the maturity of “big data” technology, the data used for business analysis are collected from various sources, which makes the data type quite complex. For example, in the case of movie box office prediction, the data from searching ...