Intelligent Data Analysis

1Intelligent Data Analysis: Black Box Versus White Box Modeling

Sarthak Gupta, Siddhant Bagga, and Deepak Kumar Sharma

Division of Information Technology, Netaji Subhas University of Technology, New Delhi, India

1.1 Introduction

In the midst of all of the societal challenges of today's world, digital transformation is rapidly becoming a necessity. The number of internet users is growing at an unprecedented rate. New devices, sensors, and technologies are emerging every day. These factors have led to an exponential increase in the volume of data being generated. According to a recent research [1], users of the internet generate 2.5 quintillion bytes of data per day.

1.1.1 Intelligent Data Analysis

Data is only as good as what you make of it. The sheer amount of data being generated calls for methods to leverage its power. With the proper tools and methodologies, data analysis can improve decision making, lower the risks, and unearth hidden insights. Intelligent data analysis (IDA) is concerned with effective analysis of data [2, 3].

The process of IDA consists of three main steps (see Figure 1.1):

1. Data collection and preparation: This step involves acquiring data, and converting it into a format suitable for further analysis. This may involve storing the data as a table, taking care of empty or null values, etc.
2. Exploration: Before a thorough analysis can be performed on the data, certain characteristics are examined like number of data points, included variables, statistical ...