Data Engineering and Data Science

1Quality Assurance in Data Science: Need, Challenges and Focus

Jasmine K.S.*, Ajay D. K. and Aditya Raj

Dept. of MCA, RV College of Engineering^®, Bengaluru, Karnataka, India

Abstract

It is widely accepted that quality is assured for a product through the process of testing. With the rapid development in the area of data science, research is going on with proper management of data and with its right usage, test engineers can learn about their users. One can predict the associated risks and with a focus on data masking based on the data model. Prescriptive and predictive analysis can be more accurate if the techniques are developed and the accuracy is measured using metrics. Preparing data with required quality and identifying the possible resources are challenging tasks faced by a data scientist. The effective and systematic use of advanced technologies like high-speed hardware and network computing, cloud computing, cross platform tools, etc., continues to be an elusive goal for many organizations. In this context, the chapter investigates the feasibility of novel and practical solutions in this aspect.

Keywords: Quality assurance, testing, data science, data analysis, decision making

1.1 Introduction

1.1.1 Quality Assurance and Testing

In the traditional software development approach, Quality Assurance was done at the later stages of the development process and feedback was collected for improvement. In almost all organizations there exists a Quality Assurance team, responsible ...