Practical Big Data Analytics

The importance of variables

During model-building exercises, datasets may have tens of variables. Not all of them may add value to the predictive model. It is not uncommon to reduce the dataset to include a subset of the variables and allow the machine learning programmer to devote more time toward fine-tuning the chosen variables and the model-building process. There is also a technical justification for reducing the number of variables in the dataset. Performing machine learning modeling on very large, that is, high dimensional datasets can be very compute-intensive, that is, it may require a significant amount of time, CPU, and RAM to perform the numerical operations. This not only makes the application of certain algorithms impractical, ...