Theory of Ridge Regression Estimation with Applications

12Applications: Neural Networks and Big Data

This book thus far has focused on detailed comparisons of ridge regression, least absolute shrinkage and selection operator (LASSO), and other variants to examine their relative effectiveness in different regimes. In this chapter, we take it a step further and examine their use in emerging applications of advanced statistical methods. We study the practical implementation of neural networks (NNs), which can be viewed as an extension of logistic regression (LR), and examine the importance of penalty functions in this field. It should be noted that the ridge‐based ‐penalty function is widely, and almost exclusively, used in neural networks today; and we cover this subject here to gain an understanding as to how and why it is used. Furthermore, it is worth noting that this field is becoming more and more important and we hope to provide the reader with valuable information about emerging trends in this area.

This chapter concerns the practical application of penalty functions and their use in applied statistics. This area has seen a rapid growth in the past few years and will continue to grow in the foreseeable future. There are three main reasons why applied statistics has gained increasing importance and given rise to the areas of machine learning, statistical learning, and data science. The first is the advance of compute power. Today, ...