Matrices are used as mathematical objects to represent images, datasets for real world machine learning applications like a face or text recognition, medical imaging, principal component analysis, numerical accuracy, and so on.
As an example, eigen decomposition is explained here. Many mathematical objects can be understood better by breaking them into constituent parts, or by finding properties which are universal.
Like when integers are decomposed into prime factors, matrix decomposition is called eigen decomposition, where we decompose a matrix into eigenvectors and eigenvalues.
Eigenvector v of a matrix A is such that multiplication by A alters only the scale of v, as shown next:
Av = λv
The scalar λ is known ...