Mastering Machine Learning Algorithms

Example of covariance rule application

Before moving on, let's simulate this behavior with a simple Python example. We first generate 1000 values sampled from a bivariate Gaussian distribution (the variance is voluntarily asymmetric) and then we apply the covariance rule to find the first principal component (w⁽⁰⁾ has been chosen so not to be orthogonal to v₁):

import numpy as nprs = np.random.RandomState(1000)X = rs.normal(loc=1.0, scale=(20.0, 1.0), size=(1000, 2))w = np.array([30.0, 3.0])S = np.cov(X.T)for i in range(10):    w += np.dot(S, w)    w /= np.linalg.norm(w)    w *= 50.0print(np.round(w, 1))[ 50.  -0.]

The algorithm is straightforward, but there are a couple of elements that we need to comment on. The first one is the normalization of ...