Machine Learning

Problems

4.1 Show that the set of equations

$\Sigma \mathit{\theta }=\mathit{p}$

has a unique solution if Σ > 0 and infinite many if Σ is singular.

4.2 Show that the set of equations

$\Sigma \mathit{\theta }=\mathit{p}$

always has a solution.

4.3 Show that the shape of the isovalue contours of the mean-square error (J(θ)) surface

$J(\mathit{\theta })=J({\mathit{\theta }}_{*})+{(\mathit{\theta }-{\mathit{\theta }}_{*})}^{\text{T}}\Sigma (\mathit{\theta }-{\mathit{\theta }}_{*})$

are ellipses whose axes depend on the eigenstructure of Σ.

Hint

Assume that Σ has discrete eigenvalues.

4.4 Prove that if the true relation between ...