Singular values lets us understand the trade-off between space and time for fidelity of the reduction.
As with evaluating values of k for clustering, in the case of SVD (and PCA), it is often useful to plot the singular values for a larger range of k, and see where the point on the graph is where the amount of additional variance accounted for by each additional singular value starts to flatten out considerably.
We will do this by first computing the top 300 singular values, as shown next:
val svd300 = matrix.computeSVD(300, computeU = false) val sMatrix = new DenseMatrix(1, 300, svd300.s.toArray) println(sMatrix) csvwrite(new File("/home/ubuntu/work/ml-resources/ spark-ml/Chapter_09/data/s.csv"), sMatrix)
We will write ...