Spectral Feature Selection for Data Mining

70 Spectral Feature Selection for Data Mining

search to determine how far we should go with this direction, and update w

i

using w

i

= w

i

+ αd (Line 13). The reason for doing this is that using w to

replace w

i

can be too aggressive, which may interfere with the convergence.

The pseudo-code of the line search procedure is shown in Algorithm 3.

Algorithm 3: Line search

Input: X, Y, λ, W, i, d ∈ R

1×k

, α

0

> 0, δ > 0, σ > 0

Output: α

1 ∆ = d

>



Y − XW

>



f

i

+ λkw

i

+ dk

2

+

L

j

2

kdk

2

2

− λkw

i

k

2

;

2 for α = α

0

, δα

0

, δ

2

α

0

, . . . do

3 if f (W

w

i

=w

i

+αd

) − f (W) ≤ ασ∆ then

4 break;

5 return α;

Algorithm 3 presents an inexact line search procedure using the Armijo

rule [131]. In the procedure, ...