Statistical Learning with Sparsity

90 GENERALIZATIONS OF THE LASSO PENALTY

Ex. 4.14 Consider the hierarchical interaction formulation in Example 4.3, and

the optimization problem (4.29)–(4.31).

(a) Give an argument why the multipliers p

1

and p

2

make sense in the third

penalty.

(b) Suppose we augment the third matrix in (4.29) with a vector of ones

[1 Z

1

Z

2

Z

1:2

], and augment the parameter vector with ˜µ. We now replace

the third group penalty term with

q

p

1

p

2

˜µ

2

+ p

2

k˜α

1

k

2

2

+ p

1

k˜α

2

k

2

2

+ kα

1:2

k

2

2

.

Show that for any λ > 0,

b

˜µ = 0.

(c) Show that the solution to (4.29)–(4.31) is equivalent to the solution to

(4.32) for any λ > 0. Show how to map the solution to the latter to the

solution to the former. ...