Modern Deep Learning for Tabular Data: Novel Approaches to Common Modeling Problems

© The Author(s), under exclusive license to APress Media, LLC, part of Springer Nature 2023

A. Ye, Z. WangModern Deep Learning for Tabular Datahttps://doi.org/10.1007/978-1-4842-8692-0_10

10. Meta-optimization

Andre Ye¹   and Zian Wang²

(1)

Seattle, WA, USA

(2)

Redmond, WA, USA

 

If you optimize everything, you will always be unhappy.

—Donald Knuth, Computer Scientist

The legendary Donald Knuth was right about many things, but meta-optimization is indeed strong evidence that most people can be pretty happy optimizing most things (not quite everything, admittedly). Optimization in its standard form concerns the reconciliation of ambiguities and uncertainties on the direct level of parameter-to-loss model. But when we construct said models, we also run into ...