Hands-On Mathematics for Deep Learning

Non-convex optimization

In convex optimization, we deal with having to find a local optimum, which also happens to be the global minimum. However, in non-convex optimization, we have to find the global minimum, which isn't the local minimum; in fact, there could be more than one local minimum, as well as saddle points.

This makes non-convex optimization far more challenging than convex optimization.