Performing optimization is a common data science task. In this recipe, we implement a simple optimization routine using the Marquardt algorithm. For more information on this, see http://mathworld.wolfram.com/Levenberg-MarquardtMethod.html or https://en.wikipedia.org/wiki/Levenberg%E2%80%93Marquardt_algorithm. In the process of optimizing, we will also discover how Julia handles linear algebra and numerical differentiation.
The basic idea of the procedure is the following. Given a twice differentiable function, , and some point, , we want to find another point, , such that . By repeating this process, we ...