Levenberg-Marquardt Method
This module demonstrates the Levenberg-Marquardt method for nonlinear
least squares. Given an approximate solution, a new approximate
solution is computed as a weighted combination of the
Gauss-Newton step and the
steepest descent direction.
This process is repeated until convergence.
The user selects a problem either by choosing a preset example or
typing in a desired function f of parameters
x, y. Contours of the least squares residual are drawn
on the plot. An initial guess (x, y) can be
selected either be typing it in or by clicking on the plot. The steps
of the Levenberg-Marquardt method are then carried out sequentially by
repeatedly clicking on NEXT or on the currently highlighted step. The
current point (x, y) is indicated by a
bullet on the plot and all values are also shown numerically in the
table below. At each iteration, the Levenberg-Marquardt step is
computed as a weighted combination of the Gauss-Newton step and the
negative gradient of the residual function. The weighting parameter
μ can be selected by the user using the slider provided. The
approximate solution is updated using the computed step, and the
process is then repeated. If the starting guess is close enough to the
true minimum, then the Levenberg-Marquardt method usually converges to
it, typically with a linear convergence rate.
Reference: Michael T. Heath, Scientific Computing,
An Introductory Survey, 2nd edition, McGraw-Hill, New York,
2002. See Section 6.6.2 and Example 6.15.
Developers: Jeffrey Naisbitt and Michael Heath