LevenbergMarquardtOptimizer

Overview¶

The LevenbergMarquardtOptimizer class in GTSAM is a specialized optimizer that implements the Levenberg-Marquardt algorithm. This algorithm is a popular choice for solving non-linear least squares problems, which are common in various applications such as computer vision, robotics, and machine learning.

The Levenberg-Marquardt algorithm is an iterative technique that interpolates between the Gauss-Newton algorithm and the method of gradient descent. It is particularly useful for optimizing problems where the solution is expected to be near the initial guess.

The Levenberg-Marquardt algorithm seeks to minimize a cost function $F(x)$ of the form:

F(x) = \frac{1}{2} \sum_{i=1}^{m} r_i(x)^2

(1)

where $r_i(x)$ are the residuals. The update rule for the algorithm is given by:

x_{k+1} = x_k - (J^T J + \lambda I)^{-1} J^T r

(2)

Here, $J$ is the Jacobian matrix of the residuals, λ is the damping parameter, and $I$ is the identity matrix.

Key features:

Non-linear Optimization: The class is designed to handle non-linear optimization problems efficiently.
Damping Mechanism: It incorporates a damping parameter to control the step size, balancing between the Gauss-Newton and gradient descent methods.
Iterative Improvement: The optimizer iteratively refines the solution, reducing the error at each step.

Key Methods¶

Please see the base class NonlinearOptimizer.

Parameters¶

The LevenbergMarquardtParams class defines parameters specific to this optimization algorithm:

Parameter	Type	Default Value	Description
lambdaInitial	double	1e-5	The initial Levenberg-Marquardt damping term
lambdaFactor	double	10.0	The amount by which to multiply or divide lambda when adjusting lambda
lambdaUpperBound	double	1e5	The maximum lambda to try before assuming the optimization has failed
lambdaLowerBound	double	0.0	The minimum lambda used in LM
verbosityLM	VerbosityLM	SILENT	The verbosity level for Levenberg-Marquardt
minModelFidelity	double	1e-3	Lower bound for the modelFidelity to accept the result of an LM iteration
logFile	std::string	“”	An optional CSV log file, with [iteration, time, error, lambda]
diagonalDamping	bool	false	If true, use diagonal of Hessian
useFixedLambdaFactor	bool	true	If true applies constant increase (or decrease) to lambda according to lambdaFactor
minDiagonal	double	1e-6	When using diagonal damping saturates the minimum diagonal entries
maxDiagonal	double	1e32	When using diagonal damping saturates the maximum diagonal entries

These parameters complement the standard optimization parameters inherited from NonlinearOptimizerParams, which include:

Maximum iterations
Relative and absolute error thresholds
Error function verbosity
Linear solver type

Usage Notes¶

The choice of the initial guess can significantly affect the convergence speed and the quality of the solution.
Proper tuning of the damping parameter λ is crucial for balancing the convergence rate and stability.
The optimizer is most effective when the residuals are approximately linear near the solution.

Files¶

GTSAM Docs

ISAM2

GTSAM Docs

LinearContainerFactor