GNC Robust Optimization with the CUDA SFM Optimizer

Created by Codex.

Use the NVIDIA CUDA structure-from-motion (SfM) optimizer as the inner solver for robust Graduated Non-Convexity (GNC) bundle adjustment.

Mathematical idea¶

Graduated Non-Convexity (GNC) solves a sequence of weighted bundle-adjustment problems,

\min_x \sum_i w_i(\mu)\,\lVert r_i(x)\rVert_{\Sigma_i}^{2},

(1)

while a continuation parameter $\mu$ gradually changes the weights from an easy surrogate toward the requested robust loss. TLS means truncated least squares, and GM denotes the Geman–McClure loss. Each inner Levenberg–Marquardt (LM) solve runs through NVIDIA’s Compute Unified Device Architecture (CUDA) on the graphics processing unit (GPU); the central processing unit (CPU) updates the GNC schedule and weights.

import gtsam

cuda = getattr(gtsam, "cuda", None)
print("CUDA SFM bindings available:", cuda is not None)

CUDA SFM bindings available: False

Overview¶

Graduated Non-Convexity (GncOptimizer) makes optimization robust to outlier measurements by iteratively re-weighting factors: it solves a sequence of weighted least-squares problems, gradually sharpening a robust cost until outliers receive weight ~0 and inliers weight ~1.

Each GNC outer iteration re-optimizes the weighted graph with an inner nonlinear optimizer. Because bundle adjustment problems are large and GNC solves many of them, the inner solve dominates the runtime — making it the natural place for GPU acceleration.

SfmLevenbergMarquardtParams plugs into GncParams as a drop-in inner-solver configuration:

GncOptimizer<GncParams<SfmLevenbergMarquardtParams>>

runs standard CPU GNC logic (weight updates, mu schedule, convergence checks) with every inner LM solve executed on the GPU.

Usage (C++)¶

#include <gtsam/nonlinear/GncOptimizer.h>
#include <gtsam/sfm/cuda/SfmLevenbergMarquardt.h>

using namespace gtsam;
using namespace gtsam::cuda;

// BAL-style graph with outlier-contaminated projection measurements
NonlinearFactorGraph graph = ...;   // GeneralSFMFactor<PinholeCamera<Cal3Bundler>, Point3>
Values initial = ...;

// Inner-solver params: the CUDA SFM LM optimizer
SfmLevenbergMarquardtParams lmParams =
    SfmLevenbergMarquardtParams::legacyDefaults();

// GNC params wrap the inner params; choose the robust loss
GncParams<SfmLevenbergMarquardtParams> gncParams{lmParams};
gncParams.setLossType(GncLossType::TLS);   // or GncLossType::GM

GncOptimizer<GncParams<SfmLevenbergMarquardtParams>> gnc(
    graph, initial, gncParams);
const Values result = gnc.optimize();

// Per-factor weights after convergence: ~1 for inliers, ~0 for outliers
const Vector& weights = gnc.getWeights();
for (size_t slot = 0; slot < graph.size(); ++slot) {
  if (weights[slot] < 0.5) std::cout << "factor " << slot << " is an outlier\n";
}

Everything else about GncOptimizer works as documented for the CPU version: setLossType, known-inlier/outlier hints, mu-update and iteration controls on GncParams, and getWeights() for the final classification.

Loss types¶

Loss	Behavior
`GncLossType::TLS` (truncated least squares)	Hard classification: converged weights are essentially binary (0/1). Outlier factors are fully rejected — their weights reach exactly 0
`GncLossType::GM` (Geman-McClure)	Soft down-weighting: outliers get small but nonzero weights

Note on TLS: weights of exactly zero scale a factor’s information matrix to zero. The CUDA backend accepts zero sqrt-information blocks specifically to support this — rejected factors contribute nothing to the normal equations, as intended.

Correctness¶

The CUDA-inner-solver GNC is validated against CPU GNC (GncOptimizer<GncParams<LevenbergMarquardtParams>>) in gtsam/sfm/tests/testCudaSfm.cpp:

TLS and GM classification parity: on a synthetic BAL problem with injected outliers, CPU and CUDA GNC classify every factor identically (inlier weights > 0.95, outlier weights < 0.05 for TLS).
Solution quality: the inlier-only reprojection error of the CUDA result matches the CPU result and reduces the initial error to < 1e-3.
A zero-information weighted-graph regression test covers the TLS weight-zero edge case.

Benchmarking¶

timing/sfm_ba/timeCudaSFMBAL.cpp has a GNC benchmark mode: it injects a configurable fraction of outlier measurements into a BAL problem (unbiased sampling, see timing/sfm_ba/GncOutlierSampling.h) and compares CPU GNC vs GNC-with-CUDA-inner-solver on wall time and classification accuracy.

Notes¶

GNC re-optimizes the same graph topology repeatedly with different weights — the CUDA optimizer’s setup (conversion, device upload, solver construction) is re-run per outer iteration. Amortizing this setup across GNC iterations is a known future optimization.
Supported problem class is the same as the inner optimizer’s: BAL-style GeneralSFMFactor<PinholeCamera<Cal3Bundler>, Point3> graphs — see SfmLevenbergMarquardtOptimizer.