SfmTrack

Created by Codex.

Extend a two-dimensional structure-from-motion (SfM) feature track with a reconstructed three-dimensional point and optional red–green–blue (RGB) color.

Authors: Frank Dellaert, et al. (see THANKS for the full author list)

See LICENSE for the license information

try:
    import google.colab
    %pip install --quiet gtsam-develop
except ImportError:
    pass

import gtsam
import numpy as np

from gtsam import symbol_shorthand

C = symbol_shorthand.C
K = symbol_shorthand.K
P = symbol_shorthand.P
S = symbol_shorthand.S
X = symbol_shorthand.X

Mathematical idea¶

A reconstructed landmark $P_j$ and its observations form the track

\mathcal T_j=\{(i,z_{ij})\mid z_{ij}\approx\pi(C_i,P_j)\}.

(1)

The point $P_j$ participates in bundle adjustment, while its red–green–blue (RGB) color is optional display metadata and does not enter the objective.

Relationship to `SfmTrack2d`¶

SfmTrack2d stores image measurements and camera indices before a landmark has been reconstructed. SfmTrack inherits that observation list and adds the current landmark estimate p. The color fields r, g, and b are floating-point metadata and do not affect optimization.

measurements = [
    (0, np.array([382.5, 215.0])),
    (1, np.array([350.1, 214.8])),
]
observations = gtsam.SfmTrack2d(measurements)
assert observations.numberMeasurements() == 2
print("2D measurements:\n", observations.measurementMatrix())

track = gtsam.SfmTrack(np.array([0.5, -0.2, 4.0]), 0.8, 0.4, 0.1)
track.addMeasurement(0, np.array([382.5, 215.0]))
track.addMeasurement(1, np.array([350.1, 214.8]))

print("point:", track.point3())
print("RGB:", (track.r, track.g, track.b))
print("observed by cameras:", track.indexVector())

Practical notes¶

SfmData.sfmFactorGraph() uses the 2D observations to construct reprojection factors, while initialization helpers use p as the starting landmark value. The class supports GTSAM serialization for datasets and checkpoints.

Source¶

SfmTrack.h

Mathematical idea¶

Relationship to SfmTrack2d¶

Practical notes¶

Source¶

Relationship to `SfmTrack2d`¶