SfmTrack

Created by Codex.

Extend a two-dimensional structure-from-motion (SfM) feature track with a reconstructed three-dimensional point and optional red–green–blue (RGB) color.

Authors: Frank Dellaert, et al. (see THANKS for the full author list)

See LICENSE for the license information

Mathematical idea¶

A reconstructed landmark $P_j$ and its observations form the track

\mathcal T_j=\{(i,z_{ij})\mid z_{ij}\approx\pi(C_i,P_j)\}.

(1)

The point $P_j$ participates in bundle adjustment, while its red–green–blue (RGB) color is optional display metadata and does not enter the objective.

try:
    import google.colab
    %pip install --quiet gtsam-develop
except ImportError:
    pass

import gtsam
import numpy as np

from gtsam import symbol_shorthand

C = symbol_shorthand.C
K = symbol_shorthand.K
P = symbol_shorthand.P
S = symbol_shorthand.S
X = symbol_shorthand.X

Relationship to `SfmTrack2d`¶

SfmTrack inherits the observation list from SfmTrack2d and adds the current landmark estimate p. The color fields r, g, and b are floating-point metadata and do not affect optimization.

track = gtsam.SfmTrack(np.array([0.5, -0.2, 4.0]), 0.8, 0.4, 0.1)
track.addMeasurement(0, np.array([382.5, 215.0]))
track.addMeasurement(1, np.array([350.1, 214.8]))

print("point:", track.point3())
print("RGB:", (track.r, track.g, track.b))
print("observed by cameras:", track.indexVector())

point: [ 0.5 -0.2  4. ]
RGB: (0.800000011920929, 0.4000000059604645, 0.10000000149011612)
observed by cameras: [0 1]

Practical notes¶

SfmData.sfmFactorGraph() uses the 2D observations to construct reprojection factors, while initialization helpers use p as the starting landmark value. The class supports GTSAM serialization for datasets and checkpoints.

Source¶

SfmTrack.h

Mathematical idea¶

Relationship to SfmTrack2d¶

Practical notes¶

Source¶

Relationship to `SfmTrack2d`¶