ReLift: Dense 3D Tracking as Lifted Trajectory Repair
Paper · https://arxiv.org/abs/2610.42136
Dense 3D tracking from monocular video requires establishing point correspondences over time, estimating their depth, and inferring their trajectories during occlusion. Existing methods typically learn these tasks jointly, a demanding objective given the scarcity of 4D supervision. We show that strong pretrained 2D trackers and 3D geometry models provide an effective foundation: lifted trajectories, with occluded segments simply interpolated, already rival dedicated 3D trackers using the same geometry. However, lifted trajectories inherit correspondence and depth errors, and under occlusion, their depth comes from the occluder rather than the tracked point. We introduce ReLift, an efficient feed-forward framework that recasts dense 3D tracking as geometric trajectory repair and completion. At its core, a trajectory corrector alternates attention along and across trajectories to preserve reliable observations, repair corrupted positions, and complete missing motion. Using only geometric inputs allows us to train directly on 3D trajectories with simulated lifting errors, decoupling training from images and from any specific tracker or geometry model. Concentrating 4D supervision on trajectory refinement yields a reusable model that consistently improves lifted trajectories across trackers and geometry providers without retraining, so it benefits directly from future advances in either component. On WorldTrack, ReLift achieves state-of-the-art mean accuracy, improving on the strongest baselines by 2.1 APD on all points and 3.0 on dynamic points, with faster configurations remaining competitive with leading 3D trackers while running faster than 4RC. Visit our supplementry page for more examples and code.
First seen by Oriel on .
AI summary
ReLift treats dense 3D tracking as geometric repair of lifted 2D trajectories, improving accuracy across trackers and geometry providers without retraining.
- Lifted trajectories from pretrained 2D trackers and 3D geometry models can rival dedicated 3D trackers, but inherit correspondence, depth, and occlusion errors.
- A trajectory corrector alternates attention along and across trajectories to preserve reliable observations, repair corrupted positions, and complete missing motion.
- Training uses only geometric inputs and simulated lifting errors, decoupling the model from images and any specific tracker or geometry provider.
- On WorldTrack, ReLift improves mean accuracy by 2.1 APD on all points and 3.0 APD on dynamic points, while faster configurations remain competitive and beat 4RC speed.
- The reusable model benefits from future improvements to either the tracking or geometry component without requiring retraining.
#3d tracking #computer vision #trajectory repair #3d geometry
Generated by gpt-5.6-sol. Check the original for details.
Metric trends
Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained
Points
Score reported by the source.
- Hugging Face Papers: 27, latest observation ; +6 over observed period since
Comments
Comment count reported by the source.
- Hugging Face Papers: 12, latest observation ; +8 over observed period since
Aliases
| Source | External ID | URL |
|---|---|---|
| Hugging Face Papers | 2610.42136 | https://arxiv.org/abs/2610.42136 Read discussion |