Oriel — AI & technology worth noticing, every day.

F2G-Pose: Geometry-Aware Foundation Feature Lifting for Direct RGB-D Category-Level Object Pose Estimation

Paper · https://arxiv.org/abs/2610.31623

This paper studies category-level RGB-D object pose estimation, which recovers an object's 3D rotation and translation without instance-specific CAD models or reference views. This task is highly ambiguous due to intra-class variations, symmetries, and partial observations. Current solutions struggle to balance reliability and efficiency: methods based on intermediate structures are prone to error propagation, while diffusion models suffer from high inference latency and complex post-hoc candidate extraction. To address this, we present F2G-Pose, a real-time direct, single-pass framework that predicts category-level pose, metric size, and camera-frame completed shape from a single RGB-D observation in one feed-forward pass. F2G-Pose lifts dense visual foundation features to partial point clouds, encodes local geometry with DGCNN tokenization, and integrates object-level structure with a Mixture-of-Experts Transformer. Joint size prediction and camera-frame shape completion encourage metric consistency through dense geometric supervision. With its task-specific network trained only on pose-annotated synthetic SOPE data and its frozen RADIO backbone externally pretrained, F2G-Pose achieves state-of-the-art accuracy on SOPE and ROPE while running at 28 FPS for pose-only inference, and further demonstrates strong camera-frame geometry on HouseCat6D and real-object transfer on HANDAL. Analyses of latency, robustness, shape completion, and GenPose++ candidate behavior further demonstrate the efficiency and reliability of F2G-Pose, with real-world manipulation experiments demonstrating deployment potential under segmented RGB-D inputs. The project page, code, and pretrained models are made available at https://jrryzh.github.io/F2G-Pose/.

First seen by Oriel on .

AI summary

F2G-Pose is a real-time, single-pass framework for category-level RGB-D object pose estimation, jointly predicting pose, metric size, and completed shape from one observation.

  • It lifts dense visual foundation features onto partial point clouds, encodes local geometry with DGCNN tokenization, and models object structure using a Mixture-of-Experts Transformer.
  • Joint size prediction and camera-frame shape completion provide dense geometric supervision to encourage metric consistency.
  • The task-specific network is trained only on pose-annotated synthetic SOPE data, while the RADIO backbone is frozen and externally pretrained.
  • The paper reports state-of-the-art accuracy on SOPE and ROPE and 28 FPS for pose-only inference, with additional evaluations on HouseCat6D, HANDAL, and real-world manipulation.

#rgb-d #6d pose estimation #point clouds #foundation models

Generated by gpt-5.6-sol. Check the original for details.

Metric trends

Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained

Points

Score reported by the source.

  • Hugging Face Papers: 22, latest observation ; +17 over observed period since

Comments

Comment count reported by the source.

  • Hugging Face Papers: 2, latest observation ; +1 over observed period since

Aliases

SourceExternal IDURL
Hugging Face Papers2610.31623https://arxiv.org/abs/2610.31623 Read discussion