Oriel — AI & technology worth noticing, every day.

Reframe: Observation-Aligned Multi-Teacher On-Policy Distillation for Agentic Visual Reasoning

Paper · https://arxiv.org/abs/2611.12886

Agentic reinforcement learning (RL) and multi-teacher on-policy distillation (MOPD) offer complementary routes to post-training. For visual agents, however, tool use presents two related challenges. Models may overlook useful tool-returned observations, while poorly grounded actions can limit the evidence available for subsequent distillation. We introduce Reframe, a post-training framework that treats the observation state, rather than only the output distribution, as an object of alignment. First, Ground-Drop-GRPO (GD-GRPO) trains spatial and temporal specialists with evidence-aware rewards and stochastic removal of the original visual input, encouraging them to acquire grounded evidence and use the returned observations. Second, Observation-Gated-MOPD (OG-MOPD) replaces deficient tool actions with canonical ones mid-rollout and resumes distillation from the repaired observation. The two stages complement each other, as learning to rely on tool-returned observations can help the policy benefit from observation repair. Neither stage requires a task-specific SFT cold start. Extensive experiments over 14 image and video benchmarks demonstrate that Reframe outperforms both Mix-RL and vanilla MOPD. Ablation results show that GD-GRPO increases observation dependence and tool gain compared with outcome-based GRPO.

First seen by Oriel on .

AI summary

Reframe is a two-stage post-training framework for visual agents that aligns both tool actions and the observations used during multi-teacher on-policy distillation.

  • Ground-Drop-GRPO trains spatial and temporal specialists with evidence-aware rewards and stochastic removal of the original visual input.
  • Observation-Gated-MOPD repairs deficient tool actions with canonical ones mid-rollout, then resumes distillation from the repaired observation.
  • Neither stage requires a task-specific SFT cold start.
  • Across 14 image and video benchmarks, Reframe outperforms Mix-RL and vanilla MOPD; ablations show increased observation dependence and tool gain over outcome-based GRPO.

#agentic reinforcement learning #visual reasoning #on-policy distillation #multimodal agents

Generated by gpt-5.6-sol. Check the original for details.

Metric trends

Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained

Points

Score reported by the source.

  • Hugging Face Papers: 39, latest observation ; +2 over observed period since

Comments

Comment count reported by the source.

  • Hugging Face Papers: 3, latest observation ; +1 over observed period since

Aliases

SourceExternal IDURL
Hugging Face Papers2611.12886https://arxiv.org/abs/2611.12886 Read discussion