Oriel — AI & technology worth noticing, every day.

History Forcing: Distilling History-Dependent Continuations for Long-Horizon Video World Models

Paper · https://arxiv.org/abs/2611.02782

Few-step autoregressive video diffusion enables efficient generation, yet maintaining consistency over long rollouts remains challenging, especially when subjects or scenes reappear after going out of sight. Despite progress in fidelity and rollout robustness, existing distillation methods do not fully exploit richer historical information to constrain the student's current prediction. To address this gap, we introduce History Forcing, a unified on-policy distillation framework that strengthens historical supervision for few-step autoregressive video generators. The framework uses the residual between teacher predictions with and without informative historical context to decouple this supervision. Such a historical context can be obtained using earlier generated chunks with less accumulated drift or a fixed-view teacher rollout that keeps the scene visible while the student looks away. The residual is then weighted and added to the score or velocity prediction used for distillation, forming a corrected target. By aligning the student with this corrected target through distillation, History Forcing incorporates long-range historical constraints into autoregressive generation. Our method achieves state-of-the-art performance among 1.3B models, with a VBench quality score of 85.07 for 15s video generation and a VBench-Long aggregate score of 79.82 for 60s generation. For out-of-sight world modeling, evaluations further demonstrate more plausible scene evolution and more consistent states when the camera returns, with Evolution Quality improving from 2.35 to 2.70 on PlayWorld and Object Consistency from 58.43 to 62.18 on MemoBench.

First seen by Oriel on .

AI summary

History Forcing is an on-policy distillation framework that uses historical context to improve consistency in few-step autoregressive video generation, especially across long rollouts and out-of-sight scene returns.

  • Teacher predictions with and without informative history are differenced, then the weighted residual corrects the score or velocity target used for distillation.
  • Historical context comes from earlier generated chunks with less accumulated drift or a fixed-view teacher rollout that keeps the scene visible while the student looks away.
  • Among 1.3B models, it reaches a VBench quality score of 85.07 for 15-second videos and a VBench-Long aggregate score of 79.82 for 60-second videos.
  • Out-of-sight evaluations improve Evolution Quality from 2.35 to 2.70 on PlayWorld and Object Consistency from 58.43 to 62.18 on MemoBench.

#video generation #knowledge distillation #world models #autoregressive generation

Generated by gpt-5.6-sol. Check the original for details.

Metric trends

Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained

Points

Score reported by the source.

  • Hugging Face Papers: 18, latest observation ; +3 over observed period since

Comments

Comment count reported by the source.

  • Hugging Face Papers: 1, latest observation ; 0 over observed period since

Aliases

SourceExternal IDURL
Hugging Face Papers2611.02782https://arxiv.org/abs/2611.02782 Read discussion