Oriel — AI & technology worth noticing, every day.

TTB: Test-time Baking for Efficient Rendering of Decoder-only View Synthesis Models

Paper · https://arxiv.org/abs/2611.17331

Decoder-only Large View Synthesis Models (LVSMs) with KV-cache have recently achieved state-of-the-art quality by regressing novel views from neural networks without reconstructing 3D geometry, while Test-Time Training (TTT) layers save computation but sacrifice quality. We propose Test-Time Baking (TTB), a strategy that learns to bake global scene information from cross-view attention into a lightweight MLP, enabling fast and high-fidelity novel-view decoding. The first key is to repurpose the fast-weight MLP of TTT to learn the cross-view attention mapping explicitly from its input-output pairs, rather than replacing attention entirely, achieving the same rendering efficiency as TTT while delivering substantially higher quality. The second key addresses a modality misalignment between context prefilling and novel-view rendering: we disentangle context tokens into camera-only context rendering tokens to serve as TTB input, and context ground-truth tokens with image content to serve as the TTB mapping target, ensuring the baked MLPs are relieved of the modality generalization challenge between context and novel-view processing. Our method achieves up to **23x** rendering speedup with quality on par with the state-of-the-art across several benchmarks. Project page: https://test-time-baking.github.io

First seen by Oriel on .

AI summary

TTB accelerates decoder-only view synthesis by baking cross-view attention into lightweight MLPs, achieving up to 23x faster rendering while maintaining state-of-the-art quality.

  • TTB repurposes the fast-weight MLP from Test-Time Training to learn cross-view attention mappings from input-output pairs instead of replacing attention.
  • The method separates context tokens into camera-only rendering tokens and ground-truth image-content tokens, resolving modality misalignment during baking.
  • Baked MLPs provide the rendering efficiency of TTT while delivering substantially higher quality than replacing attention with TTT layers.
  • Across several benchmarks, TTB achieves up to 23x rendering speedup with quality on par with the state-of-the-art.
  • The approach targets decoder-only Large View Synthesis Models using KV-cache for novel-view decoding without reconstructing 3D geometry.

#view synthesis #test-time training #neural rendering

Generated by gpt-5.6-sol. Check the original for details.

Metric trends

Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained

Points

Score reported by the source.

  • Hugging Face Papers: 24, latest observation ; +5 over observed period since

Comments

Comment count reported by the source.

  • Hugging Face Papers: 3, latest observation ; +2 over observed period since

Aliases

SourceExternal IDURL
Hugging Face Papers2611.17331https://arxiv.org/abs/2611.17331 Read discussion