Oriel — AI & technology worth noticing, every day.

Proving and Improving the Platonic Representation Hypothesis with JEPAs

Paper · https://arxiv.org/abs/2610.39302

Whether models trained on different modalities converge to the same representation of the world is central to AI. The Platonic Representation Hypothesis (PRH) conjectures that they do as they scale, but without a proof it offers neither a guarantee nor a recipe. We turn the hypothesis into a theorem. First, we prove a Platonic Representation Theorem: Joint-Embedding Predictive Architectures (JEPAs) trained separately on any number of modalities learn the same representation up to one rotation whenever their positive pairs are two observations of one scene--good views--and their misalignment is bounded otherwise. Second, we show that next-token prediction is a JEPA with bad views, parts of a single document, which explains why language models align with vision only through scale. Third, we validate the theory from 145k captions to the texts of 940M web images, and uncover a scaling law of alignment that predicts unseen runs within seed noise. Good views need 42 times fewer training samples than next-token prediction on the same corpus to reach the same alignment, and a 109M-parameter text JEPA aligns with DINOv2 and CLIP's image encoders better than every language model we evaluated, up to 72B parameters, and than CLIP's text encoder: next-token prediction is a remarkably inefficient route to platonic representations. As a showcase, one rotation turns separately trained encoders into an image captioner (CIDEr .838 on COCO). Even that rotation needs no pair: on the latest pair-free benchmark, our text JEPA beats every zero-shot accuracy reported with language models up to 8B parameters, e.g., 64\% against at most 56\% top-5 on ImageNet-100. Positive views, and not scale alone, are thus the lever to build platonic representations.

First seen by Oriel on .

AI summary

The paper proves conditions under which separately trained JEPAs converge to aligned representations and argues that positive-view quality is more effective than scale for cross-modal alignment.

  • The theorem guarantees representations match up to one rotation when positive pairs are observations of the same scene, with bounded misalignment otherwise.
  • Next-token prediction is characterized as a JEPA using bad views because its paired parts come from a single document.
  • Good views require 42 times fewer samples than next-token prediction to achieve equivalent alignment on the same corpus.
  • A 109M-parameter text JEPA outperforms evaluated language models up to 72B parameters and CLIP's text encoder when aligning with DINOv2 and CLIP image encoders.
  • A single rotation enables image captioning at CIDEr .838 on COCO, while pair-free evaluation reaches 64% top-5 accuracy on ImageNet-100 versus at most 56%.

#jepa #representation learning #multimodal ai #scaling laws

Generated by gpt-5.6-sol. Check the original for details.

Metric trends

Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained

Points

Score reported by the source.

  • Hugging Face Papers: 76, latest observation ; +22 over observed period since
  • Hacker News: 167, latest observation ; +154 over observed period since

Comments

Comment count reported by the source.

  • Hugging Face Papers: 4, latest observation ; 0 over observed period since
  • Hacker News: 84, latest observation ; +84 over observed period since

Aliases

SourceExternal IDURL
Hugging Face Papers2610.39302https://arxiv.org/abs/2610.39302 Read discussion
Hacker News50325454https://arxiv.org/abs/2610.39302 Read discussion