MemT-1.0: Towards Memory-Native Evolution
Paper · https://arxiv.org/abs/2610.26825
Memory-native evolution requires making memory an intrinsic capability of foundation models, enabling them to continuously accumulate and utilize information. To this end, we introduce MemT-1.0, a new architecture of memory foundation models that internalize memory as part of model computation. Building on Metis, MemT-1.0 employs Gated DeltaNet-2 with a stable token-level memory update mechanism within each turn. We further introduce Mixture-of-Storage to scale native memory capacity. Following a deliberate backbone preparation, we develop a two-stage training pipeline consisting of continued pre-training and post-training, supported by 34B tokens of training data and efficient training and inference infrastructure. We release a model family comprising MemT-1.0-Nano and MemT-1.0-Flash, and conduct a comprehensive evaluation of memory and general capabilities. The results show that they outperform the parametric memory baselines on average and remain competitive with retrieval-augmented methods, while largely preserving general capabilities. At a history length of 128K tokens, MemT-1.0-Nano achieves approximately 1.86times end-to-end inference speedup and a 66times reduction in persistent storage relative to full context KV cache. By internalizing retrieved procedural experience, MemT-1.0 improves task success on agentic tasks, providing a practical step toward memory-native evolution.
First seen by Oriel on .
AI summary
MemT-1.0 is a family of memory foundation models that integrates persistent memory into model computation and improves memory-task performance while largely preserving general capabilities.
- The architecture builds on Metis and uses Gated DeltaNet-2 for stable token-level memory updates within each turn, with Mixture-of-Storage to scale capacity.
- Training uses a two-stage pipeline of continued pre-training and post-training, supported by 34B tokens of data and dedicated training and inference infrastructure.
- MemT-1.0-Nano and MemT-1.0-Flash outperform parametric memory baselines on average and remain competitive with retrieval-augmented methods.
- At 128K tokens of history, MemT-1.0-Nano delivers approximately 1.86× end-to-end inference speedup and 66× less persistent storage than a full context KV cache.
- Internalizing retrieved procedural experience improves task success on agentic tasks.
#foundation models #memory systems #gated deltanet-2 #llm agents
Generated by gpt-5.6-sol. Check the original for details.
Metric trends
Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained
Points
Score reported by the source.
- Hugging Face Papers: 183, latest observation ; +38 over observed period since
Comments
Comment count reported by the source.
- Hugging Face Papers: 3, latest observation ; +1 over observed period since
Aliases
| Source | External ID | URL |
|---|---|---|
| Hugging Face Papers | 2610.26825 | https://arxiv.org/abs/2610.26825 Read discussion |