burner22/numbat
AI model · https://huggingface.co/burner22/numbat
First seen by Oriel on .
AI summary
The Hugging Face page claims to host numbat, a 14-trillion-parameter Anthropic checkpoint with a hybrid GDN/MLA architecture, MoE layers, NVFP4 weights, and a reference loader.
- The checkpoint reportedly has 114 layers, 1,224 experts with top-48 routing plus one shared expert, and a 131,072-token byte-level BPE tokenizer.
- Its 772 safetensors shards contain approximately 7.72 TB of tensor data, with each shard under 10.6 GB.
- configuration_numbat.py and modeling_numbat.py provide a complete reference implementation that supports strict loading and AutoModelForCausalLM.from_pretrained(..., trust_remote_code=True).
- The model includes Multi-Token Prediction, with NumbatForCausalLM.forward running the trunk and NumbatForCausalLM.draft running the MTP layer.
- The page describes this as a pretraining checkpoint at step 1,830,000 of approximately 1.96 million, without post-training or RLHF.
#mixture of experts #nvfp4 #gated deltanet #multi-token prediction
Generated by gpt-5.6-sol. Check the original for details.
Metric trends
Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained
Points
Score reported by the source.
- Hacker News: 28, latest observation
Likes
Total likes reported by the source.
- Hugging Face Models: 396, latest observation ; +278 over observed period since
Downloads (30d)
Rolling 30-day downloads. A change in this value is not new downloads during the observed period.
- Hugging Face Models: 10,688, latest observation ; +4,842 over observed period since
Comments
Comment count reported by the source.
- Hacker News: 9, latest observation
Aliases
| Source | External ID | URL |
|---|---|---|
| Hugging Face Models | burner22/numbat | https://huggingface.co/burner22/numbat Read discussion |
| Hacker News | 50327240 | https://huggingface.co/burner22/numbat Read discussion |