Oriel — AI & technology worth noticing, every day.

burner22/numbat

AI model · https://huggingface.co/burner22/numbat

First seen by Oriel on .

AI summary

The Hugging Face page claims to host numbat, a 14-trillion-parameter Anthropic checkpoint with a hybrid GDN/MLA architecture, MoE layers, NVFP4 weights, and a reference loader.

  • The checkpoint reportedly has 114 layers, 1,224 experts with top-48 routing plus one shared expert, and a 131,072-token byte-level BPE tokenizer.
  • Its 772 safetensors shards contain approximately 7.72 TB of tensor data, with each shard under 10.6 GB.
  • configuration_numbat.py and modeling_numbat.py provide a complete reference implementation that supports strict loading and AutoModelForCausalLM.from_pretrained(..., trust_remote_code=True).
  • The model includes Multi-Token Prediction, with NumbatForCausalLM.forward running the trunk and NumbatForCausalLM.draft running the MTP layer.
  • The page describes this as a pretraining checkpoint at step 1,830,000 of approximately 1.96 million, without post-training or RLHF.

#mixture of experts #nvfp4 #gated deltanet #multi-token prediction

Generated by gpt-5.6-sol. Check the original for details.

Metric trends

Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained

Points

Score reported by the source.

  • Hacker News: 28, latest observation

Likes

Total likes reported by the source.

  • Hugging Face Models: 396, latest observation ; +278 over observed period since

Downloads (30d)

Rolling 30-day downloads. A change in this value is not new downloads during the observed period.

  • Hugging Face Models: 10,688, latest observation ; +4,842 over observed period since

Comments

Comment count reported by the source.

  • Hacker News: 9, latest observation

Aliases

SourceExternal IDURL
Hugging Face Modelsburner22/numbathttps://huggingface.co/burner22/numbat Read discussion
Hacker News50327240https://huggingface.co/burner22/numbat Read discussion