MinimaLabs/KLM-1.6
AI model · https://huggingface.co/MinimaLabs/KLM-1.6
First seen by Oriel on .
AI summary
KLM 1.6 is a 99M-parameter English base language model with extended learning-rate decay, reporting leading benchmark scores among models below 250M parameters.
- Its learning rate decays over 100B tokens rather than 20B, while the model otherwise incrementally updates KLM 1.5.
- KLM 1.6 scores 61.07% on ARC-Easy, 33.53% on ARC-Challenge, 45.28% on HellaSwag, 70.18% on PIQA, and 55.90% on ArithMark-3.
- Training used 70,000-step Token-Superposition Training, 80B tokens of stable next-token prediction, and 100B decay-stage tokens.
- The architecture has 24 layers combining Gated DeltaNet-2 and attention, 98,975,308 parameters, 2,048-token context, and learned n-gram memory.
- It is intended for fine-tuning rather than chat, runs with plain PyTorch, and has thin, frequently incorrect knowledge, English-only support, and no safety alignment.
#language models #gated deltanet #model training #fine-tuning
Generated by gpt-5.6-sol. Check the original for details.
Metric trends
Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained
Likes
Total likes reported by the source.
- Hugging Face Models: 140, latest observation ; +47 over observed period since
Downloads (30d)
Rolling 30-day downloads. A change in this value is not new downloads during the observed period.
- Hugging Face Models: 1,948, latest observation ; 0 over observed period since
Aliases
| Source | External ID | URL |
|---|---|---|
| Hugging Face Models | MinimaLabs/KLM-1.6 | https://huggingface.co/MinimaLabs/KLM-1.6 Read discussion |