Oriel — AI & technology worth noticing, every day.

MinimaLabs/KLM-1.6

AI model · https://huggingface.co/MinimaLabs/KLM-1.6

First seen by Oriel on .

AI summary

KLM 1.6 is a 99M-parameter English base language model with extended learning-rate decay, reporting leading benchmark scores among models below 250M parameters.

  • Its learning rate decays over 100B tokens rather than 20B, while the model otherwise incrementally updates KLM 1.5.
  • KLM 1.6 scores 61.07% on ARC-Easy, 33.53% on ARC-Challenge, 45.28% on HellaSwag, 70.18% on PIQA, and 55.90% on ArithMark-3.
  • Training used 70,000-step Token-Superposition Training, 80B tokens of stable next-token prediction, and 100B decay-stage tokens.
  • The architecture has 24 layers combining Gated DeltaNet-2 and attention, 98,975,308 parameters, 2,048-token context, and learned n-gram memory.
  • It is intended for fine-tuning rather than chat, runs with plain PyTorch, and has thin, frequently incorrect knowledge, English-only support, and no safety alignment.

#language models #gated deltanet #model training #fine-tuning

Generated by gpt-5.6-sol. Check the original for details.

Metric trends

Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained

Likes

Total likes reported by the source.

  • Hugging Face Models: 140, latest observation ; +47 over observed period since

Downloads (30d)

Rolling 30-day downloads. A change in this value is not new downloads during the observed period.

  • Hugging Face Models: 1,948, latest observation ; 0 over observed period since

Aliases

SourceExternal IDURL
Hugging Face ModelsMinimaLabs/KLM-1.6https://huggingface.co/MinimaLabs/KLM-1.6 Read discussion