Oriel — AI & technology worth noticing, every day.

taobao-mnn/MNN-Pocket-0.3B

AI model · https://huggingface.co/taobao-mnn/MNN-Pocket-0.3B

First seen by Oriel on .

AI summary

MNN-Pocket-0.3B is a 300M-parameter bilingual instruction model designed for on-device use, with a 173 MB INT4 build and reported decode speeds up to 466 tokens/s.

  • The model supports up to 32K tokens in deployment and targets on-device assistants, data extraction, and lightweight agentic pipelines.
  • Its INT4 W4A16 QAT build averages 49.59 across 11 Chinese and English benchmarks, 0.41 points below the unquantized model.
  • On iPad Pro M5, the INT4 build decodes at 466 tokens/s with Metal and 231 tokens/s on CPU; Snapdragon 8 Gen 5 results vary by backend.
  • It runs with MNN, Transformers, and llama.cpp, and is released under Apache License 2.0.
  • The page cautions that sub-billion-parameter models like this are not recommended for reasoning-heavy tasks such as advanced mathematics or long-horizon code generation.

#language models #on-device inference #model quantization #mnn

Generated by gpt-5.6-sol. Check the original for details.

Metric trends

Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained

Likes

Total likes reported by the source.

  • Hugging Face Models: 364, latest observation ; +270 over observed period since

Downloads (30d)

Rolling 30-day downloads. A change in this value is not new downloads during the observed period.

  • Hugging Face Models: 5,009, latest observation ; +2,621 over observed period since

Aliases

SourceExternal IDURL
Hugging Face Modelstaobao-mnn/MNN-Pocket-0.3Bhttps://huggingface.co/taobao-mnn/MNN-Pocket-0.3B Read discussion