taobao-mnn/MNN-Pocket-0.3B
AI model · https://huggingface.co/taobao-mnn/MNN-Pocket-0.3B
First seen by Oriel on .
AI summary
MNN-Pocket-0.3B is a 300M-parameter bilingual instruction model designed for on-device use, with a 173 MB INT4 build and reported decode speeds up to 466 tokens/s.
- The model supports up to 32K tokens in deployment and targets on-device assistants, data extraction, and lightweight agentic pipelines.
- Its INT4 W4A16 QAT build averages 49.59 across 11 Chinese and English benchmarks, 0.41 points below the unquantized model.
- On iPad Pro M5, the INT4 build decodes at 466 tokens/s with Metal and 231 tokens/s on CPU; Snapdragon 8 Gen 5 results vary by backend.
- It runs with MNN, Transformers, and llama.cpp, and is released under Apache License 2.0.
- The page cautions that sub-billion-parameter models like this are not recommended for reasoning-heavy tasks such as advanced mathematics or long-horizon code generation.
#language models #on-device inference #model quantization #mnn
Generated by gpt-5.6-sol. Check the original for details.
Metric trends
Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained
Likes
Total likes reported by the source.
- Hugging Face Models: 364, latest observation ; +270 over observed period since
Downloads (30d)
Rolling 30-day downloads. A change in this value is not new downloads during the observed period.
- Hugging Face Models: 5,009, latest observation ; +2,621 over observed period since
Aliases
| Source | External ID | URL |
|---|---|---|
| Hugging Face Models | taobao-mnn/MNN-Pocket-0.3B | https://huggingface.co/taobao-mnn/MNN-Pocket-0.3B Read discussion |