Scantist/GLM-5.3-SLIM-GGUF
AI model · https://huggingface.co/Scantist/GLM-5.3-SLIM-GGUF
First seen by Oriel on .
AI summary
Scantist/GLM-5.3-SLIM-GGUF is a 179.9 GB, 192-expert GLM-5.3 quantized model designed to fit on one NVIDIA B200, with reported scores of 92.7% on HumanEval and 84.8% on GPQA-Diamond.
- Neural architecture search selected 192 of 256 routed experts per MoE layer; the removed experts were sliced from Unsloth's UD-IQ2_M GGUF without re-quantization or retraining.
- The model has 563B parameters, is 24.6% smaller than the 238.6 GB source, and removes the MTP layer; MTP speculative decoding is unsupported.
- On one B200, the weights use about 167 GiB, leaving room for roughly 100K f16 KV-cache tokens or 340K q4_0 tokens shared across requests.
- Reported evaluation results include 84.8% on GPQA-Diamond (168/198), 92.7% on serial HumanEval (152/164), and 71.4% on a 570-question MMLU subset.
- The deployment guide targets llama.cpp on a B200; smaller GPUs require multi-GPU splitting, which the authors did not test.
#llm inference #gguf #mixture of experts #quantization
Generated by gpt-5.6-sol. Check the original for details.
Metric trends
Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained
Likes
Total likes reported by the source.
- Hugging Face Models: 1,419, latest observation ; +624 over observed period since
Downloads (30d)
Rolling 30-day downloads. A change in this value is not new downloads during the observed period.
- Hugging Face Models: 20,376, latest observation ; +20,168 over observed period since
Aliases
| Source | External ID | URL |
|---|---|---|
| Hugging Face Models | Scantist/GLM-5.3-SLIM-GGUF | https://huggingface.co/Scantist/GLM-5.3-SLIM-GGUF Read discussion |