Oriel — AI & technology worth noticing, every day.

Scantist/GLM-5.3-SLIM-GGUF

AI model · https://huggingface.co/Scantist/GLM-5.3-SLIM-GGUF

First seen by Oriel on .

AI summary

Scantist/GLM-5.3-SLIM-GGUF is a 179.9 GB, 192-expert GLM-5.3 quantized model designed to fit on one NVIDIA B200, with reported scores of 92.7% on HumanEval and 84.8% on GPQA-Diamond.

  • Neural architecture search selected 192 of 256 routed experts per MoE layer; the removed experts were sliced from Unsloth's UD-IQ2_M GGUF without re-quantization or retraining.
  • The model has 563B parameters, is 24.6% smaller than the 238.6 GB source, and removes the MTP layer; MTP speculative decoding is unsupported.
  • On one B200, the weights use about 167 GiB, leaving room for roughly 100K f16 KV-cache tokens or 340K q4_0 tokens shared across requests.
  • Reported evaluation results include 84.8% on GPQA-Diamond (168/198), 92.7% on serial HumanEval (152/164), and 71.4% on a 570-question MMLU subset.
  • The deployment guide targets llama.cpp on a B200; smaller GPUs require multi-GPU splitting, which the authors did not test.

#llm inference #gguf #mixture of experts #quantization

Generated by gpt-5.6-sol. Check the original for details.

Metric trends

Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained

Likes

Total likes reported by the source.

  • Hugging Face Models: 1,419, latest observation ; +624 over observed period since

Downloads (30d)

Rolling 30-day downloads. A change in this value is not new downloads during the observed period.

  • Hugging Face Models: 20,376, latest observation ; +20,168 over observed period since

Aliases

SourceExternal IDURL
Hugging Face ModelsScantist/GLM-5.3-SLIM-GGUFhttps://huggingface.co/Scantist/GLM-5.3-SLIM-GGUF Read discussion