ISTA-DASLab/DeepSeek-V4.1-Flash-1017-GSQ-RCO-Coder-GGUF
AI model · https://huggingface.co/ISTA-DASLab/DeepSeek-V4.1-Flash-1017-GSQ-RCO-Coder-GGUF
First seen by Oriel on .
AI summary
This experimental GGUF compresses DeepSeek-V4.1-Flash-1017 for coding, agentic tool use, long context, and vision by pruning half its routed experts while retaining most benchmark performance.
- The model removes 192 of 384 experts per layer and quantizes the remaining transformer weights using IQ2_XXS types, producing 72.12 GB at an effective 1.05 bits per parameter.
- It retains 77.00 SWE-bench Verified and 88.57 LiveCodeBench v6, or 94.8% and 98.7% of the base model scores respectively, measured at reasoning effort 50.
- RCO selects experts by minimizing full-model KL divergence on calibration data with exact per-layer budgets, rather than using independent heuristic rankings.
- The complete download totals 182.72 GB because unchanged Engram lookup tables contribute 110.60 GB and remain memory-mapped, while the transformer accounts for the savings.
- llama.cpp b11870 or newer supports the release; Strata Engine provides single-NVIDIA-GPU inference with an OpenAI-compatible localhost endpoint.
#gguf #model compression #mixture of experts #llama.cpp
Generated by gpt-5.6-sol. Check the original for details.
Metric trends
Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained
Likes
Total likes reported by the source.
- Hugging Face Models: 215, latest observation ; +107 over observed period since
Downloads (30d)
Rolling 30-day downloads. A change in this value is not new downloads during the observed period.
- Hugging Face Models: 309,064, latest observation ; +280,398 over observed period since
Aliases
| Source | External ID | URL |
|---|---|---|
| Hugging Face Models | ISTA-DASLab/DeepSeek-V4.1-Flash-1017-GSQ-RCO-Coder-GGUF | https://huggingface.co/ISTA-DASLab/DeepSeek-V4.1-Flash-1017-GSQ-RCO-Coder-GGUF Read discussion |