Oriel — AI & technology worth noticing, every day.

ISTA-DASLab/DeepSeek-V4.1-Flash-1017-GSQ-RCO-Coder-GGUF

AI model · https://huggingface.co/ISTA-DASLab/DeepSeek-V4.1-Flash-1017-GSQ-RCO-Coder-GGUF

First seen by Oriel on .

AI summary

This experimental GGUF compresses DeepSeek-V4.1-Flash-1017 for coding, agentic tool use, long context, and vision by pruning half its routed experts while retaining most benchmark performance.

  • The model removes 192 of 384 experts per layer and quantizes the remaining transformer weights using IQ2_XXS types, producing 72.12 GB at an effective 1.05 bits per parameter.
  • It retains 77.00 SWE-bench Verified and 88.57 LiveCodeBench v6, or 94.8% and 98.7% of the base model scores respectively, measured at reasoning effort 50.
  • RCO selects experts by minimizing full-model KL divergence on calibration data with exact per-layer budgets, rather than using independent heuristic rankings.
  • The complete download totals 182.72 GB because unchanged Engram lookup tables contribute 110.60 GB and remain memory-mapped, while the transformer accounts for the savings.
  • llama.cpp b11870 or newer supports the release; Strata Engine provides single-NVIDIA-GPU inference with an OpenAI-compatible localhost endpoint.

#gguf #model compression #mixture of experts #llama.cpp

Generated by gpt-5.6-sol. Check the original for details.

Metric trends

Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained

Likes

Total likes reported by the source.

  • Hugging Face Models: 215, latest observation ; +107 over observed period since

Downloads (30d)

Rolling 30-day downloads. A change in this value is not new downloads during the observed period.

  • Hugging Face Models: 309,064, latest observation ; +280,398 over observed period since

Aliases

SourceExternal IDURL
Hugging Face ModelsISTA-DASLab/DeepSeek-V4.1-Flash-1017-GSQ-RCO-Coder-GGUFhttps://huggingface.co/ISTA-DASLab/DeepSeek-V4.1-Flash-1017-GSQ-RCO-Coder-GGUF Read discussion