Oriel — AI & technology worth noticing, every day.

orcarouter/DeepSeek-V4.1-Flash-1017-abliterated-GGUF

AI model · https://huggingface.co/orcarouter/DeepSeek-V4.1-Flash-1017-abliterated-GGUF

First seen by Oriel on .

AI summary

This repository provides mixed-precision GGUF builds for running the refusal-removed, 763B-parameter DeepSeek-V4.1-Flash-1017 locally, with its 197B-parameter Engram streamed from SSD.

  • The recommended Mix-256G build uses 235.8 GB memory, downloads 346.5 GB, achieves 93.74% next-token agreement, and raises perplexity 2.4% versus the lossless checkpoint.
  • Mix-128G uses 114.0 GB memory but is heavily degraded, with 78.50% next-token agreement and 54.7% higher perplexity.
  • Per-layer expert precision was selected through measured KLD and knapsack optimization, while the 110.6 GB IQ4_NL Engram remains memory-mapped on NVMe storage.
  • Serving requires llama.cpp PR #30817; measured generation reached approximately 70 tok/s on 2 x H200 for Mix-256G and 84 tok/s on one H200 for Mix-128G.
  • Safety alignment is substantially reduced, so the authors restrict intended use to legitimate research and require users to add moderation and abuse-prevention controls before deployment.

#deepseek #gguf #llama.cpp #model quantization

Generated by gpt-5.6-sol. Check the original for details.

Metric trends

Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained

Likes

Total likes reported by the source.

  • Hugging Face Models: 448, latest observation ; +332 over observed period since

Downloads (30d)

Rolling 30-day downloads. A change in this value is not new downloads during the observed period.

  • Hugging Face Models: 33,123, latest observation ; +33,120 over observed period since

Aliases

SourceExternal IDURL
Hugging Face Modelsorcarouter/DeepSeek-V4.1-Flash-1017-abliterated-GGUFhttps://huggingface.co/orcarouter/DeepSeek-V4.1-Flash-1017-abliterated-GGUF Read discussion