Show HN: DeepSeek-Coder-V2-Lite Under 5GB: 65 TPS 100% In-VRAM on 6GB GPU
AI model · https://huggingface.co/AmSach/DeepSeek-Coder-V2-Lite-Instruct-MAPLE-Sub5GB-GGUF
First seen by Oriel on .
AI summary
This Hugging Face page presents a MAPLE-pruned DeepSeek-Coder-V2-Lite-Instruct GGUF that compresses 15.7B parameters into a 4.50 GB container for full-GPU inference on 6GB laptop GPUs.
- MAPLE retains 54 of 64 routed experts per MoE layer, preserves six-way top-K routing, and leaves two shared invariant experts unchanged.
- The model reports 65.6–76.3 generation tokens per second on an RTX 4050 Laptop GPU, compared with 4.5–10.0 tokens per second for hybrid offloading.
- Benchmarks covered C++20 lock-free synchronization, Python directed hypergraph dynamic programming, and a Redis Lua token bucket, completing in 18.04–19.59 seconds.
- The page provides llama.cpp and Ollama deployment instructions, requiring the model’s control-token prompt format and recommending 0.2 temperature, 0.95 top-p, and 99 GPU layers.
#deepseek #gguf #llama.cpp #model pruning
Generated by gpt-5.6-sol. Check the original for details.
Metric trends
Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained
Points
Score reported by the source.
- Hacker News: 3, latest observation ; 0 over observed period since
Comments
Comment count reported by the source.
- Hacker News: 1, latest observation ; +1 over observed period since
Aliases
| Source | External ID | URL |
|---|---|---|
| Hacker News | 50463281 | https://huggingface.co/AmSach/DeepSeek-Coder-V2-Lite-Instruct-MAPLE-Sub5GB-GGUF Read discussion |