Oriel — AI & technology worth noticing, every day.

Show HN: DeepSeek-Coder-V2-Lite Under 5GB: 65 TPS 100% In-VRAM on 6GB GPU

AI model · https://huggingface.co/AmSach/DeepSeek-Coder-V2-Lite-Instruct-MAPLE-Sub5GB-GGUF

First seen by Oriel on .

AI summary

This Hugging Face page presents a MAPLE-pruned DeepSeek-Coder-V2-Lite-Instruct GGUF that compresses 15.7B parameters into a 4.50 GB container for full-GPU inference on 6GB laptop GPUs.

  • MAPLE retains 54 of 64 routed experts per MoE layer, preserves six-way top-K routing, and leaves two shared invariant experts unchanged.
  • The model reports 65.6–76.3 generation tokens per second on an RTX 4050 Laptop GPU, compared with 4.5–10.0 tokens per second for hybrid offloading.
  • Benchmarks covered C++20 lock-free synchronization, Python directed hypergraph dynamic programming, and a Redis Lua token bucket, completing in 18.04–19.59 seconds.
  • The page provides llama.cpp and Ollama deployment instructions, requiring the model’s control-token prompt format and recommending 0.2 temperature, 0.95 top-p, and 99 GPU layers.

#deepseek #gguf #llama.cpp #model pruning

Generated by gpt-5.6-sol. Check the original for details.

Metric trends

Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained

Points

Score reported by the source.

  • Hacker News: 3, latest observation ; 0 over observed period since

Comments

Comment count reported by the source.

  • Hacker News: 1, latest observation ; +1 over observed period since

Aliases

SourceExternal IDURL
Hacker News50463281https://huggingface.co/AmSach/DeepSeek-Coder-V2-Lite-Instruct-MAPLE-Sub5GB-GGUF Read discussion