Edge0/Edge0-145B-NVFP4
AI model · https://huggingface.co/Edge0/Edge0-145B-NVFP4
First seen by Oriel on .
AI summary
Edge0/Edge0-145B-NVFP4 is a 145B latent sparse MoE model designed to run locally on one 24 GB RTX 5090 D using NVFP4 quantization and host-memory residency.
- The model has 48 layers, 512 experts, 10 active experts per token, a 256K context, and native 4-bit training.
- Reported RTX 5090 D throughput is 98.3 tok/s for agent decoding, 3165.7 tok/s for 16K-token prefill, and 86.5 terminal-bench 2.1.
- With residency enabled, it uses 23.25 GiB VRAM and 38.27 GiB host RAM; SSD offload and configurable expert caches support lower-memory systems.
- edge0-infer provides command-line chat plus OpenAI- and Anthropic-compatible endpoints, while Edge0 Chat runs model folders locally on Windows.
- The model supports llama.cpp, vLLM, Ollama, Pi, Hermes Agent, OpenClaw, and other integrations, but commercial service or bundling requires a separate written Edge0 license.
#mixture of experts #model quantization #local inference #nvidia cuda
Generated by gpt-5.6-sol. Check the original for details.
Metric trends
Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained
Likes
Total likes reported by the source.
- Hugging Face Models: 458, latest observation ; +334 over observed period since
Downloads (30d)
Rolling 30-day downloads. A change in this value is not new downloads during the observed period.
- Hugging Face Models: 55,335, latest observation ; +41,201 over observed period since
Aliases
| Source | External ID | URL |
|---|---|---|
| Hugging Face Models | Edge0/Edge0-145B-NVFP4 | https://huggingface.co/Edge0/Edge0-145B-NVFP4 Read discussion |