Oriel — AI & technology worth noticing, every day.

Edge0/Edge0-145B-NVFP4

AI model · https://huggingface.co/Edge0/Edge0-145B-NVFP4

First seen by Oriel on .

AI summary

Edge0/Edge0-145B-NVFP4 is a 145B latent sparse MoE model designed to run locally on one 24 GB RTX 5090 D using NVFP4 quantization and host-memory residency.

  • The model has 48 layers, 512 experts, 10 active experts per token, a 256K context, and native 4-bit training.
  • Reported RTX 5090 D throughput is 98.3 tok/s for agent decoding, 3165.7 tok/s for 16K-token prefill, and 86.5 terminal-bench 2.1.
  • With residency enabled, it uses 23.25 GiB VRAM and 38.27 GiB host RAM; SSD offload and configurable expert caches support lower-memory systems.
  • edge0-infer provides command-line chat plus OpenAI- and Anthropic-compatible endpoints, while Edge0 Chat runs model folders locally on Windows.
  • The model supports llama.cpp, vLLM, Ollama, Pi, Hermes Agent, OpenClaw, and other integrations, but commercial service or bundling requires a separate written Edge0 license.

#mixture of experts #model quantization #local inference #nvidia cuda

Generated by gpt-5.6-sol. Check the original for details.

Metric trends

Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained

Likes

Total likes reported by the source.

  • Hugging Face Models: 458, latest observation ; +334 over observed period since

Downloads (30d)

Rolling 30-day downloads. A change in this value is not new downloads during the observed period.

  • Hugging Face Models: 55,335, latest observation ; +41,201 over observed period since

Aliases

SourceExternal IDURL
Hugging Face ModelsEdge0/Edge0-145B-NVFP4https://huggingface.co/Edge0/Edge0-145B-NVFP4 Read discussion