Diffuspace/Clover-35B-A3B-Preview
AI model · https://huggingface.co/Diffuspace/Clover-35B-A3B-Preview
First seen by Oriel on .
AI summary
Clover-35B-A3B-Preview is DiffuSpace’s first open-source diffusion language model, combining a 35B-parameter MoE backbone, approximately 3B active parameters per token, block diffusion, and 128K context.
- Derived from Qwen3.6-35B-A3B, Clover uses 40 layers, 256 routed experts, top-8 token routing, one shared expert per layer, and hybrid GDN/full attention.
- LowConfidence decoding begins blocks with masked tokens, resolves confident positions in parallel, and appends each completed block to the causal prefix.
- The Hugging Face implementation requires Transformers 5.19.0, PyTorch 2.10 or later, and approximately 69 GB of GPU memory for BF16 weights.
- Block-level expert selection aggregates routing scores across tokens, potentially reducing distinct experts while trading quality; it does not reduce stored model weights.
- The authors recommend the official SGLang implementation for serving and evaluation because numerical differences can produce different text from Hugging Face inference.
#diffusion language models #mixture of experts #gdn #sglang
Generated by gpt-5.6-sol. Check the original for details.
Metric trends
Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained
Likes
Total likes reported by the source.
- Hugging Face Models: 247, latest observation ; +139 over observed period since
Downloads (30d)
Rolling 30-day downloads. A change in this value is not new downloads during the observed period.
- Hugging Face Models: 3,572, latest observation ; +3,467 over observed period since
Aliases
| Source | External ID | URL |
|---|---|---|
| Hugging Face Models | Diffuspace/Clover-35B-A3B-Preview | https://huggingface.co/Diffuspace/Clover-35B-A3B-Preview Read discussion |