MACaQ: Fast, Model Agnostic Calibrated Quantization for Low-Precision Weights and Activations
Paper · https://arxiv.org/abs/2611.10649
Weight-activation quantization can reduce the memory and compute cost of neural-network inference, but recent methods often require expensive optimization and architecture-specific propagation of basis transforms across layers. We introduce MACaQ, an MXFP4 W4A4 post-training quantization method that treats every linear layer independently. MACaQ applies a per-channel scale and a block-Hadamard transform online to each layer's activations, while absorbing the corresponding inverse transforms into its weights. Retaining these transforms online avoids cross-layer dependencies and enables a simple streaming implementation of the quantization process, in which only one layer must reside in memory at a time. To improve over round-to-nearest quantization, we introduce Flip-Refinement, a Hessian-aware coordinate-descent procedure that greedily moves individual FP4 codes to adjacent representable values. Despite executing the activation transforms at inference time, MACaQ adds only 2.3% end-to-end overhead over pure MXFP4 on Qwen3-32B in vLLM. MACaQ reaches lower perplexity than DuQuant++ and MR-GPTQ, and already matches their best perplexity on Llama-3-8B in 11 times less quantization time. We further apply MACaQ without architecture-specific modifications to an image diffusion model, illustrating the generality of its layer-local design and show quantization of a 122B parameter model in 64 minutes, using only 32.5GB GPU RAM.
First seen by Oriel on .
AI summary
MACaQ is a model-agnostic MXFP4 W4A4 post-training quantization method that independently calibrates linear layers, reducing optimization time and memory while achieving competitive perplexity with low inference overhead.
- MACaQ applies online per-channel scaling and block-Hadamard transforms to activations, absorbing corresponding inverse transforms into weights without cross-layer dependencies.
- Its streaming quantization requires only one layer in memory, enabling a 122B parameter model to be quantized in 64 minutes using 32.5GB GPU RAM.
- Flip-Refinement uses Hessian-aware coordinate descent to greedily move individual FP4 codes toward adjacent representable values, improving upon round-to-nearest quantization.
- On Llama-3-8B, MACaQ matches the best perplexity of DuQuant++ and MR-GPTQ with 11 times less quantization time.
- Online activation transforms add 2.3% end-to-end overhead over pure MXFP4 on Qwen3-32B in vLLM, and the method also applies to an image diffusion model.
#post-training quantization #mxfp4 #neural networks #model inference
Generated by gpt-5.6-sol. Check the original for details.
Metric trends
Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained
Points
Score reported by the source.
- Hugging Face Papers: 32, latest observation ; +12 over observed period since
Comments
Comment count reported by the source.
- Hugging Face Papers: 2, latest observation ; +1 over observed period since
Aliases
| Source | External ID | URL |
|---|---|---|
| Hugging Face Papers | 2611.10649 | https://arxiv.org/abs/2611.10649 Read discussion |