ExecuTorch WebGPU: PyTorch Models on the GPU, in the Browser
Web page · https://pytorch.org/blog/executorch-webgpu-pytorch-models-on-the-gpu-in-the-browser
First seen by Oriel on .
AI summary
ExecuTorch’s new WebGPU backend runs torch.export-generated PyTorch .pte models locally on browser GPUs without format conversion, delivering 4–7× faster prefill and 2–10× faster decode than its WebAssembly CPU path.
- A single .pte artifact uses WGSL compute shaders across Chrome and Safari, translating to Metal Shading Language, SPIR-V, or HLSL for supported platforms.
- The WebAssembly runtime allocates GPU buffers, schedules command-buffer dispatches, and supports dynamic shapes using allocate-at-max buffers, recalculated workgroups, and specialized WGSL kernels.
- Benchmarks used quantized Llama 3.2 1B and Qwen3-0.6B in Chrome on Apple M1 Pro, spanning 128 to 8K context tokens.
- Validated workloads include LLMs, SmolVLM, SAM 2, Depth Anything V2, D-FINE, Real-ESRGAN, Whisper, Phonon-31M-EN, and on-device LoRA and DiReFT fine-tuning.
- The backend remains in early development, with stability, performance, operator, data type, and model coverage still being expanded.
#webgpu #executorch #pytorch #browser inference
Generated by gpt-5.6-sol. Check the original for details.
Metric trends
Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained
Points
Score reported by the source.
- Hacker News: 9, latest observation ; +2 over observed period since
Comments
Comment count reported by the source.
- Hacker News: 0, latest observation ; 0 over observed period since
Aliases
| Source | External ID | URL |
|---|---|---|
| Hacker News | 50467383 | https://pytorch.org/blog/executorch-webgpu-pytorch-models-on-the-gpu-in-the-browser/ Read discussion |