ashhart/TensorFold
Repository · https://github.com/ashhart/tensorfold
LLM Inference Engine for Metal, CUDA and Vulkan.
First seen by Oriel on .
AI summary
TensorFold is a Zig-based local LLM inference engine for Apple Silicon and NVIDIA GPUs, directly running Metal, CUDA, and Vulkan kernels without Python or MLX installation.
- It downloads checkpoints, tokenizes requests, and exposes OpenAI- and Anthropic-compatible APIs, including chat completions, completions, Responses, and Anthropic Messages.
- Apple Silicon uses Homebrew and Metal, while NVIDIA Linux releases require glibc 2.28 or newer, an R580 driver, and CUDA 13.0.
- Request concurrency is configurable with --parallel; automatic mode accepts up to eight streams when memory and engine capacity allow.
- TensorFold Starlight adds governed multi-node serving, identity integration, model policies, placement, token accounting, and optional offline or approved frontier-provider access.
- Living Weights experimentally writes checked lessons into model checkpoint files, but related facts can interfere, recall declines after several dozen facts, and decoding slows on Macs.
#llm inference #zig #metal #cuda
Generated by gpt-5.6-sol. Check the original for details.
Metric trends
Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained
Stars
Total repository stars at each observation.
- GitHub Trending: 2,232, latest observation ; +415 over observed period since
Stars gained (1d)
Daily star gain reported by GitHub Trending. A change compares two daily values; it is not additional stars between observations.
- GitHub Trending: 37, latest observation ; -21 over observed period since
Forks
Total repository forks at each observation.
- GitHub Trending: 521, latest observation ; +136 over observed period since
Aliases
| Source | External ID | URL |
|---|---|---|
| GitHub Trending | ashhart/tensorfold | https://github.com/ashhart/TensorFold Read discussion |