Oriel — AI & technology worth noticing, every day.

ashhart/TensorFold

Repository · https://github.com/ashhart/tensorfold

LLM Inference Engine for Metal, CUDA and Vulkan.

First seen by Oriel on .

AI summary

TensorFold is a Zig-based local LLM inference engine for Apple Silicon and NVIDIA GPUs, directly running Metal, CUDA, and Vulkan kernels without Python or MLX installation.

  • It downloads checkpoints, tokenizes requests, and exposes OpenAI- and Anthropic-compatible APIs, including chat completions, completions, Responses, and Anthropic Messages.
  • Apple Silicon uses Homebrew and Metal, while NVIDIA Linux releases require glibc 2.28 or newer, an R580 driver, and CUDA 13.0.
  • Request concurrency is configurable with --parallel; automatic mode accepts up to eight streams when memory and engine capacity allow.
  • TensorFold Starlight adds governed multi-node serving, identity integration, model policies, placement, token accounting, and optional offline or approved frontier-provider access.
  • Living Weights experimentally writes checked lessons into model checkpoint files, but related facts can interfere, recall declines after several dozen facts, and decoding slows on Macs.

#llm inference #zig #metal #cuda

Generated by gpt-5.6-sol. Check the original for details.

Metric trends

Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained

Stars

Total repository stars at each observation.

  • GitHub Trending: 2,232, latest observation ; +415 over observed period since

Stars gained (1d)

Daily star gain reported by GitHub Trending. A change compares two daily values; it is not additional stars between observations.

  • GitHub Trending: 37, latest observation ; -21 over observed period since

Forks

Total repository forks at each observation.

  • GitHub Trending: 521, latest observation ; +136 over observed period since

Aliases

SourceExternal IDURL
GitHub Trendingashhart/tensorfoldhttps://github.com/ashhart/TensorFold Read discussion