Oriel — AI & technology worth noticing, every day.

ggml-org/llama.cpp

Repository · https://github.com/ggml-org/llama.cpp

LLM inference in C/C++

First seen by Oriel on .

AI summary

llama.cpp is a dependency-free C/C++ project for minimal-setup, high-performance LLM and VLM inference across diverse local and cloud hardware.

  • It supports Apple silicon, x86 AVX variants, RISC-V extensions, NVIDIA CUDA, AMD HIP, Moore Threads MUSA, Vulkan, SYCL, Metal, and other backends.
  • Integer quantization from 1.5-bit through 8-bit reduces memory use and accelerates inference across supported models.
  • CPU+GPU hybrid inference can partially accelerate models exceeding available VRAM.
  • The project provides command-line and OpenAI-compatible server interfaces, Docker support, pre-built binaries, and source builds.
  • Users can download models directly from Hugging Face with commands such as `llama cli -hf ggml-org/Qwen3.5-0.8B-GGUF`.

#llama.cpp #llm inference #quantization #gpu computing

Generated by gpt-5.6-sol. Check the original for details.

Metric trends

Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained

Stars

Total repository stars at each observation.

  • GitHub Trending: 134,799, latest observation ; +574 over observed period since

Stars gained (1d)

Daily star gain reported by GitHub Trending. A change compares two daily values; it is not additional stars between observations.

  • GitHub Trending: 176, latest observation ; +14 over observed period since

Forks

Total repository forks at each observation.

  • GitHub Trending: 25,539, latest observation ; +72 over observed period since

Aliases

SourceExternal IDURL
GitHub Trendingggml-org/llama.cpphttps://github.com/ggml-org/llama.cpp Read discussion