ggml-org/llama.cpp
Repository · https://github.com/ggml-org/llama.cpp
LLM inference in C/C++
First seen by Oriel on .
AI summary
llama.cpp is a dependency-free C/C++ project for minimal-setup, high-performance LLM and VLM inference across diverse local and cloud hardware.
- It supports Apple silicon, x86 AVX variants, RISC-V extensions, NVIDIA CUDA, AMD HIP, Moore Threads MUSA, Vulkan, SYCL, Metal, and other backends.
- Integer quantization from 1.5-bit through 8-bit reduces memory use and accelerates inference across supported models.
- CPU+GPU hybrid inference can partially accelerate models exceeding available VRAM.
- The project provides command-line and OpenAI-compatible server interfaces, Docker support, pre-built binaries, and source builds.
- Users can download models directly from Hugging Face with commands such as `llama cli -hf ggml-org/Qwen3.5-0.8B-GGUF`.
#llama.cpp #llm inference #quantization #gpu computing
Generated by gpt-5.6-sol. Check the original for details.
Metric trends
Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained
Stars
Total repository stars at each observation.
- GitHub Trending: 134,799, latest observation ; +574 over observed period since
Stars gained (1d)
Daily star gain reported by GitHub Trending. A change compares two daily values; it is not additional stars between observations.
- GitHub Trending: 176, latest observation ; +14 over observed period since
Forks
Total repository forks at each observation.
- GitHub Trending: 25,539, latest observation ; +72 over observed period since
Aliases
| Source | External ID | URL |
|---|---|---|
| GitHub Trending | ggml-org/llama.cpp | https://github.com/ggml-org/llama.cpp Read discussion |