llama.cpp
github.com/ggml-org/llama.cpp
A high-performance C++ inference engine for running large language models on CPU and GPU with minimal dependencies, supporting quantized GGUF models for consumer hardware.
Why it is useful
The engine that made local LLMs practical on consumer hardware. Its quantization support lets a 7B model run on 8GB of RAM and a 70B model run on a machine with two mid-range GPUs. Being the most widely supported inference backend means virtually every open-weight model has a GGUF version compatible with it. Other tools like Ollama and LM Studio run llama.cpp under the hood.
More in Developer Utilities
Other developer utilities you might not have heard of.
- URLScan.ioA free service that scans URLs and renders them in an isolated browser, capturing all network requests, DOM structure, and screenshots for security analysis.
- IT ToolsA collection of handy browser-based utilities for developers and IT professionals — encoding, formatting, generation, conversion, and network tools in one place.
- ObservableReactive JavaScript notebooks for data visualization and analysis.
- KeycloakOpen-source identity and access management — single sign-on, social login, and user federation as a self-hostable alternative to Auth0.
- AppsmithOpen-source low-code platform for building internal tools and admin panels — a self-hostable alternative to Retool.
- PlaywrightMicrosoft's end-to-end testing framework that controls Chromium, Firefox, and WebKit with a single API — with auto-waiting, network interception, parallel test execution, and a visual trace debugger.