Promptfoo
promptfoo.dev
An open-source tool for testing, evaluating, and comparing LLM prompts and models through automated test suites, red-teaming, and side-by-side output comparison.
Why it is useful
Brings software testing discipline to prompt engineering — you define test cases and assertions, then run them across multiple models or prompt variants to see what actually works. Particularly useful when switching models or refining prompts, where eyeballing outputs does not scale. The red-teaming module probes for safety issues systematically rather than relying on manual testing.