LLM Evaluation & Red Teaming Tool
20.0k 2026-04-12
promptfoo/promptfoo
Test, evaluate, and red-team your LLM applications to ensure security, reliability, and optimal performance across various models.
Core Features
Automated LLM prompt and model evaluations
Red teaming and vulnerability scanning for AI security
Side-by-side comparison of multiple LLM providers (GPT, Claude, Gemini, Llama, etc.)
CI/CD integration for automated checks and code scanning
Local execution for privacy and flexibility
Quick Start
npm install -g promptfooDetailed Introduction
Promptfoo is an open-source CLI and library designed for developers to rigorously test, evaluate, and secure their Large Language Model (LLM) applications. It enables automated evaluations of prompts and models, comprehensive red teaming for vulnerability scanning, and side-by-side performance comparisons across various LLM providers like OpenAI, Anthropic, and more. With features like CI/CD integration, local execution for privacy, and a developer-first approach, Promptfoo helps teams ship reliable, secure, and data-driven AI applications, moving beyond trial-and-error to metric-based decision-making.