LLM Evaluation & Red Teaming Tool
20.5k 2026-04-24
promptfoo/promptfoo
A CLI and library for testing, evaluating, and red-teaming LLM applications to ensure security, reliability, and performance across various models.
Core Features
Automated evaluation of prompts and models.
Red teaming and vulnerability scanning for LLM apps.
Side-by-side comparison of multiple LLM providers (GPT, Claude, Gemini, Llama, etc.).
CI/CD integration for automated checks and code scanning.
Local execution for privacy and flexibility.
Quick Start
npm install -g promptfooDetailed Introduction
Promptfoo is an open-source, developer-first CLI and library designed to streamline the development of secure and reliable LLM applications. It provides robust tools for automated prompt and model evaluation, comprehensive red teaming, and vulnerability scanning, enabling developers to move beyond trial-and-error. With support for comparing various LLM providers and seamless integration into CI/CD pipelines, Promptfoo empowers teams to make data-driven decisions, enhance AI security, and ensure compliance, all while keeping evaluations private and local.