promptfoo

CLI and library for evaluating and red-teaming LLM applications; supports automated vulnerability scanning, side-by-side model comparison, and CI/CD integration across major LLM providers.

Connects to: OpenAI, Anthropic, Azure, Bedrock, Ollama, Vertex AI · · MIT 24,002★

Use it with an AI agent

Loadbay is an MCP server, so an agent can search the catalog and find this harness:

claude mcp add --transport http loadbay https://loadbay.xyz/api/mcp

• Source: https://github.com/promptfoo/promptfoo
• This harness as JSON: /api/harnesses/promptfoo
• Agent setup: /setup.md
• Browse all 700+ harnesses on Loadbay