Promptfoo 0.123.1

Promptfoo release 0.123.1 delivers major expansions in provider coverage and multiple stability fixes for testing and evaluation workflows. Key Feature Additions: - Providers: Added OpenAI Agents API, GPT-Live voice sessions, Gemini 3.8, and Vertex Live support. - Schemas: Added portable config schemas for HTTP and MCP integrations. - Bedrock: Refreshed authentication handling across HTTP adapters. - Ollama: Added Ollama 0.34 features, surfaced thinking output and finish reasons, supported streamed tool calls, migrated embeddings to /api/embed , and surfaced HTTP errors directly. - Gemini: Updated tool and media handling, ensuring prompt tool policies override passthroughs. Significant Bug Fixes and Changes: - Assertions and Evaluations: Defaulted RAG assertion thresholds to 0.5; prevented invalid inversion of grader failures on not-classifier , not-search-rubric , and RAG/context-faithfulness matchers; bound tool-call parser state; and rejected non-string rubrics or malformed webhook results. - Security and Auth: Isolated Agents endpoint credentials, hardened custom header requests, and stopped leaking local configuration into outbound request bodies. - Integrations: Improved Langfuse environment handling and prompt fetch sharing; honored --env-file for Helicone keys and Langfuse base URLs. - Redteaming: Grounded agent skill plans in application evidence and ensured vulnerabilities are retained even when scoring encounters failures.

• Verify evaluation test suites relying on RAG assertion thresholds, as defaults now explicitly apply at 0.5.
• Review inverse assertions ( not-classifier , not-search-rubric , context-faithfulness) to ensure test expectations align with the fix preventing inverted grader failures.
• Update Ollama provider setups to accommodate the /api/embed migration and test newly exposed thinking outputs or streamed tool calls.
• Examine provider configs if sharing or exporting configurations, adopting portable HTTP and MCP config schemas where applicable.