Confident AI — Developer Documentation Highlights

Confident AI exposes its functionality through open-source frameworks and a full REST API, making it straightforward to integrate into existing ML and software delivery pipelines.

Open-Source Frameworks:

DeepEval — The core LLM evaluation framework. Provides research-backed metrics for scoring LLM outputs (e.g., correctness, faithfulness, relevancy). Use it locally or connect it to the Confident AI platform for centralized reporting.
DeepTeam — The red teaming framework. Automates adversarial attack simulations against your LLM application to surface safety and robustness issues.

SDK & API Access:

Python SDK and TypeScript SDK for programmatic evaluation, tracing, and dataset management.
REST API via cURL or Postman-style HTTP endpoints for language-agnostic integration.
Full API access enables complete pipeline automation — trigger test runs, pull results, and gate deployments programmatically.

CI/CD Integration:

Native support for CI/CD pipelines and GitHub, enabling evaluation as a quality gate on every pull request or deployment.
Cloud provider integrations with AWS, Azure, and GCP for flexible infrastructure alignment.

Deployment & Data Residency:

Self-hostable (on-premises deployment available on Enterprise tier).
Multi-region data residency supported for data sovereignty requirements.
HIPAA-eligible configuration available on Enterprise.

Key Developer Workflows:

Git-based prompt versioning for diff-tracking and rollback.
Automatic dataset curation from production traces to keep evaluation sets fresh.
Multi-turn chat simulation for testing stateful, conversational LLM applications.
Trace-level observability with monitoring and alerting hooks.