> ## Documentation Index
> Fetch the complete documentation index at: https://agno-v2-docs-scavio-google-v2.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# What are Evals?

> Measure agent and team quality across expected answers, custom criteria, tool behavior, and performance.

Teams shipping agents need to know whether a prompt, model, tool, or context change improved the product. Evals turn expected behavior into repeatable checks that can run during development and in CI.

```python accuracy_eval.py theme={null}
from agno.agent import Agent
from agno.eval.accuracy import AccuracyEval
from agno.models.openai import OpenAIResponses
from agno.tools.calculator import CalculatorTools

agent = Agent(
    model=OpenAIResponses(id="gpt-5.4-mini"),
    tools=[CalculatorTools()],
)

evaluation = AccuracyEval(
    name="Calculator accuracy",
    model=OpenAIResponses(id="gpt-5.4-mini"),
    agent=agent,
    input="What is 10 factorial?",
    expected_output="3628800",
)

result = evaluation.run(print_results=True)
assert result is not None and result.avg_score >= 8
```

`AccuracyEval` runs the agent, then uses the evaluator model to compare its response with the expected output.

## Choose an Eval

| Question                                                                   | Eval                                             |
| -------------------------------------------------------------------------- | ------------------------------------------------ |
| Does the response match an expected answer?                                | [Accuracy](/evals/accuracy/overview)             |
| Does the response meet product-specific quality criteria?                  | [Agent as Judge](/evals/agent-as-judge/overview) |
| Did the agent or team call the expected tools with the expected arguments? | [Reliability](/evals/reliability/overview)       |
| How long does the code take and how much memory does it use?               | [Performance](/evals/performance/overview)       |
| Do many judge and reliability cases pass together?                         | [Eval Suites](/evals/suite/overview)             |

Accuracy and agent-as-judge checks use a model to evaluate output. Reliability checks inspect recorded tool calls and arguments. Performance checks execute a function repeatedly and report runtime and memory statistics.

## Build a Regression Suite

Eval suites run multiple `Case` definitions against agents or teams. Each case can check custom criteria, expected tool calls, or both.

```python evals.py theme={null}
import sys

from agno.agent import Agent
from agno.eval import Case, cli
from agno.models.openai import OpenAIResponses
from agno.tools.calculator import CalculatorTools

agent = Agent(
    id="calculator-agent",
    model=OpenAIResponses(id="gpt-5.4-mini"),
    tools=[CalculatorTools()],
    instructions="Use calculator tools for arithmetic.",
)

CASES = (
    Case(
        name="factorial_uses_calculator",
        agent=agent,
        input="What is 10 factorial?",
        tags=("smoke",),
        criteria="States that 10 factorial equals 3628800.",
        expected_tool_calls=("factorial",),
    ),
)

if __name__ == "__main__":
    sys.exit(cli(CASES))
```

```bash theme={null}
python evals.py --tag smoke --json-output tmp/evals.json
```

The CLI returns a failing exit code when a selected case fails and can write a JSON report for CI artifacts. See [Eval Suites](/evals/suite/overview) for selectors, timeouts, judge modes, and programmatic execution.

## What to Evaluate

Start with behavior that matters to the product:

| Product requirement                             | Check                                               |
| ----------------------------------------------- | --------------------------------------------------- |
| Support answers cite the correct policy         | Accuracy against reviewed expected answers          |
| Responses follow your tone and escalation rules | Agent-as-judge criteria                             |
| Refund requests call the approval tool          | Reliability tool-call checks                        |
| Search stays within a latency budget            | Performance thresholds tracked by your test harness |

Keep cases focused on one behavior so failures point to a clear prompt, model, context, or tool change.

## Next Steps

<CardGroup cols={2}>
  <Card title="Accuracy" icon="bullseye" href="/evals/accuracy/overview">
    Compare responses with expected answers.
  </Card>

  <Card title="Agent as Judge" icon="scale-balanced" href="/evals/agent-as-judge/overview">
    Score custom quality criteria.
  </Card>

  <Card title="Reliability" icon="shield-check" href="/evals/reliability/overview">
    Check tool calls and arguments.
  </Card>

  <Card title="Performance" icon="stopwatch" href="/evals/performance/overview">
    Measure runtime and memory usage.
  </Card>

  <Card title="Eval Suites" icon="list-check" href="/evals/suite/overview">
    Run many judge and reliability cases in CI.
  </Card>
</CardGroup>