AgentEvaluator detail printing raises TypeError: NoneType takes no arguments, discarding the eval result #8149

🔴 Required Information

Describe the Bug:

AgentEvaluator.evaluate() and AgentEvaluator.evaluate_eval_set() raise
TypeError: NoneType takes no arguments after the metrics have already been scored, so the
caller sees a crash instead of the eval result. print_detailed_results defaults to True on
both methods, so default arguments are enough to hit it, and that is the pytest path the
evaluation guide recommends.

Two cells of the detail table can hold None:

1. the threshold column, for the informational efficiency metrics (tool_call_count_v1,
   inference_call_count_v1, token_usage_v1, invocation_duration_v1). They carry no
   threshold by design and are reported for every eval, and
   _process_metrics_and_get_failures calls _print_details(..., threshold=None) for each of
   them (src/google/adk/evaluation/agent_evaluator.py:979 on main, :890 in v2.10.0).
2. the score column, when no invocation produced a score, for example when a judge model could
   not run. The crash then replaces the evaluator's own "was not evaluated" report, which is the
   message that tells you the metric did not run rather than regressed.

_print_details prints the frame with
tabulate(pd.DataFrame(data), headers="keys", tablefmt="grid", maxcolwidths=25)
(agent_evaluator.py:653 on main). With maxcolwidths set, tabulate casts every non-numeric
cell as _type(cell, numparse)(cell), which for None is NoneType(None) and raises
(tabulate/__init__.py:1516). Without maxcolwidths tabulate renders None as an empty cell,
which is why neither change caught this on its own: maxcolwidths=25 came in with #3379 / #3381
(fixing the row wrapping in #3363), and the threshold-free informational metrics came later.

Steps to Reproduce:

1. pip install "google-adk[eval]" (any version from 2.10.0 onwards; tabulate>=0.9 resolves to
   0.9.0, which is still the newest release).
2. Run any pytest eval the way the evaluation guide shows, for example
   await AgentEvaluator.evaluate(agent_module="my_agent", eval_dataset_file_path_or_dir="tests/integration/fixture/my_agent/simple.test.json").
3. The first thresholded metric prints its table and scores normally, then the first
   informational metric raises.

The script under Minimal Reproduction Code reproduces it with no agent, no model and no
credentials.

Expected Behavior:

The detail table prints with an empty cell where there is no threshold (or no score), and the
caller gets the eval's own result: the assertion failure for a metric below its threshold, the
"was not evaluated" report for a metric that could not run, or a pass.

Observed Behavior:

Summary: `EvalStatus.INFORMATIONAL` for Metric: `tool_call_count_v1`. Expected threshold: `None`, actual value: `3.0`.
Traceback (most recent call last):
  File "repro.py", line 20, in <module>
    AgentEvaluator._process_metrics_and_get_failures(
  File ".../google/adk/evaluation/agent_evaluator.py", line 885, in _process_metrics_and_get_failures
    AgentEvaluator._print_details(
  File ".../google/adk/evaluation/agent_evaluator.py", line 615, in _print_details
    tabulate(
  File ".../tabulate/__init__.py", line 2061, in tabulate
    list_of_lists = _wrap_text_to_colwidths(
                    ^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../tabulate/__init__.py", line 1516, in _wrap_text_to_colwidths
    str(cell) if _isnumber(cell) else _type(cell, numparse)(cell)
                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: NoneType takes no arguments

Environment Details:

• ADK Library Version (pip show google-adk): 2.13.0. The same two lines are in v2.14.1 and on
  main at 287c0b0e6.
• Desktop OS: macOS 15 (arm64)
• Python Version (python -V): 3.12
• tabulate 0.9.0, pandas 3.0.3

Model Information:

• Are you using LiteLLM: No
• Which model is being used: N/A. No model call is needed to reproduce; the crash is in the
  printing of metrics that have already been scored.

🟡 Optional Information

Regression:

Yes. The informational efficiency metrics were added in 2d428ead3 and first shipped in
v2.10.0 (2026-09-25). Every release since, up to and including v2.14.1, is affected. Before
that commit no caller passed threshold=None, so the maxcolwidths=25 added in 4284c6190
(2025-11-04) was harmless.

Additional Context:

A local workaround is to pass print_detailed_results=False, at the cost of losing the per
invocation table, or to wrap AgentEvaluator._print_details and skip the table when the
threshold is None.

adk eval is not affected: the CLI prints through
cli_eval._pretty_print_eval_results_across_runs and its --print_detailed_results flag
defaults to False. Only the programmatic path reaches _print_details.

Minimal Reproduction Code:

"""No agent, no model, no credentials."""
from google.adk.evaluation.agent_evaluator import AgentEvaluator
from google.adk.evaluation.agent_evaluator import _EvalMetricResultWithInvocation
from google.adk.evaluation.eval_case import Invocation
from google.adk.evaluation.eval_metrics import EvalMetricResult
from google.adk.evaluation.eval_metrics import EvalStatus
from google.genai import types


def content(text):
  return types.Content(parts=[types.Part(text=text)], role="user")


result = _EvalMetricResultWithInvocation(
    actual_invocation=Invocation(
        user_content=content("hi"), final_response=content("hello")
    ),
    expected_invocation=None,
    eval_metric_result=EvalMetricResult(
        metric_name="tool_call_count_v1",
        score=3.0,
        eval_status=EvalStatus.INFORMATIONAL,
    ),
)

AgentEvaluator._process_metrics_and_get_failures(
    eval_metric_results={"tool_call_count_v1": [result]},
    print_detailed_results=True,
    agent_module="my_agent",
)

The same crash with an unscored metric, which is what hides the "was not evaluated" report:

import pandas as pd
from tabulate import tabulate

tabulate(
    pd.DataFrame([{"score": None, "threshold": 0.8}]),
    headers="keys",
    tablefmt="grid",
    maxcolwidths=25,
)  # TypeError: NoneType takes no arguments

How often has this issue occurred?: Always (100%)

I have a fix for this with regression tests and can open the PR: print an empty cell for a
threshold or score that is None, which keeps the #3381 wrapping behaviour.