How to Evaluate Tool-Calling Agents

When you give an LLM access to tools, you introduce a new surface area for failure — and it breaks in two distinct ways: The model selects the wrong tool (or calls a tool when it should have answered directly). The model selects the right tool, but calls it incorrectly — wrong arguments, missing parameters, or hallucinated values. These are different problems with different fixes. Catching them requires measuring them separately. As tool use becomes central to production LLM systems, developers need a systematic way to measure tool calling behavior, understand where it fails, and iterate qu…