Armature Launches Analytics for AI Agent Tool Calls

Armature, founded by Theodore and Louis, has launched an analytics product providing deep observability into AI agent tool calls. This platform allows developers to understand the full context of an agent's interaction with external APIs, including the user's original request and the agent's internal thought process leading to a specific tool invocation.

The core functionality involves wrapping an agent's Multi-Agent Collaboration Platform (MCP) with a lightweight SDK, available for Typescript, Python, and Go. Once integrated, the system reconstructs entire user sessions, presenting them as a readable conversation between the user and the AI agent. This offers a window into the otherwise opaque decision-making of an AI agent.

Beyond session reconstruction, Armature clusters common use cases, helping developers identify popular interaction patterns. Crucially, it also highlights frequent issues encountered by users' agents, enabling targeted debugging and improvement. This moves beyond simple error logging to provide actionable insights into agent performance and user satisfaction.

The company's origin stems from their own struggles with understanding how users interacted with their previous AI testing tool. This led them to instrument agents to ask 'why' they were making specific tool calls and what the user's intent or frustration might be. Privacy is addressed through client-side redaction of sensitive data before it reaches Armature's servers.

Armature is currently available self-serve at armature.tech, offering a generous free tier. The setup process is designed to take less than five minutes. While the system provides immediate analytics, the founders are working towards a closed-loop system that integrates evaluations, recommends fixes, tests them at scale, and even proposes pull requests for direct implementation.

One compelling example shared by the team involved a marketing automation platform. Armature Analytics revealed users were frustrated by the inability to change target audiences post-campaign creation. After shipping the feature, subsequent evaluations caught a critical bug where smaller models hallucinated `audience_ids`, potentially sending campaigns to all contacts. This illustrates the platform's utility in preventing costly production errors.

This is the kind of story that makes what we are building feel so helpful!

For developers building complex AI applications with tool-using agents, Armature offers a specialized layer of observability that traditional logging and APM tools do not. It aims to provide comprehensive control over the 'Agent Experience', a growing concern as AI agents become more autonomous and integrate deeper into business workflows.