Agent Infrastructure

Most of the engineering in an agent product is not the agent. It is the runtime that executes the loop, the registry that describes the tools, the permission model that scopes them, the retrieval layer that fills the context, and the traces you need at three in the morning. This is the least glamorous research area and the one that decides whether anything else survives contact with customers.

The stack

Tool registry and MCP

The Model Context Protocol standardises how tools are described and invoked. It does not standardise who may invoke them - correctly, since that is an application concern. In a multi-tenant product, that leftover is most of the work: an agent serving one tenant must not reach another tenant's calendar, and the protocol has no opinion about it.

Two failure modes we design against explicitly. Schema drift: discovery is dynamic, behaviour must not be, so schemas are pinned per tenant and an upstream change is a migration with a diff rather than an automatic upgrade. Description drift: tool descriptions arriving over a protocol are still prompts, and their wording changes agent behaviour from outside your repository.

Context and retrieval

The context window is assembled fresh every turn from instructions, tool schemas, retrieved knowledge, conversation history and current input. Larger windows did not remove the need to manage it: cost scales with what you put in, attention is not uniform across a long context, and an indiscriminately filled window contains superseded answers that the model has no way to date.

Retrieval inside a synchronous conversation is a different problem from retrieval in a chat product. It sits in the caller-visible latency path, and there is room for two or three chunks in a spoken answer rather than fifteen - so precision matters far more than recall, inverting the usual tuning advice.

Guardrails

A guardrail inside the prompt drifts as the conversation grows; a guardrail outside it does not. We prefer checks that run on the output or on the proposed action, because those keep working at turn forty when the system instructions have become a small fraction of the context.

Evaluation and observability

Two different jobs that get conflated. Evaluation asks whether a change made things worse - best served by a fixed scenario suite with known-correct outcomes, run on every change. Observability asks what is happening right now, and is best served by structural signals that require no quality judgement: repeated identical tool calls, turn counts above the deployment norm, retrieval returning nothing, handovers clustering on one topic.

We use structural signals to route real conversations to human review rather than to score them automatically. Model-graded quality is useful for regression testing against fixed cases and, in our experience, unreliable as a production signal - it is confidently wrong in the same places the agent is.

Open questions

• At what catalogue size does a flat tool list stop working and require hierarchical routing?
• Is there a principled trigger for context compaction, rather than a hand-tuned threshold that is wrong in both directions?
• Can guardrails be composed and tested independently, or does every combination need its own evaluation?