Lab note

Designing human-in-the-loop AI systems

Adding a human is easy. Adding one whose attention is actually worth something is the design problem.

NEOB Research Published 5 min read

Human-in-the-loop is usually specified as a checkbox: a person approves before the action. It is more useful to ask *which* loop, because there are four, with different costs and different failure modes.

PatternHuman seesCostFails when
Approval gateEvery action before it happensHigh, per actionVolume causes rubber-stamping
EscalationOnly cases the agent declinesLowThe agent does not know it should decline
Review queueA sample, after the factModerateDamage is already done
AuditTraces, on investigationLowOnly useful retrospectively

The rubber-stamp failure

An approval gate that fires on every action gets clicked through within a week. The approval is then worse than nothing, because the system now carries a recorded human decision that nobody actually made.

Approval gates should be rare by construction: reserved for irreversible or high-consequence actions, so that seeing one is itself information. If the gate fires constantly, the classification behind it is wrong.

Escalation needs a self-assessment the agent can actually make

Asking a model whether it is confident produces confident answers. More reliable triggers are structural: a required field is missing, the retrieval returned nothing, the customer has rephrased three times, the request matches a known-sensitive category. These are checks on the situation, not on the model's self-report.

Handover quality is the whole thing

An escalation that dumps a customer into a queue to start again is worse than no escalation - the customer has now explained the problem twice. The handover has to carry what was established, what was attempted and what the agent believes the customer wants. If the context was not structured for handover, it cannot be handed over.

A design rule

Decide where the human sits by asking what is not undoable. Everything reversible can be reviewed after the fact; everything irreversible needs a person before, and a design where those two sets are clearly separated is one you can staff.