Experiment · In production

Booking an appointment end to end, without a human in the loop

The model part is trivial. The hard parts are concurrency, timezones, and what the agent says while a calendar API is thinking.

NEOB Engineering Published Updated 7 min readIn production
ai-agentstool-usevoice-aiintegrations

Problem

Booking an appointment is the canonical agent demo, and it is also the point where a demo stops being a demo: it writes to a real calendar that real people look at.

The interesting failures are not conversational. They are two callers taking the same slot at the same time, a customer in a different timezone, a calendar API that takes two seconds to answer while a caller is listening to silence, and a booking that half-succeeds.

Hypothesis

The reliability of an agent booking flow is determined by the tool layer and its failure semantics, not by the model. Making the tool idempotent and the timing conversational is sufficient for production use.

Architecture

  1. 01Intent
  2. 02Constraints
  3. 03Availability lookup
  4. 04Offer slots
  5. 05Caller picks
  6. 06Hold
  7. 07Confirm
  8. 08Notify
Booking flow. Every arrow is a place the caller can change their mind.

The hold step is the one that is usually missing. Between offering a slot and the caller accepting it, several seconds pass in which another caller can take it. Without a hold, the agent confirms a booking that then fails, and it does so after telling the caller they are booked.

The tools are deliberately coarse. Rather than exposing a generic calendar API, the agent gets three narrow operations with business rules already applied.

Tool surface - narrow on purpose
find_slots({
  service: string,          // determines duration and resource
  earliest: ISODateTime,
  latest:   ISODateTime,
  limit:    number,         // never offer more than 3 by voice
}) -> Slot[]

hold_slot({
  slotId:   string,
  ttlSec:   number,         // released automatically; no orphan holds
}) -> { holdId: string }

confirm_booking({
  holdId:   string,
  customer: { name, phone, email? },
  idempotencyKey: string,   // same key = same booking, always
}) -> Booking

A generic "create calendar event" tool would have pushed opening hours, service durations, buffer times and resource assignment into the prompt. Every one of those is a business rule that belongs in code, where it can be tested.

Implementation

Idempotency

The idempotency key is derived from the hold and the caller, not generated per attempt. If the confirm call times out and the agent retries - or the caller says "did that work?" and the agent checks - the second call returns the same booking instead of creating a second one.

Speaking while waiting

Calendar lookups are slow enough to be audible. Silence during a tool call is the single most common reason a caller thinks the line has dropped. The agent therefore emits a short filler before the call and only then awaits it - and the filler is generated from the tool being invoked, so it never claims to be doing something it is not.

tool-narration.ts
// Narrate before awaiting, not after. The filler is derived from the tool
// so it cannot describe an action the agent is not actually taking.
const NARRATION: Record<string, string> = {
  find_slots:       'Let me check the calendar.',
  confirm_booking:  'Booking that for you now.',
}

async function callWithNarration(tool: Tool, args: unknown) {
  if (NARRATION[tool.name]) void speak(NARRATION[tool.name])
  return tool.invoke(args)   // caller hears speech while this runs
}

Timezones

Everything internal is UTC; everything spoken is the business's local time, stated explicitly when the caller's number suggests a different region. "Thursday at three" is ambiguous in a way that produces no error message and one very annoyed customer.

Result

This shipped, and is the pattern behind the appointment booking in bitpull.ai, which integrates with Google Calendar and Calendly. The design rule that came out of it: put business rules in the tool, not in the prompt - the prompt then only has to be good at conversation, which is what it is good at.

The hold step removed a class of failure that had been treated as unavoidable. The narration change removed a class of complaint we had misattributed to model latency, when it was actually about silence.

What we got wrong initially: we let the agent offer five or six slots because the API returned them. By voice, more than three options is unusable - the caller has forgotten the first by the time they hear the last. The limit is now enforced in the tool rather than requested in the prompt.

Limitations

What this experiment does not establish. Listed because an experiment without limitations is an advertisement.

  • Holds require support from the underlying calendar. Where the backing system has no hold concept we emulate it in our own store, which is correct only as long as nothing else writes to that calendar.
  • Rescheduling and cancellation are harder than booking and are handled more conservatively: both currently require confirming an existing reference rather than searching by name.
  • Multi-resource booking - an appointment that needs a person *and* a room *and* equipment - was out of scope and does not follow from this design.
  • The narration approach depends on tool latency being roughly predictable. A tool that is sometimes instant and sometimes slow produces awkward pauses either way.

Next steps

  • Extend the hold/confirm pattern to other write tools - orders, callbacks, tickets - since the shape generalises.
  • Look at what a caller actually retains from spoken options, to see whether three is the right limit or merely better than six.
  • Handle the partial-failure case where the booking succeeds but the confirmation message does not.