neob.dev

Research

Research areas

Eight areas we work in. They overlap on purpose - a voice agent that books an appointment is simultaneously a realtime problem, a tool-use problem and an identity problem, and pretending otherwise is how systems get built that only work in the demo.

01

AI Agents & Agentic Systems

What separates an agent from a chatbot is not the model - it is the loop around it. We work on the control flow, the tool interface and the failure modes that appear once a language model is allowed to act.

  • The agent loop: perceive, reason, act, observe
  • Tool calling as a typed, verifiable interface
  • Planning and decomposition under real latency budgets
  • Short-term, episodic and long-term agent memory
  • Multi-agent orchestration and agent-to-agent communication
  • Human-in-the-loop gates for irreversible actions
Read the area →
02

Voice AI

Voice is the hardest interface an agent can have, because the user hears every millisecond of thinking. Our work here is about turn-taking, interruption, and holding conversational state while tools run.

  • Streaming speech-to-text and incremental hypotheses
  • Text-to-speech with first-byte latency as the primary metric
  • Voice activity detection, turn detection and barge-in
  • Latency budgets across the full mouth-to-ear path
  • SIP, WebRTC and the transport differences that matter
  • Running tool calls without leaving dead air
Read the area →
03

Realtime AI

Realtime is a systems problem, not a model problem. We study where the milliseconds actually go, which parts of a pipeline can be overlapped, and what latency budget a conversation can survive.

  • Streaming versus request/response inference
  • Speech-to-speech models against cascaded pipelines
  • Event-driven agent runtimes
  • Streaming tool calls and speculative execution
  • Conversation state under partial and revised input
  • The perceptual thresholds behind "it feels natural"
Read the area →
04

Multimodal AI

A useful agent rarely gets clean text. It gets a phone call, a photographed invoice, a screenshot and a PDF. We work on how one agent takes in several modalities without collapsing into a router of special cases.

  • Speech, text, image, video, screen and document input
  • Document understanding: layout, scans, OCR fallbacks
  • Vision as the interface of last resort
  • Modality routing and cost-aware model selection
  • Grounding answers in the artefact the user actually sent
  • Evaluating multimodal output where there is no single right answer
Read the area →
05

Computer Use & Browser Agents

When there is no API, the user interface is the API. We work on agents that drive a browser or a desktop, and on the containment that has to exist before such an agent touches anything real.

  • Browser agents: DOM, accessibility tree and pixels
  • Computer use agents beyond the browser
  • From scripted RPA to goal-driven navigation
  • When to prefer an API over the UI - almost always
  • Sandboxing, permissions and credential isolation
  • Human approval gates and stop conditions
Read the area →
06

Agent Infrastructure

Most of the engineering in an agent product is not the agent. It is the runtime, the tool registry, the permission model, the retrieval layer, and the traces you need when something goes wrong at 03:00.

  • Agent runtime and execution model
  • Tool registry, schemas and versioning
  • Model Context Protocol as a tool transport
  • Authentication, scoping and per-tenant permissions
  • Context windows, compaction and retrieval (RAG)
  • Guardrails, evaluation, tracing and observability
Read the area →
07

Conversational AI

Conversation is a transport-independent problem. The same intent arrives by phone, in a web widget or in a message thread - and the differences between those are mostly about time, not about language.

  • Dialogue state that survives channel and session changes
  • Synchronous voice against asynchronous text
  • Handover to humans without losing context
  • Knowledge grounding and refusal behaviour
  • Conversation design as an engineering artefact
  • One agent definition, several interfaces
Read the area →
08

Omnichannel AI Agents

A business should not need four disconnected AI systems for phone, website, chat and email. The interesting part is not the channel adapters - it is the shared context and the identity resolution underneath them.

  • A single agent definition behind several channel adapters
  • Shared knowledge, memory and business logic
  • Identity resolution across phone number, session and address
  • Per-channel affordances without per-channel forks
  • CRM, calendar and back-office integration as tools
  • Consistency testing across interfaces
Read the area →

How this connects

Research, experiments and engineering

These pages describe the questions. The work itself shows up in two other places, and they have different standards of proof.

Experiments

A specific attempt at one question, with its hypothesis, its result and its limitations. Some of them did not work.

Engineering

How we build things that have to keep running: trade-offs, failure modes, and what we would do differently.

Projects

Where research ended up. Currently one production system and the prototypes behind it.

There is a fourth destination that is not on this site. This lab is part of NEOB Technology GmbH, and where the research turns into work for a client rather than into a product of our own, that happens on the company side at neob.ai.