Making AI Feel Like Someone's Actually There

Four and a Half Seconds Is an Eternity in Conversation

Senior Experience Engineer · Soul Machines · 2022–2025

Conversational AI that feels slow feels like a voice interface with loading screens. I mapped the full response pipeline and cut time-to-first-response—the wait before the agent starts speaking, not the time to complete the full response—from ~4.5 seconds to ~1.2 seconds via streaming, TTS duration prediction, and pipeline parallelisation. Affirming filler held the turn while the rest of the response completed, so conversations never felt stalled. Then added vision so agents could actually see you. The same discipline runs live on this site: the chatbot you can talk to right now uses prediction and prefetching for near-zero perceived latency.

The Problem

Multi-second accumulated latency across STT/NLU/TTS/animation pipeline, zero environmental awareness.

What I Did

Mapped entire pipeline stage by stage, restructured architecture to stream/predict/parallelize, integrated vision LLMs.

What Was Built

• Pipeline instrumentation
• Streaming responses
• TTS prediction
• Vision integration
• Multimodal analysis

Outcome

• Time-to-first-response cut from ~4.5 seconds to ~1.2 seconds
• Affirming filler masked the remaining wait—conversations never felt stalled
• Vision creates unprompted "magic moments"

Every millisecond saved compounds across every conversation. Latency is the difference between demo and product.