Four and a Half Seconds Is an Eternity in Conversation
Making AI Feel Like Someone's Actually There
Recorded and edited by me. It demonstrates an AI system; it wasn’t made by one.
Conversational AI that feels slow feels like a voice interface with loading screens. I mapped the full response pipeline and cut time-to-first-response—the wait before the agent starts speaking, not the time to complete the full response—from ~4.5 seconds to ~1.2 seconds via streaming, TTS duration prediction, and pipeline parallelisation. Affirming filler held the turn while the rest of the response completed, so conversations never felt stalled. Then added vision so agents could actually see you. The same discipline runs live on this site: the chatbot you can talk to right now uses prediction and prefetching for near-zero perceived latency.
Problem Space
Multi-second accumulated latency across STT/NLU/TTS/animation pipeline, zero environmental awareness.
What I Did
Mapped entire pipeline stage by stage, restructured architecture to stream/predict/parallelize, integrated vision LLMs.
What I Built
- Pipeline instrumentation
- Streaming responses
- TTS prediction
- Vision integration
- Multimodal analysis
Adoption
Applied across agents I built, authored, or materially improved.
Outcome
- Time-to-first-response cut from ~4.5 seconds to ~1.2 seconds
- Affirming filler masked the remaining wait—conversations never felt stalled
- Vision creates unprompted "magic moments"
Why This Matters
Every millisecond saved compounds across every conversation. Latency is the difference between demo and product.