Stream low-latency speech from AI agents using Kokoro TTS on Amazon SageMaker AI
Voice agents often wait for an agent's full response before starting speech synthesis, adding latency users feel directly. This post shows how to stream each sentence to a Kokoro-82M text-to-speech model on Amazon SageMaker AI as the agent produces it, and how to get that audio back to a browser over Server-Sent Events or a unified binary transport.
First seen by Oriel on .
AI summary
The post demonstrates streaming Kokoro-82M speech from AI agents through a SageMaker AI endpoint, starting synthesis sentence by sentence to reduce time to first audio.
- The model runs in a custom container on an ml.g5.xlarge real-time endpoint, with HTTP and persistent bidirectional WebSocket routes.
- A sentence chunker releases completed sentences as text arrives; the WebSocket handler must accept binary frames and keep sessions open between messages.
- Sending the first sentence separately reduced mean time to first audio byte from 329.3 ms to 156.7 ms, a 52.4% improvement in five trials.
- Base64 audio over SSE adds about 33% payload overhead and requires buffering across delivery chunks; a unified binary and JSON stream avoids those costs when building a voice path from scratch.
- The sample pins an older PyTorch with advisory PYSEC-2026-2286, fixed in torch 2.10.0; the post recommends upgrading before non-proof-of-value use.
#text-to-speech #aws sagemaker #websockets #ai agents
Generated by gpt-5.6-sol. Check the original for details.
Aliases
| Source | External ID | URL |
|---|---|---|
| AWS Machine Learning Blog | 351e07d8236b8a05fa15e510dcb5b937d7a9dec8 | https://aws.amazon.com/blogs/machine-learning/stream-low-latency-speech-from-ai-agents-using-kokoro-tts-on-amazon-sagemaker-ai/ Read discussion |