Oriel — AI & technology worth noticing, every day.

Stream low-latency speech from AI agents using Kokoro TTS on Amazon SageMaker AI

Web page · https://aws.amazon.com/blogs/machine-learning/stream-low-latency-speech-from-ai-agents-using-kokoro-tts-on-amazon-sagemaker-ai

Voice agents often wait for an agent's full response before starting speech synthesis, adding latency users feel directly. This post shows how to stream each sentence to a Kokoro-82M text-to-speech model on Amazon SageMaker AI as the agent produces it, and how to get that audio back to a browser over Server-Sent Events or a unified binary transport.

First seen by Oriel on .

AI summary

The post demonstrates streaming Kokoro-82M speech from AI agents through a SageMaker AI endpoint, starting synthesis sentence by sentence to reduce time to first audio.

  • The model runs in a custom container on an ml.g5.xlarge real-time endpoint, with HTTP and persistent bidirectional WebSocket routes.
  • A sentence chunker releases completed sentences as text arrives; the WebSocket handler must accept binary frames and keep sessions open between messages.
  • Sending the first sentence separately reduced mean time to first audio byte from 329.3 ms to 156.7 ms, a 52.4% improvement in five trials.
  • Base64 audio over SSE adds about 33% payload overhead and requires buffering across delivery chunks; a unified binary and JSON stream avoids those costs when building a voice path from scratch.
  • The sample pins an older PyTorch with advisory PYSEC-2026-2286, fixed in torch 2.10.0; the post recommends upgrading before non-proof-of-value use.

#text-to-speech #aws sagemaker #websockets #ai agents

Generated by gpt-5.6-sol. Check the original for details.

Aliases

SourceExternal IDURL
AWS Machine Learning Blog351e07d8236b8a05fa15e510dcb5b937d7a9dec8https://aws.amazon.com/blogs/machine-learning/stream-low-latency-speech-from-ai-agents-using-kokoro-tts-on-amazon-sagemaker-ai/ Read discussion