Cost-optimizing AssemblyAI speech-to-text inference on AWS with Amazon EC2 Spot
Learn how AssemblyAI runs cost-optimized speech-to-text inference at scale on AWS, using Amazon ECS, Amazon EC2 Spot GPU instances, and an Amazon SQS-based retry architecture to achieve 70% cost savings while maintaining SLA commitments across European Regions.
First seen by Oriel on .
AI summary
This AWS post describes AssemblyAI’s resilient GPU inference architecture using ECS, diversified EC2 Spot capacity, and SQS retries, achieving 70 percent inference compute savings while maintaining SLA commitments.
- AssemblyAI mixes On-Demand baseline capacity with Spot capacity across g4dn, g5, and g6 GPU instance families and Availability Zones.
- Amazon ECS managed scaling expands or reduces GPU capacity according to capacity provider weights and pending inference tasks.
- An EventBridge-triggered Lambda returns interrupted jobs to an Amazon SQS queue, allowing another GPU task to retry them.
- Custom AMIs containing model artifacts and container images can warm up in approximately 2 minutes, reducing interruption recovery time.
- Independent regional deployments in us-west-2 and eu-west-2 support data residency requirements for regulated customers.
#amazon ecs #ec2 spot #amazon sqs #gpu inference
Generated by gpt-5.6-sol. Check the original for details.
Aliases
| Source | External ID | URL |
|---|---|---|
| AWS Machine Learning Blog | 62c3e92270a83395aaeaae770a977c92ced34043 | https://aws.amazon.com/blogs/machine-learning/cost-optimizing-assemblyai-speech-to-text-inference-on-aws-with-amazon-ec2-spot/ Read discussion |