Lend and borrow resources across teams with AWS Batch’s quota management for SageMaker Training jobs
AWS Batch quota management is a new scheduling capability you can use to divide GPU capacity into quota shares per team or project, automatically lend idle capacity across shares, and use preemption to run your highest-priority SageMaker Training jobs first, all without manual coordination.
First seen by Oriel on .
AI summary
This post demonstrates AWS Batch quota management for SageMaker Training jobs, enabling teams to share GPU capacity through configurable quotas, lending, borrowing, and priority-based preemption.
- Administrators define quota shares for teams, projects, or workloads, each with capacity limits and RESERVE, LEND, or LEND_AND_BORROW strategies.
- AWS Batch automatically lends idle capacity, allows eligible shares to borrow it, and reclaims borrowed resources when the owning share receives work.
- Optional in-share preemption lets higher-priority jobs preempt lower-priority jobs in SCHEDULED, STARTING, or RUNNING states.
- The example creates three quota shares, each limited to one ml.g5.xlarge instance, using SageMaker Python SDK v3 and AWS Batch resources.
- QS1 demonstrates borrowing and preemption, QS2 lends without in-share preemption, and QS3 reserves capacity without lending or borrowing.
#aws batch #sagemaker #gpu scheduling #machine learning
Generated by gpt-5.6-sol. Check the original for details.
Aliases
| Source | External ID | URL |
|---|---|---|
| AWS Machine Learning Blog | d83c80cde37528e374033f116e48863ebb80f55e | https://aws.amazon.com/blogs/machine-learning/lend-and-borrow-resources-across-teams-with-aws-batchs-quota-management-for-sagemaker-training-jobs/ Read discussion |