Fine-tuning GPT-OSS-20B for multilingual reasoning on Amazon SageMaker HyperPod
Learn how we fine-tuned GPT-OSS-20B for multilingual reasoning on Amazon SageMaker HyperPod, and how to decide between supervised fine-tuning (SFT) and reinforcement learning with GRPO. SFT took reasoning-language accuracy from 22% to 100%, and GRPO added measurable gains where headroom existed.
First seen by Oriel on .
AI summary
The post fine-tunes GPT-OSS-20B for multilingual reasoning on Amazon SageMaker HyperPod, finding that SFT establishes missing language behavior while GRPO improves answer-language accuracy when the model has room to improve.
- On a 50-case evaluation, base GPT-OSS-20B reasoned in the requested language 22% of the time; LoRA SFT raised this to 100% and achieved 62% answer-language accuracy.
- Adding GRPO to the SFT checkpoint increased answer-language accuracy from 62% to 70%, while preserving 100% reasoning-language accuracy.
- The setup used Amazon SageMaker HyperPod with Amazon EKS, TRL, LoRA, Amazon FSx for Lustre, and Elastic Fabric Adapter; Qwen control experiments used veRL, Ray, KubeRay, and vLLM.
- In the Qwen2.5-7B control, SFT+GRPO modestly improved multilingual metrics, but GRPO did not improve GSM8K beyond the base model's 90.5% accuracy.
- The authors recommend SFT for behaviors the model lacks and GRPO for refining behaviors it already produces sometimes; GRPO needs measurable headroom to provide a learning signal.
#large language models #supervised fine-tuning #grpo #amazon sagemaker
Generated by gpt-5.6-sol. Check the original for details.
Aliases
| Source | External ID | URL |
|---|---|---|
| AWS Machine Learning Blog | e95a562fb3248db86864f803ae6d83c71f277e90 | https://aws.amazon.com/blogs/machine-learning/fine-tuning-gpt-oss-20b-for-multilingual-reasoning-on-amazon-sagemaker-hyperpod/ Read discussion |