Oriel — AI & technology worth noticing, every day.

Fine-tuning GPT-OSS-20B for multilingual reasoning on Amazon SageMaker HyperPod

Web page · https://aws.amazon.com/blogs/machine-learning/fine-tuning-gpt-oss-20b-for-multilingual-reasoning-on-amazon-sagemaker-hyperpod

Learn how we fine-tuned GPT-OSS-20B for multilingual reasoning on Amazon SageMaker HyperPod, and how to decide between supervised fine-tuning (SFT) and reinforcement learning with GRPO. SFT took reasoning-language accuracy from 22% to 100%, and GRPO added measurable gains where headroom existed.

First seen by Oriel on .

AI summary

The post fine-tunes GPT-OSS-20B for multilingual reasoning on Amazon SageMaker HyperPod, finding that SFT establishes missing language behavior while GRPO improves answer-language accuracy when the model has room to improve.

  • On a 50-case evaluation, base GPT-OSS-20B reasoned in the requested language 22% of the time; LoRA SFT raised this to 100% and achieved 62% answer-language accuracy.
  • Adding GRPO to the SFT checkpoint increased answer-language accuracy from 62% to 70%, while preserving 100% reasoning-language accuracy.
  • The setup used Amazon SageMaker HyperPod with Amazon EKS, TRL, LoRA, Amazon FSx for Lustre, and Elastic Fabric Adapter; Qwen control experiments used veRL, Ray, KubeRay, and vLLM.
  • In the Qwen2.5-7B control, SFT+GRPO modestly improved multilingual metrics, but GRPO did not improve GSM8K beyond the base model's 90.5% accuracy.
  • The authors recommend SFT for behaviors the model lacks and GRPO for refining behaviors it already produces sometimes; GRPO needs measurable headroom to provide a learning signal.

#large language models #supervised fine-tuning #grpo #amazon sagemaker

Generated by gpt-5.6-sol. Check the original for details.

Aliases

SourceExternal IDURL
AWS Machine Learning Bloge95a562fb3248db86864f803ae6d83c71f277e90https://aws.amazon.com/blogs/machine-learning/fine-tuning-gpt-oss-20b-for-multilingual-reasoning-on-amazon-sagemaker-hyperpod/ Read discussion