Amazon shows GPT-OSS-20B can learn to reason in French, but only after SFT

Amazon Web Services has published a walkthrough on fine-tuning GPT-OSS-20B to reason in French, German, Spanish and Italian. Supervised fine-tuning took correct-language reasoning from 22% to 100% on AWS’s 50-case test.

The problem is easy to reproduce, according to AWS. GPT-OSS-20B is an open-weight reasoning model that thinks in English. Told to reason in another language, it acknowledges the instruction and then writes its chain of thought in English anyway. On AWS’s test, the base model reasoned in the requested language 22% of the time, and that figure came almost entirely from the English cases: English scored 100%, French and Spanish 0%, German 10% and Italian 0%.

SFT does the heavy lifting, GRPO adds a few points

AWS ran its experiments on Amazon SageMaker HyperPod with Amazon EKS orchestration. The data came from the Hugging Face dataset HuggingFaceH4/Multilingual-Thinking, roughly 949 training and 50 validation examples. The evaluation covered 10 math and logic problems in each of five languages.

The supervised run used LoRA adapters (r=8, alpha=16) for 1,000 steps on four ml.g6e.12xlarge instances, 16 Nvidia L40S GPUs in total. AWS says the model learned to reason in the target language by around step 200, and to answer in it by around step 1,000. At the end, reasoning-language accuracy was 100% (50/50), but answer-language accuracy was only 62% (31/50). Spanish was weakest at 30%, and the other non-English languages sat at 60%.

AWS then applied Group Relative Policy Optimization (GRPO) with TRL’s GRPOTrainer, starting from the SFT checkpoint. It used eight generations per prompt and a reward that weighted answer language most heavily. Reasoning stayed at 100% and answer-language accuracy rose to 70% (35/50), an 8-point gain. By language: French 70%, German 80%, Spanish 60%, Italian 40%. AWS notes the metric peaked early in the run, so users should pick a checkpoint against held-out data.

That sentence, from AWS’s post, is the rule the rest of the results are built around. GRPO samples a group of responses per prompt and pushes the model toward the better ones, so if the desired behavior has essentially no probability to begin with, there is little for it to reinforce.

A Qwen control that tests the rule

To check whether the pattern holds elsewhere, AWS ran an internal experiment on Qwen2.5-7B-Instruct using veRL, Ray, KubeRay and vLLM on two g6e.48xlarge instances. On the multilingual task, reasoning-language accuracy went from 0% for the base model to 96% with SFT and 98% with SFT plus GRPO. Answer-language accuracy went from 0% to 38% to 44%.

On GSM8K, where base Qwen already scores 90.5% (181/200), GRPO at step 30 scored the same 90.5%. AWS’s explanation is that when nearly every sampled group is all-correct, the advantage signal collapses to zero. Its advice is to check that the model’s best-of-k results are well above its single-attempt results before paying for a GRPO run.

What the numbers can and cannot support

The evidence is narrow. The test has 50 cases, 10 per language, so one example moves a language score by 10 points and the 8-point overall gain equals four examples. AWS reports a single run per configuration and gives no confidence intervals. The Qwen experiment is described as internal, and AWS says its repository has not been released. All figures are the company’s own and have not been independently checked.

The training data is also small and synthetic. According to its Hugging Face page, Multilingual-Thinking has 1,000 training rows, built by sampling from the SystemChat subset of SmolTalk2 and having another language model translate the reasoning traces into four languages. It is licensed under Apache 2.0, the same license OpenAI used for the gpt-oss family, which we covered at launch. The 20B model was pitched then as able to run in 16GB of memory, which is why a LoRA adapter touching about 0.068% of its parameters (13.6M of 20B, per AWS) is a practical route for teams that want a language-specific variant.

For readers who want to try it, the scripts and Kubernetes manifests are in the awsome-distributed-ai repository under 3.test_cases/pytorch/trl/gpt-oss-lora-grpo/. The setup assumes a HyperPod cluster with EKS, an FSx for Lustre file system, EFA-enabled instances and a Hugging Face token. AWS gives no cost figures, so the price of reproducing the work is not established. Answer-language accuracy of 70% is also still well short of a model a French-speaking customer could rely on, and the post does not report whether answer quality or math accuracy changed after training, only language compliance.

The decision rule AWS offers is simple: use SFT for behavior the model cannot do, SFT then GRPO for behavior it does sometimes, and skip GRPO when the model is already at its ceiling or the target has no base probability.