AMD trained DeepSeek-V4 Flash with RL on 24 MI308X GPUs, lifting AIME to 49.2%

AMD researchers have run a full reinforcement learning loop for DeepSeek-V4 Flash on 24 Instinct MI308X GPUs, according to a post on the ROCm blog. Over 75 optimizer steps, held-out AIME-2024 accuracy rose from 30.0% to 49.2%.

Three nodes, split between training and generation

The test used three eight-GPU MI308X servers. Two nodes host a BF16 actor, the copy of the model that is trained, running as 16 Megatron workers. The third node runs the rollout engine, a vLLM server with eight-way tensor parallelism that generates answers from an FP8 version of the weights. Ray places and coordinates the two groups.

AMD says the split follows the memory needs of each job. The BF16 actor keeps the numerical precision that gradient and optimizer updates need, while the smaller FP8 weights leave more GPU memory for the KV cache, which holds the context of long generations. The actor uses tensor, pipeline and expert parallelism of 4 each, with optimizer state offloaded to CPU memory. The rollout side runs at 90% GPU-memory utilization, which AMD says leaves room for online policy updates.

The model is the 284-billion-parameter mixture-of-experts design, with roughly 13 billion parameters active per token, as described in DeepSeek’s technical report. The run capped context and sequence length at 16,384 tokens, with responses limited to 4,096 tokens, far below the one-million-token window the model supports.

Each step sampled 32 prompts from the DAPO-Math-17K dataset and generated eight responses per prompt, for 256 trajectories. Because rollout and training use different number formats, the workflow applies a token-level correction capped at 2.0, and it replays the recorded expert routing during actor rescoring. After each update, the new weights travel over RDMA to the rollout node, which converts them to FP8 before generation resumes. Checkpoints are saved every 25 steps.

Reward and AIME gains, with caveats AMD itself states

Average reward accuracy climbed through the run: about 45% across the first 25 steps, roughly 53% in the middle stretch and about 59% in the final 25, although single steps swung between 26.6% and 77.7%. AIME-2024 accuracy, averaged over eight generations for each of the 30 problems, went from 30.0% to 49.2%. AMD notes that individual evaluations on a 30-problem benchmark are noisy, so the trend matters more than any one point.

Rollout throughput averaged 311 generated tokens per second, in a range of 287 to 331, drifting from 315.2 in the first third of the run to 308.6 in the last.

AMD also flags a drift it is still working on. By step 75, the KL divergence between training and rollout engines was 0.4706 and effective sample size was 79.1%. The company calls these calibration signals for improving training-inference consistency, especially for long sequences and online FP8 reloads.

The published setup is aimed at reproduction. AMD lists pinned images on Docker Hub, rocm/atom-dev:rl-dsv4-308-train and rocm/atom-dev:rl-dsv4-308-rollout, and says the model, datasets, checkpoint storage and RDMA settings must be supplied by the user. The BF16 model weights come from the RedHatAI/DeepSeek-V4-Flash-BF16 repository on Hugging Face. The experiment ran from August 17 to 21, 2026, on Supermicro AS-8125GS-TNMR2 servers with ROCm 7.2 and vLLM 0.25.1.

What developers can use, and what the post leaves out

For teams that train or fine-tune open-weight models, the practical content is a worked recipe on hardware other than Nvidia’s. The post covers a smoke test of three optimizer steps, a check that ray status reports 24 GPUs, and the parallel layouts to match. It does not offer a managed service or a price, and it does not give a cost per step or a time-to-train figure.

The results are AMD’s own and are not independently verified. AMD stresses that they apply only to this configuration and software stack. Seventy-five steps is a short window, and the post makes no comparison with Nvidia hardware or with other MI-series chips. The 311 tokens per second figure is also an aggregate for the whole rollout node, so it is not a per-GPU or per-user speed. Nor does the post say whether AIME gains continue beyond step 75.

The model matters beyond this test. DeepSeek’s current API lists a 1M-token context and 384K output tokens, with Flash priced at 1 yuan per million input tokens on a cache miss at off-peak rates, according to our guide to the DeepSeek API. Showing that a model of this class can be post-trained on MI308X gives AMD a reference point for customers who want to adapt it themselves.

A software-first push alongside AMD’s big deals

The post lands a day after AMD said it would put $1 billion into Meta’s equity offering while Helios racks are being tested at Meta. That announcement concerned hardware supply at gigawatt scale. This one is about software: the ROCm, AITER and vLLM pieces that decide whether customers can run training, not just inference, on Instinct GPUs. Nvidia, for its part, disclosed a $3 billion investment in the same Meta offering the day before.

The MI308X is a different class from the Helios generation, so this experiment says little about AMD’s newest racks. It does show AMD documenting a multi-node RL workflow in enough detail for outsiders to try, which is the kind of proof developers tend to ask for before moving workloads.