AMD squeezes 1.72x more throughput from MI355X for coding agents

AMD says it raised peak per-GPU throughput for GLM-5.2 on Instinct MI355X by 1.72x, to 18,327 tokens/s, on the AgentX coding-agent benchmark, according to an engineering post on the ROCm blog.

Two sweeps, two peak numbers

The post compares a baseline sweep from August 11 with a final sweep from October 6. In the baseline, the best per-GPU throughput was 10,625 tokens/s (input and output combined), reached with 10 active clients. In the final sweep, the best was 18,327, reached with 16 clients. Peak P90 interactivity, measured in tokens per second per user, rose 2.46x, from 110.25 to 271.00, at a single client.

AMD is explicit about the limits. These are two deployed configurations, not a controlled A/B test and not an official InferenceX aggregate score. GPU allocation also changed, from 16 to four. The model is GLM-5.2 in MXFP4, a 4-bit microscaling format, served with the open-source SGLang framework.

AMD also plots the nine measured concurrency points against Nvidia reference runs. Using its own polygon-area index, with the September 2 B200 reference set to 100, MI355X went from 35 to 123, while the selected B300 reference sits at 132. AMD says the index is its own visualization, not an InferenceX metric, and that the references and MI355X snapshots were taken on different dates.

What AgentX does to a server

AgentX replays coding sessions in a closed loop, so a client sends its next request only after the last response completes. Per AMD, the v1.0 corpus has 393 sessions reconstructed from Claude Code usage that users agreed to share. The median request carries about 142,000 input tokens and 444 output tokens, and 44% of sessions include subagents. The replay uses synthetic payloads and measures serving speed, not answer quality.

That workload leans on key-value (KV) cache. Above concurrency 12, AMD says the active KV working set approached GPU cache capacity. Its fix was HiCache, SGLang’s hierarchical cache, which offloads KV data to host memory. ROCm-specific changes made the host-transfer path work for the FP8 cache rows used here. Below that point, faster decode matters more because finished turns release cache pages sooner.

The published recipe uses TP8/EP1 with GPU-resident KV at three low-load points. It switches to TP4/EP4 with HiCache offload at six higher-load points. It also uses FP8 KV cache and EAGLE multi-token prediction with six draft tokens.

A fused indexer and a kernel found with agents

One merged SGLang change replaces a twelve-launch decode chain in the DeepSeek Sparse Attention indexer with four HIP kernels. It applies only on gfx950 hardware with AITER preshuffle and the GLM-5.2 indexer shape. In a controlled AgentX A/B run at concurrency 1, the pull request reports P90 inter-token latency falling from 3.81 ms to 3.56 ms, or 6.7%.

AMD used its HyperLoom coding-agent tooling on indexer and MLA kernel work, with engineers choosing changes and checking numerical behavior. On one representative decode shape, kernel latency fell from 91.24 microseconds to 35.56, or 2.57x faster. That took about 16 hours of effective working time across eleven candidate stages, according to AMD’s internal record. The team says this microbenchmark is separate from the serving results.

Reading the claims against AMD’s recent record

The headline figure is a combined result of many merged pull requests. AMD lists more than 20 across operators, graphs and HiCache, and says their effects cannot be summed. The only isolated gain it reports is the 6.7% latency change, which is small next to the 1.72x headline. Most of the throughput gain therefore likely comes from cache offload and the TP4/EP4 setup, though the post does not split it out.

The work fits a pattern in AMD’s software push. Last week we reported that AMD trained DeepSeek-V4 Flash with reinforcement learning on 24 MI308X GPUs, lifting its AIME score to 49.2%. The commercial stakes are visible in the company’s numbers: data center sales nearly doubled to $8.5B, with fourth-quarter guidance above $16B, as we covered on November 3. Serving long-context agent workloads efficiently is the kind of evidence buyers weigh when comparing accelerators.

The public results rest on a process rule worth noting. The InferenceX team at SemiAnalysis required every sweep to use a publicly released serving image, with all parameters and environment variables documented in the recipe. The October 6 curve uses the September 30 image, so later merged code is not included. Anyone with MI355X hardware can in principle reproduce the configuration from the public recipe and the SGLang GLM-5.2 cookbook.

What the post leaves open

The comparison with B200 and B300 is AMD’s own visualization, built from runs taken on different dates, so it is not a head-to-head. AMD does not give cost per token or power figures. It also says the concurrency 12 boundary is specific to this model and configuration, not a general MI355X limit. Because AgentX does not score answer quality, the post says nothing about whether outputs changed, beyond mentioning accuracy checks in its one-hour confirmation runs.

AMD says the next steps are cutting HiCache transfer cost at high concurrency and tuning the slowest prefill and decode kernels, with measured AgentX results promised in a later update.