Google trains a small Gemma 4 model to win 95% of Plane Strike games
In a Google tutorial, 100 steps of multi-turn reinforcement learning on one TPU v6e-8 host took Gemma 4 E2B from 9.4% to 95% wins on unseen Plane Strike boards.

Google’s developer team has published a tutorial on multi-turn agentic reinforcement learning with its Tunix library. It trains Gemma 4 E2B to play a tool-calling board game, lifting its win rate on unseen boards from 9.4% to 95% in 100 steps.
One TPU host, 100 steps, a board game
The post, written by Google developer advocate Wei Wei and dated November 19, 2026, follows two earlier Tunix guides on supervised fine-tuning and single-turn GRPO (Group Relative Policy Optimization). This one targets agents that have to act over several turns, use tools and react to feedback.

The test case is Plane Strike, played on a 6×6 grid from A1 to F6. A hidden 8-cell plane sits in one of 4 orientations. On each turn the model sees the result of its last strike plus the visible board, reasons in thinking mode, and calls strike(cell="..."). A game ends when all 8 cells are hit or after 23 strikes.
Everything runs on a single Cloud TPU v6e-8 host. According to Google, the untrained google/gemma-4-E2B-it wins only 9.4% of held-out games, averaging 22.44 strikes. After 100 GRPO steps the win rate is 95% and games last about 16 strikes.

Win rate climbs from 9.4% to 95% on held-out boards after 100 steps
Four components, and the settings that matter
Google splits the pipeline into four parts: a task environment (BaseTaskEnv in env.py), a tool and chat-template parser (strike_tool.py), a conversation agent (agent.py), and the trajectory collection engine with the GRPO learner (train_planestrike.py).

Each training step uses 8 boards with 8 games per board. A seed fixes the hidden plane, so all 8 rollouts in a group face the same board. Rewards from each strike are summed into one reward for the whole game.
Several configuration choices are called out. The rollout engine runs in vLLM server mode on TPU with the JAX backend, so variable-length games generate asynchronously, and prefix caching keeps the growing conversation history cheap to reuse. The config sets loss_algo="gspo-token", advantage_estimator="rloo" and exact_token_continuity=True, which Google says preserves exact token IDs across turns so retokenization does not cause drift. Because the host has only 8 chips, the actor, the reference model and the rollout workers share them.
Training is started by running train_planestrike.py with a run directory and a learning rate that decays from 2e-6 to 2e-7 over the run. The code is in the planestrike-gemma4-rl GitHub repository, and Tunix is documented publicly.
Why Gemma 4 tool calls need special handling
The technically interesting part is how Gemma 4 calls tools. The model declares tool schemas in the system prompt and, when it calls one, stops at a native <|tool_response|> token. The environment’s reply is then inserted as a tool message, and the model continues the same assistant turn. Ordinary chat templates and rollout code assume one prompt and one answer, so Google had to subclass the Tunix parser to render these in-turn responses and to stop generation at the right token.
Only a final answer containing exactly one valid strike call counts. Reasoning inside the thought channel is ignored, and anything else returns INVALID_ACTION. Google’s training curves show tool-call compliance converging first: the tool-call rate reaches about 100%, invalid actions fall to near zero, and repeated strikes drop from about 0.55 to 0.15. Spatial deduction improves after that.
The hardware context comes from Google’s own recent announcement. In its November 9 post on the TPU 8tx and 8i Max, Google said the earlier TPU 8t and 8i systems offer up to 2.7x better training performance and 80% better inference price performance than the previous generation. It also said the 8tx has up to 2x lower latency than the 8t. The Plane Strike run uses the older v6e, so those figures do not describe this result, but they show where Google is putting its agent-focused TPU work.
What the result does and does not establish
Anyone with Cloud TPU v6e-8 access can reproduce the run from the published code, and the pipeline is built from Tunix’s own tunix.rl.agentic classes, with ready-made ToolEnvironment, ToolAgent and ToolManager abstractions for standard tool-using agents. The post gives no TPU cost or training time, so the price of reproducing it is unknown.
The numbers are Google’s own and have not been independently checked. Plane Strike is a small, fully observable puzzle with a single tool, so the 95% figure says little about messier agent work such as browsing or coding. The win rate on training rollouts starts near 0.20, against 9.4% on held-out boards, and the post does not explain that gap. Google also notes that sharing one reward across all strikes in a game is a limit, and says finer per-turn credit assignment could improve results further.
For developers, the value is mostly the recipe: a working example of multi-turn RL on Gemma 4 with native tool calling, a TPU-friendly rollout setup and a small model that runs on one host.
Sign Up for the Daily AI Pulse
One email a day. All the stories that matter.
By signing up, you agree to our Terms of Use and acknowledge the data practices in our Privacy Policy. You may unsubscribe at any time.
Alex CarterAI News & Big Tech CorrespondentAlex Carter writes for Aiholics, keeping readers updated on the fast-paced world of AI and Big Tech.
You may also like

Reward hacking: How AI models learn to win the test without doing the task

Google Cloud Storage can now tag your images and PDFs for AI agents, in preview

Google’s Project Genie now lets you type changes into a running world
