Amazon tells engineers that not using AI can be a finding, not a failing

Amazon has told its software engineers that counting tokens or generated code is a “deeply flawed” way to judge AI adoption, CNBC reported. The memo, circulated internally in September, was posted to Amazon’s website on Tuesday, according to CNBC.

We could not locate the posted memo itself, so everything below comes from CNBC’s account and its quotations.

What the memo tells engineers and managers

CNBC says the document revises how Amazon defines the roles of software development engineers and their managers in the age of AI. Amazon wrote that the company is “just seeing the beginning of AI’s effect on software engineering”, and that AI will be at the center of the practice whatever the eventual best practices turn out to be.

On measurement, CNBC reports that Amazon names three metrics as flawed gauges of a team’s or organization’s AI use: token usage, the rate at which AI tools are adopted, and the number of code changes or reviews. Amazon says they can raise costs and add complexity. The memo does not ask staff to ignore them. Token usage and code generation should be weighed “with care and judgment” next to other data.

The preferred yardstick is outcomes. “We believe that it is best to measure, and be measured by, long-term outcomes,” the company wrote, per CNBC.

Pickier use, with room to say AI does not help

The memo also tells staff not to use AI where it does not suit the task. “If your work does not benefit from current AI tooling, that is a finding, not a failing,” Amazon said. An engineer who can explain where AI does not yet help is, in its words, showing the judgement the company wants.

Marc Brooker, a vice president and distinguished engineer at AWS, told CNBC that Amazon looks at “feature velocity”, meaning launches and successful production changes. He framed AI spending as return on investment and said Amazon wants to avoid tokens spent on problems that do not matter, while keeping some appetite for experimentation.

A reversal of the earlier AI push

The memo follows months of reporting on tokenmaxxing, where developers use AI heavily with little regard for the result. CNBC notes Amazon shut down an internal leaderboard in May after finding employees were gaming it to climb the ranks. Search results from earlier this year also point to a leaked memo from a senior vice president, reported by the Financial Times, that asked staff to stop using AI just for the sake of using it. We did not open those reports, so treat that detail as unverified here.

An Amazon spokesperson told CNBC that the company has not capped employees’ token usage. CNBC adds that other companies have set limits or moved some tasks to cheaper open-source models to slow rising AI bills.

What the reporting does not establish

The outcome metrics are described only in general terms. The reporting gives no thresholds, no cost figures and no detail on how managers will weigh token data against other signals. Whether the guidance changes how engineers are rated is not stated, so the effect on day-to-day practice remains unclear until the full memo can be read.