Amazon warns its engineers against tokenmaxxing in new AI guidelines

Amazon says token usage, AI tool adoption and counts of code changes are “deeply flawed” ways to measure its engineers’ work with AI. The company published the warning on Tuesday in a memo against tokenmaxxing. It came with an updated role guideline for its software development engineers.

Terk Terkowitz, vice president and distinguished engineer for Worldwide Amazon Stores, signed the memo. Amazon first shared it with staff in September, CNBC reported.

“If your work does not benefit from current AI tooling, that is a finding, not a failing.” Terk Terkowitz, Amazon

Measures, not targets

Amazon has used input metrics such as token usage and AI tool adoption, according to the memo. It has also counted code changes and reviews. If teams treat these numbers as targets rather than measures, costs and complexity could rise, the memo says.

The memo frames this in terms of Amazon’s leadership principles. It says “tokenmaxxing is at odds with frugality”, and that measuring lines of code works against “invent and simplify”.

Tokenmaxxing is the practice of using AI as much as possible with little regard for output. Amazon shut down an internal leaderboard in May after finding that employees were tokenmaxxing to climb the ranks, CNBC reported. An Amazon spokesperson told CNBC the company has not capped employees’ token usage.

Amazon tracked engineers’ AI usage, token consumption and lines of code, and used leaderboards to rank some teams, Business Insider reported. Those programmes created friction among some employees, according to the outlet.

AI stays at the centre

The memo does not tell engineers to drop the metrics. It says they correlate well with the outcomes that matter, and that engineers should use them “with care and judgement”.

Teams that have embraced AI the most, which Amazon calls frontier teams, show higher productivity and delivery, the memo says. Teams and individuals that are not adopting AI are falling behind.

The memo asks managers to study the bottom 10% of teams by AI adoption and find out why. They should also check whether the top 10% deliver more to match.

Amazon expects leaders to use the tools on production work themselves. They should be able to describe one task where AI helped and one where it did not.

“We are certain that whatever future best practices are, AI will be at the center of software engineering practice.” Amazon, in the memo

Marc Brooker is a vice president and distinguished engineer at Amazon Web Services. He told CNBC that Amazon has looked at measures such as “feature velocity”. That covers how many launches teams make and how many successful changes reach production.

“What we don’t want to see is wasted effort where we are going and spending those tokens solving problems that don’t matter.” Marc Brooker, AWS, to CNBC

Other companies have moved on AI spending this year. Microsoft set division-level AI budgets in August, and Uber’s tech chief said the tokenmaxxing era is ending. Accenture capped staff Claude spending at $29 a month this month, 404 Media reported.