Amazon replaces Claude Sonnet 4.6 with Strands Decider for ticket labeling at 98% lower cost
- Amazon engineers swapped Claude Sonnet 4.6 for Strands Decider (the E4B Gemma 4 build) on an internal ticketing system that sorts tickets into 20 categories, cutting token cost by 98% and saving millions of dollars a year.
- Strands Decider is a decision model, not a text generator: it takes state, a question and a set of options, then returns a probability for each option plus a confidence score.
- The E4B build beat production Sonnet on a separate set of tickets after the team tuned the confidence threshold required before a label is applied; they had started with the smaller Strands Decider 2B.
- E4B ran on SageMaker ml.g5.xlarge instances (one A10G, 24 GB), used about 19 GiB, and answered one request in about 640 ms, with the endpoint autoscaling on traffic inside an isolated network.
- Commenters estimate the claimed savings imply at least 92 million tickets a year at roughly $21,700 saved per million, and an Amazon employee says a single small org there handles over 100,000 tickets a month.
Hacker News 의견들
If their model costs 98% less than Sonnet and Amazon is saving millions, these folks are raking in 2% of millions. Congrats I guess?
You're missing that Strands is an open source model AWS released, not some external startup Amazon is paying for.
Why didn't they just upgrade to a newer Sonnet?
If they're running on Bedrock, no batch API support for recent Anthropic models would be my guess.
Six years ago we built a ticket classification pipeline on SVMs. Training took a couple of minutes, classification took milliseconds. Watching the field go to LLMs and back to cheap classifiers is fascinating. The paper was Linnaeus, accepted at ICSE 2021.
There is zero reason to use an LLM as a classifier. It's 1000x slower and 1000x more expensive, but people with no technical knowledge push for it anyway.
Building an SVM isn't tricky, but you need a data scientist and a carefully labeled dataset. LLMs are the electron apps of modeling: they waste resources but give you an easier development experience.
What changed is that pre-LLM most projects died at data collection and problem definition. With LLMs you can iterate on 20 ticket categories in days against real traffic, then train a cheap model once the prototype is proven.
At about $21,700 saved per million tickets, saving millions a year implies at least 92 million tickets annually. Plausible once you count alarms, security findings and capacity requests.
I work for a small org inside Amazon and we do more than 100,000 tickets a month. There are way more than 92M tickets a year, tickets get cut for basically everything administrative.
This will pop up more often. Software engineering learned long ago that giant monoliths aren't the answer, you want smaller specialized units you can upgrade independently.
I've been using a Rust crate called bonsai to build logic trees around cheap models, way more control and better results. Not sure how that beats just running a small decision model internally though.