Fable 5 thinking tokens fell sharply in August after Anthropic opened the model to subscription plans, six-week measurement finds
- @Lon measured five different ways and found Fable 5 produced dramatically fewer thinking tokens in August than in July, starting right after Anthropic made the model permanently available in subscription plans. The post drew 1,277 likes.
- Even while running at xhigh or max effort, most invocations to the model received little to no thinking tokens, and the longer thinking runs that did happen almost never reached published benchmark levels.
- The drop was not one-time: reasoning fell across the whole six-week data capture and moved in multi-day episodes, some of which lined up with specific product announcements and releases.
- His recommendation is to stop asking whether the model was nerfed and instead ask about the inference regime you were served, since day-to-day quality tracked those regimes.
Hacker News opinions
Anthropic has gone on record saying they don't touch model performance to stretch compute. I've seen big week-to-week swings myself, though with Opus rather than Fable. I'd like to hear their official explanation for this.
Last time people called this out, the explanation was a regression in Claude Code itself. Either way it wasn't a good look for vibecoding, and it got brushed over.
Part of it is novelty wearing off. The same capabilities feel mundane after a while and you start noticing the flaws. But if he's actually measuring reasoning tokens, that's a different claim, and I can't tell how he measures them.
They ship optimizations weekly, sometimes daily, with a pile of AB tests. I don't think they manipulate model quality on purpose, but they are definitely running experiments on live users.
They did walk it back before. Wired covered the policy change that could have sabotaged Claude for AI researchers, and there's still an open GitHub issue about the same behavior.
How do you even measure thinking tokens? They don't send those back to the client.
They do report how many tokens you used, otherwise you couldn't track your own consumption.
This is the standard cycle. Model X is basically AGI and wins every benchmark, the next day Y and Z win them all too, then weeks later people find out it got quantized or served with worse inference. Buy decent coffee instead of the $200 subscription.
You skipped a couple of stages: end of all things, then basically AGI, then limited release, then the 20 tier, then Google shoving it into every search whether you want it or not.
I like coffee and my $200 AI plan. If Blue Bottle watered down its coffee, would the answer be to switch to tea? Evidence that vendors mislead about what they deliver is worth sharing either way.
Anecdotally it's the same for me. Week 1 to week 8 the drop is massive. It starts as a capable research assistant and ends up a puppy eager to please its master for treats.
Reminds me of slot machine players swearing the odds changed. The providers design the UX like slot machines too, run it more times for better results, and that multiplies your spend.
How do you build repeatable tests in a non-deterministic system? Same prompt, different answer every time.
The tokens are non-deterministic, but you can use proxy measures that should be invariant, like benchmark correctness or thinking depth on hard problems. Anthropic has an engineering post on demystifying evals.
I think the real Fable was what we had for a few days in June. They nerfed it after the government pulled it off the market, and what we have now is less.
Anthropic is clearly looking for automatic degradation to save money on requests it thinks don't need much reasoning. I run max reasoning and I can see the difference 3 to 4 weeks after release, and I suspect new accounts get an intelligence boost.
I made gpt-5.6-luna my default in Hermes during launch week and it was as good as 5.5. Now I have to be explicit about everything. I asked whether a service finished its overnight job and it told me the service isn't installed on my host, I had to tell it to ssh in and run journalctl. Same medium reasoning level the whole time.
Same experience with Fable and Sol, daily for hours. The last two weeks both are nearly on par with DeepSeek4.1, still better but the frustration is the same. My friends and I are looking at an Ultra M5 256 to split the cost.
I'm fairly sure it's luck of the draw whether you land on a quantized model. I've watched luna xhigh change intelligence from one day to the next.
Is this only hitting people in certain areas like AI or distributed systems? I work on distributed systems and today Fable is mostly unusable, it feels like Opus.
Embedded systems here, same thing day by day with Opus. Some days it's fine, other days I have to repeat information already in the prompt before compaction. It's a slot machine behind opaque paywalls, and I'm on a business plan.