Cognition's SWE-2 claims near-Fable coding scores at 64% lower cost
- Cognition released SWE-2, a coding model available in Devin Desktop and CLI, reporting 50.0% on FrontierCode 1.1 Main, within 0.9 points of Fable 5.1 while costing 64% less per rollout
- SWE-2 is post-trained from the 2.8T-parameter Kimi K3 using reinforcement learning that trains multiple reasoning-effort levels in one run, with linear cost penalties tuned to the base model's cost-performance curve
- On FrontierCode 1.1 Main, SWE-2 medium used 53 mean steps versus SWE-1.7's 127, made its first code edit after a median 18 steps versus 48, and Cognition reports 81% lower average cost
- Cognition reports SWE-2 scores of 73.0% on DeepSWE 1.1, 92.8% on Terminal-Bench 2.1, and 27.3% on Terminal-Bench 4, compared with Kimi K3's 68.5%, 88.3%, and 21.5%
- Training changes include tripling RL environments, adding instruction-following overlays, using prior SWE-2 checkpoints to harden verifiers, and improving rollout scheduling and draft-model decoding throughput
Hacker News opinions
Terminal-Bench 4 is the result I care about, and 27.3% is not impressive. Cognition made FrontierCode itself, so I want to see more independent benchmarks before taking the comparison seriously.
I used SWE-1.5 and found it surprisingly good. Cognition seems like a solid player that gets less attention while Anthropic and OpenAI dominate the headlines.
Since SWE-2 is post-trained from Kimi K3, I expect performance to be spiky on SWE work. It is derived from an already heavily trained model, though 64% cheaper could still make it worthwhile.
I expected a new base model, but RL-tuning Kimi K3 up toward Fable 5 capability shows that this route can work.
I do not want to use a bespoke Devin platform just to try the model, so I probably will not use it.
I work at Cognition, though not on SWE-2. It is free for the next month, and the CLI should support almost all normal usage.
Whether the weights are open matters a lot. Strong enough performance would make adapting existing tools worthwhile.
The 92.8% Terminal-Bench 2.1 result versus 27.3% on Terminal-Bench 4 is a huge gap. TB4 only arrived a few weeks ago, so that gap looks like a test of generalization to new problems.
TB2.1 is saturated and TB4 is not, so the scores do not prove benchmark gaming. Sol xhigh gets 90% on TB2.1 but 37% on TB4, and newer benchmarks are simply harder.
A benchmark stops being useful once training targets it. I would make benchmarks for my own workloads instead of trusting small score differences between models.
Even 27.3% on TB4 is in the current range: Grok 4.6 scores 20.3%, Sol 37.3%, and Opus 5 51.8%. Qwen 3.8-Flash-Next gets 25.3% while running locally in under 190GB of RAM.
Where are the model details, and are the weights open? If not, I do not see why I would choose this over DeepSeek Flash 4.1, especially if open-weight models keep getting cheaper through API providers.
This likely exists so Cognition can reduce spending on Anthropic and OpenAI API tokens. Any successful AI service with enough task data will want to fine-tune its own model at scale.
DeepSeek Flash still needs roughly 160GB to 250GB of memory at practical quantization levels, and serious work will still rely on cloud services. Local control is not the whole comparison.