Hightouch: a Jev judge that escalates to Claude Opus matches frontier accuracy at 4x lower cost
- Hightouch tested cheap decision models Jev and GPT-6 Luna as judges on identity resolution: out-of-the-box accuracy was 88.8 to 97.3%, and a hybrid Jev + Claude Opus harness reached frontier accuracy at more than 4x lower cost.
- Decision models cost $0.05 to $0.10 per 1,000 identities and answer in about 0.15 seconds, up to 90x cheaper and 10x faster than frontier models, but trail Opus by roughly 6 percentage points on harder benchmarks.
- Jev's confidence is better calibrated: most of its mistakes come with low confidence, so it can hand those cases to Opus, while most of Luna's mistakes come with high confidence. That is why Jev beats Luna in a hybrid harness even though Luna scores better alone.
- Identity resolution maps records to real people, and the dangerous failure mode is over-merging. There is no answer key, so teams sample identities by hand at about 50 per hour and need more than 1,000 samples to get the over-merge rate error under 1 point.
- The result covers one task, identity resolution judging, and the article claims no general usage or coding agent gains; several commenters flagged that scope.
Hacker News 의견들
Wait, does it have to be Opus? What about Sonnet or Haiku? I think the real trick here is that Jev has well-calibrated confidence for this specific task, so whichever expensive model it defers to matters less as long as that model is stronger than Jev.
I run the same hybrid setup for a lot of security engineering work and it holds up well after some initial calibration. We rebuilt our own auto mode exactly like this to judge tool calls and intent.
Same idea here, I extended opencode-warden to use a local decision model to validate tool inputs and outputs. d1 is fast enough to run on an old 1080 GTX. It works... ok. I wouldn't rely on it though.
I go the other direction with Qwen3.8 27B: let it orchestrate and do shallow research, then have it ask Claude. Qwen can pull nearly as good answers out of Haiku, Sonnet and Opus. My theory is it fills in the details, so directing Claude through it gets great results way cheaper.
How is that different from just telling the model you're already using to fill in the gaps itself? If you're not manually reviewing the intermediate hop, how does that add any understanding?
I do the opposite: Qwen3.8-27B as subagents, Opus 5.5 as orchestrator and planner, never the implementer. Cuts my token usage a lot. Never thought of running it the other way around, I'll try it soon.
Anthropic has a native version of this now: /advisor lets you run Sonnet with Opus as advisor, or Haiku with Sonnet or Opus as advisor, and the weaker model calls out automatically when it decides something warrants it.
Good read, but I wish there were code examples for how the Jev and Luna harnesses were actually built. I'm still trying to wrap my head around the paradigm.
This has me believing decision-model-controlled context is the future of context management beyond a raw AGENTS.md file. Why give all the context to an agent when the context can be curated and trimmed automatically for each new problem?
Dynamic context management has a lot of room but it will tank your cache hit rate right now. We need caching that isn't just prefix matching before this really makes sense.
4x lower is such a mathematically awkward way to put it.
GPT-6 Luna is a small LLM, not a decision model.
There is a decision-model version of GPT-6 Luna, the naming is just confusing. Small LLM plus no reasoning plus fixed JSON-only output is basically a decision model.
I'm numb to these headlines. It's easy to produce these with proof that only proves the sandboxed scenario it was run in. I keep getting disappointed by how well these approaches hold up on real work.
We've been doing this for a while and fine-tuning the decision model helps a lot. The catch is you still need a pretty low confidence threshold, like 10%, but that cuts 70% of the work with 0% quality loss in our benchmarks.
Been doing this on Discourse for spam and locale detection, mostly d3-turbo with Qwen 3.8 Flash Next.
Cool result, but it's a benchmark for one very specific task, identity resolution. Despite the title, the article isn't claiming general usage or coding agent improvements.