Anthropic's FlashProve proves LLM inference with zkSNARKs at 110 tokens/sec on one H100, 70-95x the cost of plain inference
- Anthropic released FlashProve, a zkSNARK-based provable inference system: on Llama 4 Maverick (400B-parameter mixture of experts, 4-bit quantized weights, integer arithmetic in prefill) a single H100 generates proofs at about 110 tokens/second with a proof size near 19 MB per 1,024-token request.
- Proof generation costs roughly 70 to 95x the GPU time of running the same model on vLLM, and the post says prior open-source systems are orders of magnitude slower.
- Five artifacts go out under Apache 2.0: the blog post, a technical report, a Claude-written protocol specification, executable code with a prover plus a C++ fast verifier and a Lean provably correct verifier, and Lean proofs for the top-level security statements.
- Coverage is narrow today: proofs assert each checked token is the committed model's greedy choice on an exact integer version of the published weights, so sampled decoding and long contexts are outside the demo. The commitment hides weights but not the architecture.
- Anthropic frames the use cases as independent evaluators checking that a served model matches the released one, clients of third-party routing services, detection of token-stream tampering, and a toolkit for verifying compliance with AI pacing requirements.
Hacker News opinions
Funny PR stunt coming from a company that was just in the news for silently routing ML-related requests to an inferior model. The model still responds, it just uses interventions to limit Claude's effectiveness without telling the user.
OpenRouter and similar providers can route to other models, so this would catch that. But I've also heard some hosts quietly serve quantized versions of big models. Would this catch that? The post says it proves a set of weights, not how they're represented.
It proves an exact integer version of the published weights, and the proof asserts each checked token is that model's greedy choice. A floating point implementation or sampled decoding could depart at some position.
Maybe you can tell from the size of the error? They say the integer version is within a small error rate of true float inference, though that doesn't cover adversarially designed prompts.
For open weights it does seem to work: you commit with the public weights and publish the seed, so anyone can recompute the commitment from the Hugging Face files and check the checkpoint and quantization they claim. Only greedy decoding and short contexts though, and 70-95x GPU time, so TEEs might be cheaper.
I'm not sure the property it attests is one we care about. It tells you Anthropic isn't stiffing you with a cheaper model and wasn't MITM'd, which is table stakes. It says nothing about whether your data was exfiltrated or whether the answer is any good.
At first I thought this was about proving LLM output correct against some source of truth. Instead it's proving which model produced the output. If you don't trust the provider at all, it's already game over.
It matters when there are content-level proxies between the client and Anthropic's API server, and there are a lot of those now.
Read past the opening paragraph. Independent evaluators need to know the model they're testing is the model being served, and clients on third-party routing want assurance the tokens came from the model they paid for.
Three benefits: it proves your hardware config is delivering correct outputs, it proves an external provider is running the inference they said, and quantization or fine-tuning breaks the proof so silent substitution gets caught. Right now it's limited to int-only weights and 1k context.
Customers care about the output being correct, Anthropic cares about the output being generated by Anthropic. The end goal is FUD against cheaper model routing.
Chinese resellers promise Claude access and substitute something else. This would kill one kind of fraud, and might add a guarantee for OpenRouter and Fireworks users.
I've wanted provable inference for years and this beats prior zk results by a lot. But for a closed provider, four independent inference providers running the same non-batched prompt give a probabilistic proof far more cheaply than a 90-100x compute hit. Provable training corpus information would be far more valuable.