Anthropic Tool Proves Which Model Answered

Anthropic on Nov. 2 released FlashProve, an open-source research prototype that produces a cryptographic proof showing which committed model generated a given reply, without revealing the model's weights. Running on a single NVIDIA H100 GPU, it proves around 110 tokens per second for Meta's 400-billion-parameter Llama 4 Maverick, well ahead of earlier published open-source systems.

Key takeaways

• FlashProve lets anyone check, on an ordinary CPU, that a reply came from a specific model the provider committed to in advance.
• It proves about 110 tokens per second on one H100, with a proof of about 19 MB for a 1,024-token request, but costs roughly 70 to 95 times the GPU time of normal inference.
• Anthropic calls it intermediate progress: it covers only greedy decoding, an integer version of the weights and a 1,024-token context.

What is provable inference?

When an AI company serves a model through an API, users generally have to take its word that they are getting the model it advertises. Provable inference aims to replace that trust with math. Anthropic's post lists uses such as confirming a served model is the one a developer claims, detecting tampering or model substitution, and supporting regulatory compliance.

"Cryptographic proofs of inference solve this problem," the authors wrote. The proof states that "the committed model produces these output tokens when evaluated." It hides the weights but not the model's architecture.

How FlashProve works

The system is built on a zkSNARK, a type of zero-knowledge proof, compiled with hash functions that the team says offer post-quantum security. In a one-time offline step that takes about 39 minutes, the provider publishes a short commitment to its weights. Afterward, for each request, the prover generates a proof file that anyone holding the commitment, prompt and reply can check.

Several design choices make it fast. The prover handles only the roughly 17 billion parameters active per token in the mixture-of-experts model, batches requests after the fact, and streams weight segments from host memory so the job fits on one GPU. The team formally verified the protocol in the Lean theorem prover, covering completeness, soundness and zero knowledge.

The test used a public 4-bit version of Llama 4 Maverick, about 232 GB, on one 80 GB H100 with 24 CPU cores. Verification took about 17 seconds on 24 threads, or 21 seconds on a single thread.

How much faster than earlier systems

The post compares FlashProve with prior work. zkLLM, published in 2024, ran LLaMA-2-13B at about 2.5 tokens per second on one A100. SASH's prover reached about 1.25 tokens per second on Llama 3.1 8B on an H100, and DeepProve reported about 2.9 tokens per second for GPT-2. A system called VerInf took 14.3 hours for 1,000 tokens of the same Maverick model and produced a 93.6 GB proof, according to the post.

The speed still has a price. Anthropic says proving costs "~70-95x the GPU time of running the model itself on vLLM," the open-source serving library.

Context: Anthropic's safety roadmap

Anthropic's Frontier Safety Roadmap set a goal of building a provable inference prototype by Sept. 30, 2026. The company says it met that goal through a separate project that signs its inference traffic and recomputes it on a trusted server, and that it later renamed the goal "inference verification." FlashProve is a parallel, software-only approach that the team frames as complementary to hardware-based trusted execution.

The work comes from Anthropic's Frontier Red Team with researchers from EPFL, MIT and StarkWare. The code is on GitHub under the Apache 2.0 license; model weights are not included and must be downloaded under Meta's Llama 4 license.

What's next

The authors list clear limits. The proof covers only greedy decoding of an exact integer model, so sampled outputs may diverge, and context is capped at 1,024 tokens, far short of frontier chat models. The fast C++ code is not formally guaranteed to match the Lean protocol, and timing side channels are out of scope.

"This research post is meant to share intermediate progress, and not a completed research agenda," the team wrote.

FAQ

Does FlashProve reveal a model's weights? No. The proof hides the weights, though it does reveal the model's architecture.

Can I verify a proof without a GPU? Yes. Verification runs on a CPU and took about 17 to 21 seconds in Anthropic's tests.

Is it ready for production chatbots? Not yet. Anthropic calls it a research prototype limited to greedy decoding and 1,024-token contexts.

References

• Progress on Provable Inference for Large Language Models — Anthropic (Nov. 2, 2026)
• anthropics/FlashProve — GitHub