Alternatives for a real LLM response proof¶
2026-10-22. Static primary-source review only: no model inference, compilation or proof reproduction. Target: a real pretrained model, a complete short greedy continuation, minutes–hours proving, fast verification in a separate process, public witnesses acceptable, no general-purpose zkVM. This note does not change the protocol or qualify a backend.
Recommendation¶
Keep SASH as the leading implementation candidate subject to its separate source audit. Among alternatives examined here, DeepProve is the strongest functional match, but its core and crypto dependency have restrictive licences. Neither its current release nor the apparent older MIT/Apache root metadata is a clean permissive integration path. zkGPT and zkLLM public artifacts need major work before they demonstrate independently verifiable generated text; EZKL is a fallback compiler, not an evidenced efficient small-LLM solution.
| Rank among alternatives | Candidate | Judgment |
|---|---|---|
| 1, conditional on rights | DeepProve | Real GPT-2/Gemma support, generation linkage and serializable proofs; use as a technical reference or permitted internal evaluation, not an assumed OSS dependency. |
| 2, expensive fallback | EZKL | Separate prove/verify interface; full pretrained LLM generation is not demonstrated by the checked example. Cost and current source licence need resolution. |
| 3, protocol reference | zkLLM | MIT, specialized CUDA operators; authors explicitly say separate noninteractive verification needs systematic redesign. |
| 4, protocol reference | zkGPT artifact | MIT source, but released demo uses random weights and a colocated verifier. Paper timing does not establish an actual pretrained response proof. |
DeepProve: technically close, licensing is decisive¶
- Pin:
9d1a53e2ef49ffa2c902b8689cd3c58057a4e662, dated 2026-05-26. The August engineering article is newer than this repository head; do not assume it implies a newer implementation. - The root licence, sections 2 and 4, limits grants to internal testing/evaluation and unmodified use with Lagrange's platform, and restricts derivatives/distribution. This is an observation of the text, not legal advice.
- A tempting older pin,
26124dc233c9a7e50c07cf4b6746350851b90aea, has root MIT/Apache files and Cargo metadata, butzkml/LICENSEalready has the same restrictions. That file was introduced bybcaf0d4on 2025-03-08. Its parent treed3d958b8525c8720f30ac49ec090a1f3fa0e7d51contains no LLM/GPT/transformer/attention paths. No usable permissive LLM revision was established. - The locked
dp-cryptodependency is22e8e93cd0f94a7638616a1ae22190feb0a6b275, with its own Lagrange licence. Swapping only the main repo revision would not resolve this. - Comparing
26124dc…to9d1a53e…: no changes tozkml/src, core verifier, transformer operators or examples. Sixteen paths differ, principally licensing, client/API serialization and versions. Cargo.lock changes package versions and the client dependency from rmp-serde to postcard; the crypto pin is unchanged. There is no later core fix in that interval that rescues the older licence ambiguity.
Model and generation coverage. The loader tests explicitly name google/gemma-3-270m-it; the README recommends GPT-2, Gemma-3-1B-IT and Llama-2-7B. The 270M checkpoint is a more meaningful instruction-following demonstration than GPT-2 completion if rights are resolved. SmolLM2 support is not established by these tests. None was run here.
The LLM driver performs autoregressive inference with caches, resets them, then makes one causal full-sequence forward trace. Its proof covers embeddings, position encoding, transformer arithmetic and argmax. Verification checks supplied prompt tokens and the shifted relationship between input tokens and output argmax tokens. This certifies the resulting sequence mathematically; it does not attest a historical KV-cache execution.
The benchmark uses random prompts: --sequence 64 means 62 random prompt tokens and two newly generated tokens, not 64 generated tokens. Its reported throughput divides total context length by proof time. The chunk example uses real text The sky is, but a Q2_K checkpoint and a short maximum context; it is not a ready arbitrary-prompt receipt CLI.
Artifacts and trust. LLMProof contains a serializable proof and public input/output tensors. LLMVerifierContext::verify(proof, user_input, io) does not require the full prover trace or checkpoint. The bench serializes proof bytes to measure size but does not persist a reusable response proof and run verify-only in another process. Its .pk/.vk flags persist contexts. A thin application would still need exact tokenizer/decoded-answer checking, independently enrolled verifier context, proof/IO persistence, bounds, and mutation rejection. Do not accept a verifier context supplied solely by the prover.
Two concrete review gates precede any integrity claim: context setup calls PCS::test_setup; a trusted production SRS and enrollment story are not demonstrated by that path. In the LLM verifier, EOS is searched after skipping prompt positions but its relative position is used as the sequential-check bound. Empty prompts, early EOS, final-token inclusion and malformed shapes need direct negative tests. This is a static concern, not a reproduced exploit or security audit.
Resources. The maintainer README reports a 24-core/504 GB CPU server: GPT-2/512 sequence, 7.6 min proving, 1.3 s verification, 10.7 MiB proof; Gemma/512, 19 min, 4.3 s, 27 MiB. Installed RAM is not measured peak usage. These figures cannot establish fit on a 48 GiB Mac or prove a 512-token continuation. CPU and macOS-capable wgpu paths exist; wgpu accelerates inference while PCS remains CPU. The toolchain file pins nightly-2026-01-27 despite README wording about stable Rust. No Mac build or timing was attempted.
zkGPT: paper result is not the released pretrained demo¶
Artifact: Zenodo 15719324, zkGPT-main.zip, embedded git archive ID c09edb103c589f7ef1bc4b90d3daae6590b14c8d, SHA-256 3921fbef03e227d195e716d9d7b92e024abdb865a5f03d0d7904371140ad6d6c. Included LICENSE.md is MIT. The USENIX paper reports sub-25-second GPT-2 inference proving, not this project's independently reproduced complete response.
In the released archive, main_demo_llm.cpp creates a 12-layer/768-width model, then constructs verifier v(&p, p.C) in the same process. neuralNetwork.cpp::readFconWeight assigns rand()%1024 weights; models.cpp fixes sequence length 30 and contains a legacy VGG path. The README recommends at least 200 GB RAM. No pinned pretrained-weight loading, tokenizer-to-output generation chain, portable proof file or standalone verifier was established. The short reported timing is not a reason to select this artifact for the user-facing MVP.
zkLLM: useful techniques, unsuitable drop-in verifier¶
Pin 993311ea8e2346b90efacb35337b2e3cfe0d6f8d, MIT, archived July 2025. The authors' README explicitly says prover/verifier execute side by side and that separating them and applying Fiat–Shamir needs systematic redesign. It gives CUDA/A6000 and A40 environments and says basic types generally cannot execute on CPU. Demo stages implement Llama-2 operators over activation files; do not treat those files as verifier inputs or an accepted generated response.
The paper reports an A100 40 GB and 124.5 GB host memory environment, up-to-13B models and 2,048-token forward-pass workloads. This is strong operator/protocol research evidence; it does not supply the missing standalone verifier or prove every token of a short generated answer in the released demo.
EZKL: integration surface exists; useful-LLM cost is unestablished¶
Pin e196b111c1bafaa61b92ae431cd3c3fe9371da05. Official docs describe ONNX compilation and separate setup/prove/verify, with public input/output choices and portable artifacts. It uses Halo2 rather than a zkVM. Source inspection did not find a root licence file or Cargo licence field at this pin; GitHub API returns license:null. Do not assume rights based on historical descriptions without resolving the exact version.
The nanoGPT example instantiates a fresh four-layer, width-64, vocabulary-65 network and random input, without loading pretrained weights. It exports logits; that is not a real short response proof. The 2024 primary paper reports a 250K-parameter nanoGPT at 2,781 s proving, 2.69 s verifying, and a 219 GB proving key. These are historical measurements, not a current benchmark. They motivate measuring cost before committing to a larger model, not claiming contemporary EZKL cannot improve.
Acceptance gate for any selected alternative¶
Freeze one real model revision, tokenizer/chat template, quantization and exact deterministic decoding. Produce a meaningful short answer; persist proof plus public token IDs; start a fresh verifier process with caller-enrolled model/key/input/output expectations and no witness/checkpoint; reject changed prompt, changed response, swapped model, omitted/reordered tokens, early-EOS manipulations and malformed proof. Report setup separately from warm proving, verification, proof size and peak memory. A one-token or random-weight benchmark does not pass this gate.
Local research resource note: the initial DeepProve clone unintentionally began Git LFS retrieval and was interrupted; about 898 MB of local LFS storage remained. Subsequent checkout used GIT_LFS_SKIP_SMUDGE=1. No downloaded model code was executed, no cloud job ran, and no credentials were accessed. Further clones should disable LFS smudging explicitly.