Architecture: what runs, what is proved¶
Hallmark has two provers and one verifier role. SASH proves quantized TinyLlama inference; direct STWO proves the rule, MLP and CNN classifiers. Both are Circle STARKs. The caller pins every model, program and parameter before proving, and the verifier checks each proof against those pins. The composed workflow then checks, in host code, that both proofs refer to the same request; it does not aggregate them recursively. Decision 0090 makes Circle the only classifier backend; models, arithmetic and research status are unchanged.
Three roles, kept apart on purpose:
- Caller. Owns the expected request and enrolls model roots, programs and proof parameters before any proof exists. Enrollment is the trust anchor.
- Prover. Runs the computation, builds the witness and produces proofs. It is trusted with nothing; a root it supplies by itself proves nothing.
- Verifier. Receives proofs, the caller's enrollment and public inputs. Never weights, witnesses or native traces.
Choose the workflow¶
| Command | Actual computation | Proofs and verification | Where it runs |
|---|---|---|---|
make e2e |
Train/export an MLP and CNN; evaluate them plus a fixed rule on three synthetic prompts | 9 STWO Circle STARK proofs, checked by fresh proof-only processes | Mac; no LLM, credentials or paid jobs |
make e2e-llm |
Generate the fixed eight-token TinyLlama answer, including EOS | 23 SASH proof segments, linked and verified by SASH | Mac preparation; three sequential paid HF CPU jobs |
make verify-llm LLM_RUN=… |
Check retained LLM proofs; no new inference or proving | Same 23 segments, using an adapted native verifier | Mac; no weights, full witness or HF credentials |
make benchmark-llm-gpu … |
Reuse frozen public witnesses; compare complete SIMD and CUDA proving | Same 23 SASH segments per backend, byte equality and fresh CPU/Mac verification | HF CPU build, one H200 measurement job, separate CPU verification, Mac verification |
make e2e-composed |
Fresh LLM workflow, fresh classifier preparation, then classify the LLM prompt | 23 SASH segments + 3 STWO Circle classifier proofs | Both workflows plus fresh native LLM verification |
make e2e-composed LLM_RUN=… CLASSIFIER_RUN=… |
Reuse caller-owned preparation; freshly verify LLM proofs and prove classifiers for that request | Retained 23 SASH segments + 3 new Circle classifier proofs | Mac; no paid jobs or checkpoint download |
make verify-flashprove |
Check two stored upstream FlashProve proofs; no inference or proving | 2 FlashProve proofs accepted, 99 altered sets rejected | Mac or Linux; no GPU or credentials |
make e2e-flashprove … |
Commit to and prove upstream's synthetic small MoE kit | 1 FlashProve proof, checked on the worker and the Mac, 6 altered copies rejected | HF cpu-xl build, one H200 job, Mac verification |
These are counts for the current fixed cases, not arbitrary-model support. See classifier prerequisites, LLM prerequisites and cost ceiling and composition options before running them. The GPU measurement guide gives its separate cost bounds, host witness-generation paths, generated CUDA constraints, strict zero-fallback checks, device sampling and cache policy.
Classifiers: make e2e¶
Each of the nine proofs states the same thing about one prompt and one classifier:
The detailed data flow, including where caller inputs enter:
flowchart TD
D[Frozen synthetic fixtures] --> T[Train MLP and CNN or admit exact frozen quantizations]
T --> M[Native rule, MLP and CNN exports] --> C[Compile fixed gates: exact SHA-256 and integer operators]
C --> E[Caller pins program, preprocessing root and parameters before proving]
X[Caller-owned bytes, salt and request expectations] --> W[Construct complete classifier witnesses]
C --> W
E --> P[Direct STWO: nine serialized Circle STARK proofs]
W --> P
P --> V[Fresh proof-only STWO verifier processes]
E --> V
X --> H[Host reconstructs public inputs and exact classifier roster]
H --> V
V --> A[Verified verdicts and separate all-pass policy decision]
The graphs constrain the complete 96-byte input, salted canonical SHA-256 commitment and integer decisions. The 100 logical public fields use 103 M31 transport values: two threshold limbs replace one 32-bit value, and public zero/one wires supply the other two values. Context semantics and native expectation reconstruction remain caller checks.
This path uses the pinned STWO 2.2.0 dependency, with Hallmark's gate AIR, LogUp wiring and SHA/integer/classifier compiler. It does not run SASH's Llama application or a general-purpose VM. Caller-enrolled public preprocessing supplies no security or witness-hiding claim. The frozen MLP is always-allow within this input domain; the rule and CNN provide the workflow's block controls.
The Circle guide gives source ownership, concrete proof parameters, commands and conformance failures. The composed controller joins three Circle classifier proofs to the retained SASH LLM bundle: all 26 proofs and 22 residual links verify freshly, with 16 changed-binding rejections. This remains host composition.
Orchestration is run_circle_e2e.py. Native preparation validates the fixed model hashes and creates caller-owned controls. The proof driver compiles all gates, pins binary/source identities and checks the independently admitted enrollment root before proving. It generates nine proofs, freshly verifies each, rejects 63 changed public statements, 18 cross-request substitutions and three invalid rosters, and checks native/circuit trace parity.
The exact relation is complete integer inference plus salted canonical SHA-256 input hashing; training history is outside the proof. See the public statement. Caller reconstruction binds context, model, classifier and policy; the caller currently reruns native inference to derive expected verdicts. This is not yet a demonstration of cheaper verification than rerunning these small classifiers.
LLM: make e2e-llm¶
The detailed data flow:
flowchart TD
M[Pinned TinyLlama checkpoint and tokenizer] --> N[Mac: quantized greedy generation and witness extraction]
S[Hash-pinned SASH source archive] --> N
S --> B[HF CPU job 1: build original Rust prover]
N --> P[HF CPU job 2: derive roots and generate 23 proof segments]
B --> P
E[Caller-enrolled model, shape, calibration roots and expected token case] --> P
P --> F[Serialized proofs, fixed statements and claims]
F --> V[HF CPU job 3: fresh SASH cryptographic grid verification]
B --> V
E --> V
F --> L[Optional Mac: adapted native proof-only verifier]
E --> L
Yes: this produces and cryptographically verifies real SASH proofs of the
disclosed quantized LLM inference. It is more than a native rerun or hash
receipt. The current case is TinyLlama-1.1B-Chat-v1.0 at a pinned revision with
the named mixed quantization profile y-attnout-w12; that name does not mean
uniform 12-bit weights. It does not prove equivalence to unmodified floating-point
Transformers inference.
The prompt has 25 tokens. Eight output tokens spell “The capital of France is Paris.” and EOS. One 32-position input block contains the prompt plus the first seven generated tokens. Output positions 24–31 produce those eight next-token IDs. Verification checks the seven generated-input/previous-argmax equalities, closing the autoregressive chain through EOS.
The bundle covers all 22 transformer layers plus the final normalization/output
head: 23 segments and 22 residual links. Embedding is bound in the first
segment. Each segment's proof is checked, followed by the model roots and
inter-segment/token links. boundary-grid --verify performs these checks;
--expect-token-links 0 describes the inter-block axis of this single-block
case, not an absence of generated-token checks. The result is a linked bundle,
not a single recursive proof. See the full LLM statement.
How SASH is integrated¶
Hallmark uses pinned upstream source plus command-line adapters. It neither
installs SASH as a Hallmark Python dependency nor maintains a tracked full copy
of SASH. The controller downloads a hash/size-checked archive of
upstream commit eb95bfa
into an ignored run directory. Upstream includes its nitroo/ Circle STARK
implementation and locked Rust/Python dependencies.
| Component | Ownership and modifications |
|---|---|
| Local witness preparation | Upstream Python harness plus Hallmark's preparation adapter and exact-GEMM patch. The patch preserves the exact float64 accumulator instead of rounding it through float32. |
| Linux CPU prover/verifier | Built from the unchanged pinned upstream source, separately from the patched local witness checkout. Hallmark orchestrates model-roots, per-segment compose, serialization and boundary-grid --verify. |
| Linux CUDA prover | Uses the same pinned source, plus upstream-generated constraint kernels and Hallmark's strict fallback/build patch. Modified/generated source and binary hashes are retained; witness/interaction generation stays on CPU. See the GPU build and measurements. |
| Native Mac verifier | Adapted upstream Rust, using the native-verifier patch: avoids CUDA linkage, restricts the CLI to verification, uses compiled enrolled shape roots and strengthens boundary validation. It does not rederive shape trees at runtime. |
| Case admission and orchestration | Hallmark's case, controller, proving adapter, verification adapter and native controller. These enforce caller expectations and reject incomplete/unsupported claims. |
The proof-only worker receives serialized proofs/statements/claims and expected public parameters, without checkpoint weights or the full witness. It runs the real verifier in fresh processes. This is the same SASH verifier implementation, not an independently developed verifier. The optional native variant shares the cryptographic checks but changes public preprocessing/trust: its compiled shape roots must be admitted by the caller. Its faster recorded timing is not a matched backend speedup.
Tokenizer/chat-template/detokenization checks, checkpoint-to-quantized preparation and initial root provisioning remain host responsibilities. A proof binds the enrolled quantized relation; it cannot authenticate its own enrollment. Optional signed enrollment requires a signer's public key supplied through an independently authenticated channel. Neither implementation has an independent security qualification or a witness-hiding claim. The retained fresh run's serialized proof files total 737,858,641 bytes; proof-only verification is not a claim of tiny artifacts. Recorded results give the measurement context.
Composition: one request, two verified relations¶
flowchart LR
E[Caller-owned prompt, response, model and context] --> L[Freshly verify 23 SASH segments]
E --> C[Prove three classifier verdicts on that prompt]
C --> V[Fresh proof-only STWO verifier]
L --> J[Host checks exact request and model binding]
V --> J
E --> J
J --> I[Integrity result]
J --> A[All-pass policy decision]
The composed controller first verifies LLM inference,
then proves all three classifier decisions with the same opened request/model
context. It checks exact roster/coverage and binding; no saved passed flag is
accepted in place of verification. A valid proof of a blocking verdict remains
valid. The frozen synthetic CNN blocks the ordinary France prompt, so the current
composed case has verified integrity and release: false.
This is host composition, not protected response release or hardware provenance. It proves computation under admitted parameters, not policy quality, authentic capture, training history, operational authorization or real-world safety.
Trade-offs and next steps¶
SASH and direct STWO are specialized applications within the same proof family, not interchangeable prover binaries.
| Current path | Useful property | Remaining trade-off |
|---|---|---|
| SASH Circle STARK LLM | Working specialized quantized transformer proofs; no general-purpose VM | Large fixed-case bundle, public parameter enrollment and shared prover/verifier implementation |
| STWO Circle STARK classifiers | Complete repeated rule/MLP/CNN workflow and retained-LLM join; exact input hashing and integer operators | Higher memory and much larger JSON proofs; repeated per-proof preprocessing; no backend qualification |
Classifier and 1.1B LLM proof sizes/timings remain different workloads.
The Circle STARK classifier epic #112 tracks the implementation:
- #113: freeze/replay the identical relation, exports, inputs, failures and measurement protocol.
- #114: complete byte-rule proof, including the existing SHA-256 commitment and caller binding.
- #115: exact trained integer MLP proof.
- #116: complete contextual byte CNN proof.
- #117: common workflow, composed request and repeated benchmarks.
The classifier implementation uses direct STWO 2.2.0 as a dependency, separately from SASH's pinned Llama application/fork. It adapts the upstream gate-AIR/LogUp patterns and adds Hallmark's own SHA-256/integer/classifier circuits. The source map explains ownership. The epic is integrity-only. It does not reopen the completed MVP or silently satisfy the broader private/hardware v1 requirements.