Skip to content

Data model

Existing research contracts

The exact executable field definitions remain in contracts.py; hallmark schema emits their JSON schemas. Frozen strict models reject extra fields, nonfinite floats and boolean labels. Dataset rows include record/group/family/source/revision/licence/policy/split/language/prompt/response/label. Dataset size is bounded to 8 MiB; text fields to 32,768 characters. The fixture is 52 records and is not a policy dataset.

Experiment identity pins the raw dataset hash, policy, scope, seed, chunk size, target FPR, alpha and configured allowlists. Integer floor artifacts have 256 int8 weights, int32 bias/threshold, power-of-two scale and safe int32 accumulation. Run manifests require exactly five named artifact hashes. Research report/source schemas are partly implicit today; v1 hardening must make their outputs versioned and validated without relabeling old results.

v1 types

The implemented primitive hallmark.wire.WireValue is None | bool | int | bytes | str | list[WireValue] | dict[str, WireValue]. Integers are bounded to [-2^63, 2^64-1], strings are strict UTF-8 with no normalization, and containers use the codec resource limits. The artifact schemas below remain implementation targets except ModelManifest, FileRef, GenerationConfig, ClassifierManifest, InputProfile, Policy, Request, RequestExpectation, ChainExpectation, TranscriptOpening, ProofInputs, ReceiptBody, SignedReceipt, EpochRecord, ProofArtifact, RegistryBody, RegistryEntry, RegistrySignature, SignedRegistry, TrustPolicy, Limits, RegistryState, Bundle, EmbeddedFile, ResponseEnvelope, BundleStatus, Claim, VerificationResult, RegistryTransition, SignedTransition and BackendQualification, implemented in manifests.py and classifier_manifest.py, with policy/request contracts and opening contracts plus receipt/evidence containers and registry enrollment, with offline bundle and result contracts. Key-transition records are in registry.py. The backend record binds declarations and file bytes; it does not establish that a backend is qualified. Enrollment is an in-process verified result, with no wire decoder or public constructor. Primitive decoding does not validate fields, versions or cross-field invariants. The model JSON schema is a hex-byte projection; the manifest decision defines the executable path checks, nested records and file-inspection boundary. The classifier contract decision defines input recipes, typed arithmetic graphs and operator parameter files, including their separate declaration and parameter-validation boundaries. The policy/request decision defines classifier resolution, caller expectations, byte limits and continuity. The opening decision defines exact transcript/context/input preimages and the caller's independent tokenization obligation. The receipt contract decision defines structural/time checks, public-input packing and opaque evidence boundaries. The enrollment decision defines signed history, trust limits, process-local admission and atomic checkpoint persistence. The bundle contract decision defines exact embedded-file checks, private opening separation, total resource limits and the distinction between reported results and verified evidence. The registry authoring decision defines immutable entry metadata, irreversible retirement, dual-authorized transition signatures, retained history and persistent trust binding.

Notation below is a normative schema algebra, not executable Python. B32 is exactly 32 bytes; Digest=B32; U64 is integer 0 through 2^64−1 excluding booleans; I32 is signed int32; Id is the existing restricted ASCII Identifier; Text is strict UTF-8. Vec[T,n] has exactly n elements, List[T,a..b] is bounded, and Optional[T] is T or null. Every map has the listed fields only. Each top-level artifact carries schema_id: Id with its specific hallmark/<kind>/1 value. All digest identities use RFC-0001.

Type Required fields beyond schema_id
FileRef path:Text, size:U64, sha256:Digest; normalized relative path, no .., links, duplicate paths or absolute paths
ModelManifest files:List[FileRef,1..4096] sorted by path; architecture:Digest, tokenizer:Digest, chat_template:Digest, runtime:Digest, numerical_mode:Id, generation:GenerationConfig
GenerationConfig max_new_tokens:U64 in 1..4096, sampling:Literal["greedy"], seed:U64 fixed to 0; v1 has no stochastic serving contract
ClassifierManifest kind:Literal["rule","mlp","encoder"], program:Digest, weights:Optional[Digest], tokenizer:Optional[Digest], input_profile:Digest, arithmetic:Digest, threshold:I32, scope:Literal["prompt"], labels:Vec[Id,2], backend:Digest, verifier_key:Digest, setup:Digest
InputProfile encoding:Literal["utf8-bytes","token-ids"], tokenizer_files:List[FileRef,0..64], prefix_tokens:List[U64,0..8], suffix_tokens:List[U64,0..8], chunk_size:U64 in 1..512, overlap:U64 fixed to 0, max_units:U64 in 1..32768, padding_id:U64, padding_side:Literal["right"]
Policy rubric:Digest, classifiers:Vec[Digest,3] in rule/MLP/encoder order, combination:Literal["all-pass"], language:Literal["en"], scope:Literal["prompt"], max_prompt_bytes:U64 ≤32768, max_response_bytes:U64 ≤32768
RequestExpectation audience:Id, nonce:B32, interaction_id:B32, model:Digest, policy:Digest, continuity:Optional[ChainExpectation]
Request expected:RequestExpectation, prompt:Text, system_ref:Digest; request is admitted once, nonce generated by requester CSPRNG
TranscriptOpening system_ref:Digest, prompt:Text, response:Text, salt:B32
ProofInputs context:Digest, classifier:Digest, input_digest:Digest, unit_count:U64, chunk_count:U64, threshold:I32, verdict:bool
ReceiptBody expected:RequestExpectation, issued_at:U64, expires_at:U64, epoch:Digest, sequence:U64, previous:Optional[Digest], transcript:Digest, inputs:Vec[ProofInputs,3], release:Literal["released","blocked"]
SignedReceipt body:ReceiptBody, key_id:Digest, signature:bytes[64]
ProofArtifact backend:Digest, verifier_key:Digest, public_inputs:ProofInputs, proof:bytes bounded by enrolled limits
EpochRecord epoch_id:B32, key:bytes[32], model:Digest, measurement:Digest, issued_at:U64, expires_at:U64, cpu_evidence:bytes, accelerator_evidence:bytes, collateral:List[FileRef,1..256]
Bundle receipt:SignedReceipt, epoch:EpochRecord, registry:SignedRegistry, history:List[SignedRegistry,0..1024], proofs:Vec[ProofArtifact,3], files:List[EmbeddedFile,1..8192]; EmbeddedFile={ref:FileRef,content:bytes}
ResponseEnvelope receipt:SignedReceipt, response:Optional[Text], opening:Optional[TranscriptOpening], proof_status:Literal["pending","ready","blocked","failed","expired"]; blocked never includes response or opening

ChainExpectation={sequence:U64, previous:Optional[Digest]} pins caller-held receipt continuity when requested; sequence 0 requires null previous, later sequences require a digest. A null continuity expectation makes no chain-continuity claim.

PreparedInput={units:List[U64,0..32768], chunks:List[List[U64,0..512],1..32768], masks:List[List[bool,0..512],1..32768], digest:Digest} is internal and checked against InputProfile; chunk and mask dimensions match, and nonpadding units exactly reconstruct the complete input. Witness={prepared:PreparedInput, input_salt:B32, program_files:List[FileRef,1..4096]} is private and never logged.

RegistryEntry={kind:Literal["model","classifier","policy","backend","epoch-measurement"], digest:Digest, valid_from:U64, valid_until:U64, retired_at:Optional[U64], review:Digest}. RegistryBody={schema_id:"hallmark/registry/1", sequence:U64, previous:Optional[Digest], issued_at:U64, expires_at:U64, entries:List[RegistryEntry,1..4096]}. SignedRegistry={body:RegistryBody, signatures:List[{key_id:Digest, signature:bytes[64]},1..16]}.

TrustPolicy={schema_id:"hallmark/trust/1", registry_keys:List[bytes[32],1..16], signature_threshold:U64, min_registry_sequence:U64, accepted_models:List[Digest,1..64], accepted_policies:List[Digest,1..64], accepted_backends:List[Digest,1..16], hardware_roots:List[Digest,0..64], measurements:List[Digest,0..64], required_claims:Literal["A+B","B"], binding:Literal["B1","B2"], max_age_seconds:U64, limits:Limits}. B2 requires claim A and hardware roots; B1 with required B may use empty roots.

Limits={bundle_bytes:U64≤67108864, proof_bytes:U64≤16777216, nesting:U64≤16, verify_seconds:U64≤60, worker_memory_bytes:U64}; fields must be positive and no caller can exceed implementation hard limits. JobBudget={protocol:Digest, host:Id, max_seconds:U64, max_memory_bytes:U64, threads:U64, max_jobs:U64} requires a separately approved protocol for non-bootstrap work.

RegistryState={sequence:U64, digest:Digest} is trusted local rollback state, persisted atomically. Enrollment={registry:Digest, state:RegistryState, trust:Digest, admitted:List[RegistryEntry,1..4096]} is created only by verified enrollment.

RegistryTransition={schema_id:"hallmark/registry-transition/1", old_trust:Digest, new_trust:Digest, old_keys:List[Digest,1..16], new_keys:List[Digest,1..16], activation_sequence:U64>=1, previous:Digest, reason:Id, review:Digest} binds both external policies and the exact activation link. Key ID lists are sorted/unique. SignedTransition={schema_id:"hallmark/signed-transition/1", body:RegistryTransition, old_signatures:List[RegistrySignature,1..16], new_signatures:List[RegistrySignature,1..16]}. Both thresholds are independently required. Decoding is not authorization.

VerificationResult={schema_id:"hallmark/verification-result/1", accepted:bool, mode:Literal["B1","B2"], claim_a:Claim, claim_b:Claim, policy_pass:Optional[bool], codes:List[Id,0..64], evidence:List[Digest,0..8192]}. Claim={status:Literal["accepted","rejected","not-accepted"], strength:Literal["hardware-attested","cryptographic","none"], assumptions:List[Id,0..64], residuals:List[Id,1..64]}. BundleStatus={receipt_id:Digest, state:Literal["pending","ready","blocked","failed","expired"], bundle:Optional[FileRef], error:Optional[Id]}.

Invariants

INV-SCHEMA: reject extra keys, duplicates, unknown versions, wrong primitive types, noncanonical encodings, trailing bytes, invalid Unicode, bound violations and inconsistent dimensions before cryptographic work.

INV-IDENTITY: changing any manifest, input profile, graph, threshold, policy or backend/setup identity creates a new digest. Registry entries are not mutable names for interchangeable artifacts.

INV-NO-CYCLE: classifier manifests name the program/key/setup, never themselves; public inputs name the completed manifest. Context excludes proofs and receipt signatures. See RFC-0001 for the acyclic hash order.

INV-TIME: issued_at < expires_at, all timestamps are Unix seconds, and verification uses a caller clock. There is no silent skew allowance.

W33 adds hallmark/conformance-case/1, hallmark/conformance-corpus/1 and hallmark/conformance-report/1 records. The corpus binds source and input file hashes, independently supplied fixture contexts, exact expected outcomes and mutation ancestry. The report distinguishes observed component outcomes from unavailable evidence and has independent:null. See the schema and identity decision.

W10 adds hallmark/rule-program/1 and hallmark/rule-result/1. The program uses the existing program domain and binds literal patterns, input/arithmetic identities and exact provenance bytes. Native results bind the classifier and complete input and report every chunk's score; they are private diagnostics, not proof evidence. See decision 0020.

W14 adds hallmark/source-row/1, hallmark/source-manifest/1, hallmark/near-duplicate-review/1, hallmark/data-import/1, hallmark/lineage/1, hallmark/split-manifest/1 and hallmark/dataset-manifest/1. The source manifest binds reviewed adapter bytes, licence/purpose/privacy declarations and review pointers. The protocol-domain import plan is externally pinned; the corpus-domain dataset manifest binds normalized Record/1 JSONL, exact raw-row lineage and deterministic group splits. Large split manifests use file SHA-256 references. Each record's source revision is its source-manifest byte hash, with the upstream revision retained inside that manifest. See decision 0021 for schema snapshots, grouping, resource limits and evidence boundaries.

W11 adds hallmark/mlp-program/1, hallmark/mlp-weights/1 and hallmark/mlp-result/1. Program identity uses the existing program domain; classifier weights bind raw SHA-256 of the complete canonical weight file. The fixed graph, exact numerical semantics, private result traces and bounds are defined in decision 0014.

Evolution

New or changed semantics require a new schema ID, explicit decoder dispatch, migration fixtures and a decision. No “best effort” decoding or coercion. Optional fields are explicitly null; omission is invalid. Artifact paths are identifiers inside a bundle, not permission to access the filesystem. The research JSON format remains distinct from the v1 binary wire format.