API Reference¶
The 1.0-frozen public surface (Public API). lensemble.__version__ is the
package SemVer string. Generated from the docstrings of the symbols re-exported from lensemble.
Configuration¶
lensemble.config
¶
Model¶
lensemble.model
¶
lensemble.model — encoder, predictor, action heads, objective (docs/rfcs/RFC-0008).
Objective
¶
The three-term SIGReg-JEPA + frame-anchor loss (RFC-0008 5).
Constructed per round from the broadcast sketch seed s_t (INV-SKETCH-CONSISTENCY) and the
injected anchor. lambda_anc == 0.0 with anchor=None is the bare LeJEPA objective.
__init__(*, lambda_pred, lambda_sig, lambda_anc, sketch_seed, sketch_dim=64, ep_knots=17, anchor=None, sketch=None, target_stop_gradient=True)
¶
Construct the per-round objective.
sketch defaults to build_sketch(sketch_seed, d, sketch_dim) built lazily at first call,
once d is known from the encoder. An explicit sketch overrides the seed-built one; it is
used by the gauge-invariance test, which co-rotates the SIGReg frame (A -> QA) under the
O(d) gauge transform. ep_knots is the Epps-Pulley integration grid (RFC-0008 6).
target_stop_gradient=False is the claim-grade LeWorldModel base recipe: no EMA/teacher/target
stop-gradient in the prediction term (#191).
__call__(encoder, predictor, window, action_embedding)
¶
Encode, condition, predict, and evaluate the three terms; return the weighted total.
window.obs is the (num_steps + 1, *modality) frame stack (03 5); it is encoded as a
batch so that frames [:-1] are the inputs f(x_t) and [1:] the targets f(x_{t+1}).
action_embedding is (num_steps, cond_dim). When target_stop_gradient is true the
existing helper stop-gradients f(x_{t+1}); when false, claim-grade LeWM base mode compares to
the live target branch.
Raises ContractViolation (INV-WMCP) if the encoder output is non-conformant, and
surfaces GaugeError from the injected anchor when the landmark anchor is under-determined.
build_encoder(cfg)
¶
Construct an :class:Encoder per the model config.
Pre: cfg.model.latent_dim > 0 and the patching is consistent with cfg.model.num_tokens.
Post: an Encoder whose forward emits a conformant LatentState (INV-WMCP).
Raises: :class:~lensemble.errors.ConfigError on inconsistent dims/patching.
build_predictor(cfg)
¶
Construct a :class:Predictor from the model config (RFC-0008 3).
Reads cfg.model: d (latent dim), num_tokens (N), cond_dim (default d),
predictor_depth/predictor_width (defaults depth/d), num_heads (default a divisor
of width). Raises :class:~lensemble.errors.ConfigError on inconsistent dims (non-positive, or a
width not divisible by num_heads).
build_action_head(cfg, spec)
¶
Construct a per-embodiment :class:ActionHead from a validated ActionSpec (RFC-0007/0008; #8).
Reads cond_dim = getattr(cfg.model, "cond_dim", cfg.model.latent_dim) (the federation-fixed conditioning
width). Validates spec (dim > 0; for a discrete space num_classes present with
len == dim and each >= 2) and enforces spec.wmcp_version == WMCP_VERSION (INV-WMCP).
Raises :class:~lensemble.errors.ConfigError on an invalid config/spec.
INV-ACTIONHEAD-LOCAL: the returned head is per-embodiment LOCAL state, never serialized into a
shared artifact (the artifact boundary fail-closes on a non-encoder/predictor param group).
Gauge¶
lensemble.gauge
¶
lensemble.gauge — frame anchoring, Procrustes alignment, drift (docs/rfcs/RFC-0002).
frame_drift(embeddings, *, round_index=0, probe=None, expected_probe_hash=None, degenerate_safe=False)
¶
Compute the frame-drift diagnostic from per-participant probe embeddings (RFC-0002 9).
embeddings maps participant_id -> f_c(P) of shape (|P|*N, d); the reserved key
"global" is the aggregated model. For each participant pair, :func:procrustes_align recovers
the inter-frame rotation; rotation_angle_deg is the headline figure and procrustes_residual
the alignment residual. drift_from_global is each participant's angle against "global".
Probe pin (INV-PROBE-PIN): when probe is given its full content hash is recomputed over
points, landmark indices, and landmark targets and checked against both its stored pin and
expected_probe_hash (when given). Any mismatch raises
:class:~lensemble.errors.ProbeError; the verified full hash populates probe_hash.
degenerate_safe (#262): a STRONG frame anchor (the converged regime) can pin two participants onto
a near-identical frame, whose inter-pair Procrustes M is rank-deficient — that is the GOOD anchored
case (drift → 0), NOT an error. With degenerate_safe=True a :class:~lensemble.errors.DegenerateProcrustes
on a pair is recorded as 0.0° drift (coinciding frames) instead of raising, so the per-round drift
metric stays alive through convergence. The default False keeps the strict "raise rather than emit a
meaningless angle" contract for the bare diagnostic.
procrustes_align(source, target, *, singular_floor=_DEFAULT_SINGULAR_FLOOR)
¶
Closed-form proper-rotation Procrustes: (source, target) -> (Q*, residual) (RFC-0002 5).
Finds the rotation Q* in SO(d) that best maps source onto target and the Frobenius
residual ||source @ Q* - target||_F. source/target are (n, d); Q* is (d, d).
Inputs are upcast to fp32 (fp64 kept) before the SVD for determinism. Raises
:class:~lensemble.errors.DegenerateProcrustes (PROCRUSTES_DEGENERATE) when M = T^T S is
near rank-deficient (the error carries min_singular_value, condition_number, tol).
Federation & runtime¶
lensemble.federation
¶
lensemble.federation — DiLoCo outer loop, round state machine, roles (docs/rfcs/RFC-0013).
Coordinator
¶
Orchestrates the DiLoCo outer rounds, holds the canonical global model, runs the outer optimizer.
Untrusted w.r.t. raw data; a single sequential round loop (RFC-0013 §1/§6). Constructed with the
signature fixed by conventions §5 / RFC-0013 §1: Coordinator(config, *, transport).
close()
¶
Remove the coordinator's OWNED temp artifacts dir (the tempfile.mkdtemp it created).
Idempotent and safe: a caller-provided artifacts_dir is never touched (it persists with its
committed checkpoints). For an auto-created temp dir this removes it eagerly and cancels the
GC finalizer (#178). Use the coordinator as a context manager (with Coordinator(...) as c:) to
clean up automatically, or pass an explicit artifacts_dir for a persistent run.
run(num_rounds)
¶
Drive num_rounds outer rounds through the RoundState machine (RFC-0013 §1/§2).
Each round is OPEN → COLLECTING → AGGREGATING → ALIGNING → COMMITTING → CLOSED or
short-circuits to ABORTED. On CLOSED the committed global hash advances and a
:class:~lensemble.provenance.ledger.ContributionRecord is appended.
RETRY SEMANTICS (the choice). run is the FAIL-FAST driver and try_round is the
EXPLICIT-RETRY driver; both share the SAME single round body (:meth:_run_one_round), so the
sequential-loop invariant (round t+1 does not open until round t is CLOSED, §6) is
coherent across both. run SURFACES a below-quorum round as a raised
:class:~lensemble.errors.FaultToleranceExceeded (code FAULT_TOLERANCE_EXCEEDED, carrying
contributing/quorum); try_round does NOT raise on below-K, returning
RoundState.ABORTED instead so staging-and-re-attempting the SAME round t is the supported
elastic path (the round index/hash are unchanged on abort). A non-reproducible reduction always
raises :class:~lensemble.errors.NonDeterministicAggregation (security-critical, never swallowed) —
an abort drives the round to ABORTED with the global hash unchanged (no partial commit).
END STATE. run opens the next round only BETWEEN rounds, so after the last requested round the
driver rests in CLOSED (the next round is not opened speculatively); try_round opens round
t+1 immediately after a commit so the next try_round attempts t+1 (its end state after a
commit is OPEN on the new round).
try_round()
¶
Attempt the CURRENT round once; return the resulting :class:RoundState (RFC-0013 §1/§3).
The explicit-retry entry point. Drives the current round t through
COLLECTING → AGGREGATING → ALIGNING → COMMITTING → CLOSED over the PRESENT contributing set and,
on success, advances the canonical hash and opens round t+1 (so the next try_round attempts
t+1). If the present count is below the quorum
K = max(fault_tolerance_min_participants, secure_agg_threshold) the round goes to ABORTED
and this returns RoundState.ABORTED WITHOUT raising and WITHOUT advancing the round index or the
global hash — the caller may stage more updates and re-attempt the SAME round t. (Contrast
:meth:run, which surfaces the below-K case as a raised
:class:~lensemble.errors.FaultToleranceExceeded.) A non-reproducible reduction still raises
:class:~lensemble.errors.NonDeterministicAggregation (security-critical, never swallowed).
Elastic completion is over the present C_t (the absent participants are simply not in the
ContributionRecord); the in-process present set models the collect_timeout_s drop (the
wall-clock timeout is the #45 seam) and a delta for a PAST round is never back-applied.
round_state()
¶
The current :class:~lensemble.federation.round.RoundState (observability / test hook).
global_state()
¶
The current canonical broadcast :class:~lensemble.federation.state.GlobalState.
global_state_hash()
¶
The canonical committed global-model content hash (advances only on a successful commit).
global_params()
¶
The current canonical flat global params θ_t⊕φ_t (encoder θ then predictor φ, canonical).
ledger_records()
¶
The contribution-ledger records appended so far (RFC-0014 §7).
frame_drift_report()
¶
The last frame-drift report measured at ALIGNING (None before the first measured round).
Participant
¶
Holds sovereign data; runs the inner loop; emits a released, bound PseudoGradient (RFC-0013 §1).
Trusts neither coordinator nor peers with raw data (INV-RESIDENCY). One Participant per
federation member; constructed with the signature fixed by conventions §5 / RFC-0013 §1.
join(coordinator_endpoint)
¶
Register with the coordinator and recover the latest committed GlobalState (RFC-0013 §3).
Registers participant_id at coordinator_endpoint, recovers the committed state, then
VALIDATES it by fetching its θ/φ refs through transport.fetch_params (which hash-verifies the
weights, INV-CHECKPOINT-HASH). A tampered θ/φ artifact raises
:class:~lensemble.errors.CheckpointIntegrityError from the fetch and join does not return.
RECOVERY CONTRACT (RFC-0013 §3). A long-absent rejoiner adopts the recovered GlobalState as the
SOLE source of truth and discards any stale local state — it does not carry forward an old round's
(θ, φ) or round index, so the rejoiner is byte-identical to a fresh joiner from this point.
When the recovered GlobalState.round_index == 0 this ALSO revalidates INV-WARMSTART-T0: the
fetched encoder's content hash must equal the pinned warm-start (_warmstart_hash), else the
gauge is not closed at t=0 and :class:~lensemble.errors.GaugeError is raised here — the same
check :meth:local_round runs, hoisted to the recovery path so a round-0 rejoiner fails fast.
local_round(global_state, round_seed)
¶
Run H inner AdamW steps, form Δ_c, clip+noise, bind to R_c, return it (RFC-0013 §1).
See the module docstring for the full pre/postcondition contract (INV-PROBE-PIN,
INV-WARMSTART-T0, INV-SKETCH-CONSISTENCY, INV-DP-BOUND, INV-ACTIONHEAD-LOCAL,
INV-COMMIT-BINDING, INV-RESIDENCY).
RoundState
¶
Bases: str, Enum
The outer-round lifecycle state (RFC-0013 2). String values agree with 03 8 RoundPhase.
train_local(config, *, run_dir=None)
¶
Single-site Stage-A training: warm-start → inner loop → commit checkpoint → manifest (RFC-0001).
The single-site convenience path of [02-public-api §1.2]: it builds the encoder/predictor/action-head
from config (the #166/#168 LensembleConfig→architecture bridge), resolves local windows through
the SAME #22 data layer the federated Participant uses (cfg.data.data_source →
load_episodes → windows(cfg.data.window_steps)), runs cfg.federation.inner_horizon inner
AdamW steps minimizing the composite SIGReg-JEPA objective via the shared :func:_inner_loop, then
hash-commits the trained (θ, φ) with :func:~lensemble.artifacts.checkpoint.save_checkpoint
(INV-CHECKPOINT-HASH) and binds the run to a deterministic train_local-mode RunManifest.
Single-site == round 0: the objective's SIGReg sketch seed is round_sketch_seed(root_seed, 0)
(INV-SKETCH-CONSISTENCY). The anchor: with cfg.objective.lambda_anc > 0 it is built from the
pinned probe (cfg.data.probe_path) exactly as the participant builds it (round-0 reference snapshot
f_ref); with lambda_anc == 0 the bare LeJEPA objective runs (anchor=None).
Returns a :class:RunResult (checkpoint dir + content_hash + manifest hash + final loss). Only
the shared encoder/predictor weights enter the artifact (INV-ACTIONHEAD-LOCAL: the artifact
boundary fail-closes on a non-encoder/predictor group; the local head is never serialized here). No raw
observation/action crosses a boundary — the windows are RAW and local (INV-RESIDENCY).
Raises :class:~lensemble.errors.RoundError when no local windows resolve (no data_source and the
store yields nothing), and :class:~lensemble.errors.ConfigError on an invalid cfg.model/spec.
Evaluation¶
lensemble.eval
¶
lensemble.eval — latent MPC planner, eval harness, metrics (docs/rfcs/RFC-0005).
Planner
¶
Latent MPC planner over a batched latent dynamics (CEM / iCEM / MPPI), RFC-0005 3.
from_config(eval_cfg, *, action_dim, seed=0)
classmethod
¶
Construct from cfg.eval (planner, horizon, planning_samples).
plan(dynamics, initial_latent, goal_latent)
¶
Search for the action sequence minimizing the L1 latent goal-energy (RFC-0005 3).
dynamics rolls a batch of latents one step under a batch of actions; initial_latent and
goal_latent are (d,) latents from the frozen encoder. Returns the best sequence found and
the planning-cost telemetry. Raises :class:~lensemble.errors.EvaluationError if the rollout
diverges (a non-finite cost).
evaluate(checkpoint, env_id, *, cfg, num_episodes=_NUM_EPISODES, planner_iters=4)
¶
Run latent-MPC evaluation on a held-out env and return an :class:EvalReport (RFC-0005 §3).
Preconditions: checkpoint is a hash-verified artifact (INV-CHECKPOINT-HASH) whose weights are
keyed encoder.* / predictor.*; env_id resolves to a registered or stable-worldmodel://
environment. cfg.eval fixes the planner family / horizon / sample count and
cfg.determinism.root_seed pins the episode seeds. cfg.model must expose the encoder/predictor
build fields build_encoder / build_predictor read (the same V-JEPA shape namespace the CLI
supplies): the typed ModelConfig->architecture bridge is not in the tree yet (the CLI hand-builds
it), so pass a config whose model carries those fields.
Postconditions: returns an :class:EvalReport carrying success_rate (true succeeded() held-out
MPC success fraction; reported but non-binding for RFC-0017 because planning still optimizes latent
goal-energy), planning_samples, time_per_action_ms (mean planning wall-cost per action),
effective_dim (the collapse guard over the per-episode latents), state_probe_r2 when the world
exposes resident true state labels, checkpoint_hash (the artifact content_hash), and
run_manifest_hash (a deterministic hash over the eval-mode RunManifest, excluding the
non-semantic created_at). The report carries no tensor (INV-RESIDENCY).
Raises: :class:~lensemble.errors.CheckpointIntegrityError on a tampered checkpoint and
:class:~lensemble.errors.SchemaVersionMismatch on a too-new artifact (both propagate from
load_checkpoint, never caught); :class:~lensemble.errors.EvaluationError on an unresolvable
env_id or a diverging plan; :class:~lensemble.errors.ConfigError on an invalid cfg.model /
ActionSpec. Determinism is best-effort and seed-pinned (conventions 9).
Provenance¶
lensemble.provenance
¶
lensemble.provenance — episode hashing, Merkle commitments, contribution ledger (RFC-0014).
DatasetCommitment
¶
Bases: BaseModel
Tamper-evident commitment binding a participant's dataset to a Merkle root R_c (03 §9).
On-disk metadata: pydantic v2 JSON, frozen, extra="forbid", with an explicit integer
schema_version. merkle_root is R_c as 64-char lowercase hex (Phase-1 SHA-256). Validation
rules: len(merkle_root) == 64 and hex; episode_count >= 1; hash_algorithm is the pinned
algorithm. Load via :func:parse_dataset_commitment so an unknown/too-new schema_version raises
:class:~lensemble.errors.SchemaVersionMismatch.
ContributionLedger
¶
Append-only, hash-chained log of :class:ContributionRecord entries (RFC-0014 §7).
Backed by a JSON Lines file. :meth:append enforces the append-only invariant (strictly increasing
round_index and a prev_record_hash that chains to the tail); :meth:verify_chain re-validates
the whole chain. Phase-1 tamper-evidence: a rewritten past record breaks every subsequent link.
records
property
¶
The records appended so far, in order (read-only snapshot).
open(path)
classmethod
¶
Open (or initialize) a JSONL ledger at path, loading and parsing any existing records.
A missing file is an empty ledger. A structurally corrupt line raises
:class:~lensemble.errors.MerkleVerificationError; a too-new schema_version raises
:class:~lensemble.errors.SchemaVersionMismatch.
append(record)
¶
Chain record to the tail, persist it, and return its 32-byte content hash.
Sets the stored record's prev_record_hash to the current tail's content hash. Raises
:class:~lensemble.errors.ProvenanceError (PROVENANCE_FAILED) if round_index is not
strictly greater than the tail's, or if the supplied prev_record_hash is non-None and does
not match the tail (a caller asserting the wrong link). The first record must chain to None.
verify_chain()
¶
Walk the chain: each prev_record_hash equals the prior record's content hash and
round_index strictly increases. Returns False (not raise) on any break.
A structurally corrupt record — a prev_record_hash or global_model_hash that is not a
64-char lowercase-hex digest — raises :class:~lensemble.errors.MerkleVerificationError (the
chain cannot be meaningfully evaluated), distinct from a well-formed-but-broken link's False.
commit_dataset(dataset)
¶
Build the Merkle commitment R_c for a participant-local dataset (RFC-0014 §4).
Pure function of the episode set + the :class:CommitmentScheme; deterministic merkle_root across
platforms (leaves are sorted, no RNG). An empty dataset raises
:class:~lensemble.errors.ProvenanceError (a dataset with zero episodes cannot commit). A dataset
whose episodes declare conflicting wmcp_version values raises ProvenanceError rather than
silently recording one (prefer an explicit failure over a coerced commitment).
Verification¶
lensemble.verify
¶
lensemble.verify — Phase-2 verifiable layer + public recomputation (docs/rfcs/RFC-0006).
recompute_alignment(committed_weights, probe)
¶
Publicly recompute the committed model's frame alignment to the reference (RFC-0006 §4; SPEC 02 §1.8).
The frozen 1.0 signature recompute_alignment(committed_weights: Path, probe: Path) ->
FrameDriftReport. Deterministic given (committed weights, pinned probe): reconstructs f_θ from
the self-describing checkpoint header (#171), recomputes f_θ(P) and the closed-form Procrustes
alignment Q* to the probe's pinned reference targets E_ref = f_ref(P), and returns a
:class:~lensemble.gauge.drift.FrameDriftReport whose drift_from_global['committed'] is the
recovered rotation angle of the committed model to the reference frame.
The #18 caveat: this MEASURES the committed model's alignment to the reference frame; it does NOT verify the Layer-3 re-alignment backstop was applied, because that backstop rotates in activation space at aggregation (the recorded #18 decision), not as a fold into the committed weights.
Fail-closed preconditions (never swallowed): CheckpointIntegrityError (committed-weights tamper) /
SchemaVersionMismatch (too-new checkpoint) from the load; ArtifactError from
Encoder.from_header on a non-self-describing checkpoint; ProbeError (PROBE_INVALID) on a
probe whose recomputed content hash differs from its pin (INV-PROBE-PIN); and
DegenerateProcrustes (PROCRUSTES_DEGENERATE) on a rank-deficient probe embedding — never a
silent garbage Q* (RFC-0006 §4).