Skip to content

Tapestry-like LeWM TwoRooms Demo Card

This card is the public evidence surface for the Tapestry-like browser-federated LeWorldModel adapter experiment (#314; contract in TAPESTRY_LEWM.md). It states exactly what was real, what the evidence binds, and what is not claimed. It exists so the real-checkpoint path can never be confused with the older surrogate-swipe-dot educational path (BROWSER_FEDERATED_DEMO.md).

The claim

Lensemble demonstrates a Tapestry-like browser-local federated adaptation run: federated adapter continuation on a frozen checkpoint. Each browser trains only a 12,512-parameter (0.069%) residual adapter on the frozen predictor output, then submits its clipped adapter delta and derived summaries to one local coordinator. This is not full-model or browser world-model training.

"Tapestry-like" means the concepts — sovereign participants, a shared protocol, bounded update artifacts, coordinator aggregation, observability, claim-bounded evidence — borrowed from Project Tapestry (AI Alliance), not its primitives and not its scale.

What was real

Component What actually happens Binding evidence
Base model The released quentinll/lewm-tworooms LeWorldModel checkpoint (MIT), pinned at revision 77adaae0bc31…, reconstructed in-tree and strict-loaded (303/303 tensors, 18,034,478 params); parity vs the actual upstream implementation ≤ 2e-5 max abs diff checkpoint manifest, reference report
Browser inference The browser downloads hash-pinned, weight-bearing ONNX graphs (encoder+projector, action encoder, predictor+pred_proj) exported deterministically from the pinned checkpoint, with ImageNet pixel normalization and the expert-dataset action z-score baked in; PyTorch-vs-onnxruntime parity ≤ 1e-4 export manifest, action stats
Pipeline correctness On held-out official expert episodes, the exported pipeline's teacher-forced next-latent MSE is 0.070 vs 1.306 for the copy-last baseline (ratio 0.054) real-data check
TwoRooms environment A deterministic JS port of the upstream env at the released default variations, validated pixel-level against the dataset frames (max abs diff 1/255); labeled a TwoRooms-compatible probe with its deviations documented web/federated-demo/tworooms_env.mjs
Browser-local training A bounded zero-init residual adapter (12,512 parameters, 0.069% of the checkpoint) on the frozen predictor output, trained with manual-gradient Adam in the browser; SIGReg Epps–Pulley diagnostics ported exactly from lensemble.model.sigreg (torch parity ≤ 2e-7 rel); one-browser overfit on real latents: pred loss 0.0559 → 0.00061 adapter overfit
Federation Each lewm-adapter-delta/1 clipped adapter delta is visible to and validated by a single local coordinator (schema, byte budget, shapes, norms, checkpoint/export-hash freshness, replay, fabricated-metric rejection), then aggregated as a deterministic per-round mean over a shared deterministic adapter init. This path has no secure aggregation or differential privacy tests/ml/test_lewm_federation.py
Before/after probe (system-composed) The system-composed probe is the headline: real node-trained adapter deltas flow through FederatedDemoService.submit_update (real validation) and _close_round_lewm (real deterministic-mean aggregation + hash-chained revisions), and the probe scores the server-produced final modelRevisionId, not an offline-recomputed mean. Held-out MSE improved 0.0604 → 0.0530 (+12.3%, verdict improved, 0 claim-audit violations) on the seed-20260612 draw. scripts/lewm_probe_check.py is only an offline math cross-check system-composed probe; offline math cross-check federated probe
Seed robustness The headline is not a single favorable draw: across 5 independent seeds / episode splits the system-composed probe improves on every seed with no collapse — mean +16.8%, worst case +5.4% (seed 2), best +32.6% seed sweep
Surprise-meter card The browser meter renders scalar CLS-latent surprise, a recorded fallback trajectory, the evidence-bound system-composed +12.3% run result, +16.8% seed mean, and +5.4% worst seed with non-claims visible in the page surprise evidence, web/surprise-meter/data/result_card.json, web/surprise-meter/data/surprise_trajectory.json
Held-out collapse check The gain is bias-correction, not the #259 held-out magnitude collapse: on the validation pairs the adapted latent std holds vs the frozen baseline (0.90 → 0.91) and effective rank is preserved (9.86 → 9.80), so an "improved" MSE cannot hide a collapse — a materially lower std/rank overrides the verdict to collapse-risk held-out diagnostics in system-composed probe
Honest failure states Round metrics carry server-side health flags (effective-rank collapse, latent-std magnitude collapse, flat/worsened loss, SIGReg outliers); the dashboard probe reports flat/worse verdicts instead of hiding them — the first probe configuration was flat and is recorded as such in the development history round metrics in every evidence export

Privacy and residency

  • Raw rollouts, labels, actions, intermediate activations, and latents remain in each participant browser in this demo protocol. The evidence audit rejects raw-data-like fields fail-closed.
  • The browser downloads hash-pinned, weight-bearing ONNX graphs. The residency boundary applies to participant-generated training material, not to secrecy of the frozen model weights.
  • The upload boundary contains clipped adapter deltas and derived summaries. A single local coordinator sees each individual delta before computing the deterministic mean; the deltas are neither anonymous nor hidden from that coordinator.
  • Secure aggregation (RFC-0011) and differential privacy (RFC-0012) exist elsewhere in Lensemble, but there is no secure aggregation or differential privacy in this demo path. The demo makes no cryptographic privacy claim for uploaded adapter deltas.

Non-claims

This demo is not:

  • full from-scratch LeWorldModel base training inside every browser — it is bounded federated adapter continuation on a frozen checkpoint;
  • full-model or browser world-model training;
  • production browser training;
  • paper-scale TwoRooms or PushT benchmark parity (the +12.3% probe result is a fixed held-out validation probe for this demo, single local coordinator with mean-of-clipped-deltas and no robust aggregation / DP in this path, not a benchmark win);
  • a cryptographic proof of honest computation;
  • closed-loop physical robotics success;
  • a privacy guarantee for individual adapter deltas or a model-secrecy mechanism.

Reproduce

uv run python scripts/lewm_tworooms_ingest.py
uv run --with onnx --with onnxscript --with onnxruntime python scripts/lewm_tworooms_export.py
uv run --with onnxruntime --with hdf5plugin python scripts/lewm_tworooms_realdata_check.py --h5 <tworoom.h5>
uv run --with onnxruntime --with hdf5plugin python scripts/lewm_adapter_overfit_check.py --h5 <tworoom.h5>
uv run --with onnxruntime --with hdf5plugin python scripts/lewm_probe_check.py --h5 <tworoom.h5>     # offline math cross-check
uv run --with onnxruntime --with hdf5plugin python scripts/lewm_system_probe.py --h5 <tworoom.h5>    # system-composed headline (#327)
uv run --with onnxruntime --with hdf5plugin python scripts/lewm_probe_seedsweep.py --h5 <tworoom.h5> # seed robustness (#330)
uv run --with onnxruntime --with hdf5plugin python scripts/surprise/run_clean_round.py --h5 <tworoom.h5> # offset sidecar + served fixture
uv run python scripts/lewm_surprise_check.py # surprise evidence + served fallback result/trajectory
uv run lensemble demo federated --port 8765   # then create a real-lewm-tworooms run

Run-level evidence (demo-evidence/1 with runMode: real-lewm-tworooms) is exported from the host dashboard and validated by lensemble.demo.evidence_audit — the claim audit fails closed on missing claim-boundary phrases, unbound revisions, missing hashes, raw participant data, or unnegated overclaims.