{"blocked_controls":[],"claim_boundary":"Historical evidence status: these training and gauge rows predate the outer-update direction correction and the public-probe-v2 target-binding contract. They are retained for audit history only and do not validate the corrected runtime; rerun tracking is recorded in GitHub issue #335. Historical claim boundary: This Phase 3 eval report supports consortium-runtime engineering evidence from a local tiny-model run and explicit blockers for public task-scale controls. Post-commit sum cross-checks are not counted as secure aggregation, and deterministic-noise epsilon rows are one-round mechanism snapshots rather than effective DP. It does not claim paper-scale LeWorldModel performance or SO-100 robotics task success.","eval_plan":{"generated_at":"2026-06-05T00:00:00Z","matched_control_policy":"A matched Phase 3 control must use the same participant ids, dataset/probe registry, model size, seeds, DP policy, secure-aggregation mode, eval budget, and report bindings as the anchored federation run.","plan_id":"phase3-consortium-eval-plan-v1","raw_data_in_report":false,"tasks":[{"blocker":null,"expected_outcomes":["all declared participant agents submit every assigned round","every closed round reports its optimizer-input source, any post-commit sum cross-check, and DP snapshot status"],"falsifying_outcomes":["any missing participant update without an explicit dropout row","any cross-check mislabeled as an optimizer-consumed secure sum","any deterministic replay row mislabeled as effective DP"],"goal_policy":"not applicable; lifecycle metrics evaluate round closure, participant release, post-commit aggregation cross-checks, and one-round DP mechanism-accounting snapshots","held_out_policy":"synthetic participant-local windows are generated inside each Phase3ParticipantAgent and never serialized in this report","metrics":["closed_round_completion_rate","participant_submission_rate","secure_sum_round_rate","dp_accounted_round_rate","latent_frame_drift_deg","effective_rank","latent_frame_drift_deg","effective_rank","latent_frame_drift_deg","effective_rank","latent_frame_drift_deg","effective_rank"],"planner_budget":{"action_dim":0,"eval_episodes":0,"horizon":0,"notes":"No latent-MPC planner is executed for the local consortium lifecycle smoke; task-scale planner eval is blocked separately.","planner":"not_applicable","planner_iterations":0,"planning_samples":0},"seeds":[0],"status":"completed","task_env_id":"phase3://consortium-long-run-smoke","task_scale":"local deterministic tiny-model consortium lifecycle eval"},{"blocker":"No public Phase 3 task-scale checkpoint and held-out SO-100 eval dataset are published yet; #230 owns final artifact publication.","expected_outcomes":["published checkpoint loads and executes latent-MPC on held-out task episodes"],"falsifying_outcomes":["checkpoint cannot be loaded","metric rows cannot be bound to checkpoint/config/run-manifest hashes"],"goal_policy":"task goals and planner budget must be fixed before launch and cited in the model card","held_out_policy":"final local participant episodes or an equivalent public held-out SO-100 split must be published before this eval can run","metrics":["task_success_rate","time_per_action_ms","effective_dim"],"planner_budget":{"action_dim":2,"eval_episodes":4,"horizon":2,"notes":"Reserved task-scale budget matching the compact Phase 2 downstream eval until public Phase 3 checkpoints/data exist.","planner":"icem","planner_iterations":4,"planning_samples":8},"seeds":[0],"status":"blocked","task_env_id":"so100-heldout://phase3-public-task-scale","task_scale":"public SO-100 held-out task-scale downstream eval"}]},"evidence_status":"historical_pre_correctness_fix","generated_at":"2026-06-05T00:00:00Z","metric_rows":[{"checkpoint_hash":"ed3081ee514af142a226443f113a37c24d7d5872bfb707f11abe10893a0ad50d","config_hash":"a4f6bbabb919735c7679320c3f204cd0b1298e046e30cce8d11cbbbc8df07e57","control_role":"consortium-runtime-smoke","metric":"closed_round_completion_rate","notes":"closed rounds divided by declared target rounds","planner_budget":{"action_dim":0,"eval_episodes":0,"horizon":0,"notes":"No latent-MPC planner is executed for the local consortium lifecycle smoke; task-scale planner eval is blocked separately.","planner":"not_applicable","planner_iterations":0,"planning_samples":0},"row_id":"consortium-runtime-smoke.closed-round-completion-rate","run_manifest_hash":"cfaf14dcbe7c3fe9e64f2257729c047df448bd35218b387412f6a5d479c3169a","seed":0,"source_report_sha256":"573c4248cabafe994478ae716f39fadd6d10ba009f64fa489add940f9abe77a0","task_env_id":"phase3://consortium-long-run-smoke","value":1.0},{"checkpoint_hash":"ed3081ee514af142a226443f113a37c24d7d5872bfb707f11abe10893a0ad50d","config_hash":"a4f6bbabb919735c7679320c3f204cd0b1298e046e30cce8d11cbbbc8df07e57","control_role":"consortium-runtime-smoke","metric":"participant_submission_rate","notes":"participant submitted rounds divided by assigned participant-rounds","planner_budget":{"action_dim":0,"eval_episodes":0,"horizon":0,"notes":"No latent-MPC planner is executed for the local consortium lifecycle smoke; task-scale planner eval is blocked separately.","planner":"not_applicable","planner_iterations":0,"planning_samples":0},"row_id":"consortium-runtime-smoke.participant-submission-rate","run_manifest_hash":"cfaf14dcbe7c3fe9e64f2257729c047df448bd35218b387412f6a5d479c3169a","seed":0,"source_report_sha256":"573c4248cabafe994478ae716f39fadd6d10ba009f64fa489add940f9abe77a0","task_env_id":"phase3://consortium-long-run-smoke","value":1.0},{"checkpoint_hash":"ed3081ee514af142a226443f113a37c24d7d5872bfb707f11abe10893a0ad50d","config_hash":"a4f6bbabb919735c7679320c3f204cd0b1298e046e30cce8d11cbbbc8df07e57","control_role":"consortium-runtime-smoke","metric":"secure_sum_round_rate","notes":"closed rounds whose optimizer consumed a secure sum; post-commit cross-checks do not count","planner_budget":{"action_dim":0,"eval_episodes":0,"horizon":0,"notes":"No latent-MPC planner is executed for the local consortium lifecycle smoke; task-scale planner eval is blocked separately.","planner":"not_applicable","planner_iterations":0,"planning_samples":0},"row_id":"consortium-runtime-smoke.secure-sum-round-rate","run_manifest_hash":"cfaf14dcbe7c3fe9e64f2257729c047df448bd35218b387412f6a5d479c3169a","seed":0,"source_report_sha256":"573c4248cabafe994478ae716f39fadd6d10ba009f64fa489add940f9abe77a0","task_env_id":"phase3://consortium-long-run-smoke","value":0.0},{"checkpoint_hash":"ed3081ee514af142a226443f113a37c24d7d5872bfb707f11abe10893a0ad50d","config_hash":"a4f6bbabb919735c7679320c3f204cd0b1298e046e30cce8d11cbbbc8df07e57","control_role":"consortium-runtime-smoke","metric":"dp_accounted_round_rate","notes":"closed rounds with a one-round mechanism-accounting epsilon snapshot; not cumulative or effective DP","planner_budget":{"action_dim":0,"eval_episodes":0,"horizon":0,"notes":"No latent-MPC planner is executed for the local consortium lifecycle smoke; task-scale planner eval is blocked separately.","planner":"not_applicable","planner_iterations":0,"planning_samples":0},"row_id":"consortium-runtime-smoke.dp-accounted-round-rate","run_manifest_hash":"cfaf14dcbe7c3fe9e64f2257729c047df448bd35218b387412f6a5d479c3169a","seed":0,"source_report_sha256":"573c4248cabafe994478ae716f39fadd6d10ba009f64fa489add940f9abe77a0","task_env_id":"phase3://consortium-long-run-smoke","value":1.0},{"checkpoint_hash":"223b406732cf2e078626ec65d9bc18a46620068d3d0d8bbacf8421c3fcbe8c7c","config_hash":"477315589725ec35563289f43a29b64e455e804dec715f3cbc644acf213d33fe","control_role":"anchored-federation","metric":"latent_frame_drift_deg","notes":"Anchored-federation (lambda_anc=0.01) control; the producer extracts its round-0 gauge values from the immutable report. (round-0 inter-participant latent frame-drift at aggregation); checkpoint revision 567755d2e928327ae1624e86807b0ab8170fc786 of abdelstark/lensemble-phase3-consortium-anchored-probe","planner_budget":{"action_dim":0,"eval_episodes":0,"horizon":0,"notes":"No latent-MPC planner is executed for the local consortium lifecycle smoke; task-scale planner eval is blocked separately.","planner":"not_applicable","planner_iterations":0,"planning_samples":0},"row_id":"anchored-federation.latent-frame-drift-deg","run_manifest_hash":"3be2335bb5f26f8b0b54ffd10b219114d1d09a7110031ba409f93220257482d4","seed":0,"source_report_sha256":"35e32ab2e0f2a7c382ad612e402bf2108d4b06dc7aeff3fb30c460541b62491f","task_env_id":"phase3://consortium-control-anchored-federation","value":48.96953973360461},{"checkpoint_hash":"223b406732cf2e078626ec65d9bc18a46620068d3d0d8bbacf8421c3fcbe8c7c","config_hash":"477315589725ec35563289f43a29b64e455e804dec715f3cbc644acf213d33fe","control_role":"anchored-federation","metric":"effective_rank","notes":"Anchored-federation (lambda_anc=0.01) control; the producer extracts its round-0 gauge values from the immutable report. (round-0 global-representation effective rank); checkpoint revision 567755d2e928327ae1624e86807b0ab8170fc786 of abdelstark/lensemble-phase3-consortium-anchored-probe","planner_budget":{"action_dim":0,"eval_episodes":0,"horizon":0,"notes":"No latent-MPC planner is executed for the local consortium lifecycle smoke; task-scale planner eval is blocked separately.","planner":"not_applicable","planner_iterations":0,"planning_samples":0},"row_id":"anchored-federation.effective-rank","run_manifest_hash":"3be2335bb5f26f8b0b54ffd10b219114d1d09a7110031ba409f93220257482d4","seed":0,"source_report_sha256":"35e32ab2e0f2a7c382ad612e402bf2108d4b06dc7aeff3fb30c460541b62491f","task_env_id":"phase3://consortium-control-anchored-federation","value":2.2253127098083496},{"checkpoint_hash":"56ceb16e55528e0fbab20467f61ca38a9d9593f030176e4a3759bcd8b5c695fe","config_hash":"c71fc2843d3fa5e424434492ea3189709bd6054fc8348b7596121f24a5081d93","control_role":"naive-fedavg","metric":"latent_frame_drift_deg","notes":"Naive-FedAvg (lambda_anc=0) unanchored control: round-0 frame-drift 180 deg, the maximal divergence the frame anchor reduces to 48.97 deg. (round-0 inter-participant latent frame-drift at aggregation); checkpoint revision 1aace2251fe95dfe10517c19e24603d6c14d32f8 of abdelstark/lensemble-phase3-consortium-naive-fedavg","planner_budget":{"action_dim":0,"eval_episodes":0,"horizon":0,"notes":"No latent-MPC planner is executed for the local consortium lifecycle smoke; task-scale planner eval is blocked separately.","planner":"not_applicable","planner_iterations":0,"planning_samples":0},"row_id":"naive-fedavg.latent-frame-drift-deg","run_manifest_hash":"b57a5a8109d30769d69838a01a8f4043776797ad03c18b48a742071b129e6676","seed":0,"source_report_sha256":"a9f93f8adb8d1fdaf6dc80997e055cba2c8a478f50e89673f104515d24ee29a7","task_env_id":"phase3://consortium-control-naive-fedavg","value":180.0},{"checkpoint_hash":"56ceb16e55528e0fbab20467f61ca38a9d9593f030176e4a3759bcd8b5c695fe","config_hash":"c71fc2843d3fa5e424434492ea3189709bd6054fc8348b7596121f24a5081d93","control_role":"naive-fedavg","metric":"effective_rank","notes":"Naive-FedAvg (lambda_anc=0) unanchored control: round-0 frame-drift 180 deg, the maximal divergence the frame anchor reduces to 48.97 deg. (round-0 global-representation effective rank); checkpoint revision 1aace2251fe95dfe10517c19e24603d6c14d32f8 of abdelstark/lensemble-phase3-consortium-naive-fedavg","planner_budget":{"action_dim":0,"eval_episodes":0,"horizon":0,"notes":"No latent-MPC planner is executed for the local consortium lifecycle smoke; task-scale planner eval is blocked separately.","planner":"not_applicable","planner_iterations":0,"planning_samples":0},"row_id":"naive-fedavg.effective-rank","run_manifest_hash":"b57a5a8109d30769d69838a01a8f4043776797ad03c18b48a742071b129e6676","seed":0,"source_report_sha256":"a9f93f8adb8d1fdaf6dc80997e055cba2c8a478f50e89673f104515d24ee29a7","task_env_id":"phase3://consortium-control-naive-fedavg","value":1.0893148183822632},{"checkpoint_hash":"cbcf3bbcc97d5fa42be3663b207ae0933c09a5424ec75c3d1fa1b60e61b1aa7f","config_hash":"b3ce1aa02ac26ade217bec8badcd0d5515d605c10205f1cf3dd3452e927efc8f","control_role":"fork-a-frozen-encoder","metric":"latent_frame_drift_deg","notes":"Fork-A frozen-encoder safe-degrade baseline: constant 0 deg frame-drift and constant effective rank, the structural lower bound on divergence. (round-0 inter-participant latent frame-drift at aggregation); checkpoint revision 148e4217fc25ef8a29f2a8bbc4a344c7a07b8a7d of abdelstark/lensemble-phase3-consortium-fork-a","planner_budget":{"action_dim":0,"eval_episodes":0,"horizon":0,"notes":"No latent-MPC planner is executed for the local consortium lifecycle smoke; task-scale planner eval is blocked separately.","planner":"not_applicable","planner_iterations":0,"planning_samples":0},"row_id":"fork-a-frozen-encoder.latent-frame-drift-deg","run_manifest_hash":"37504acd556d6fc45b32044152cde86d22ca98bdbcaf2a3c03e14984dcebf03c","seed":0,"source_report_sha256":"bec998d6be48f8e16e76e6abef6519b8d53cf579b78741587b9a6fec49f6fe14","task_env_id":"phase3://consortium-control-fork-a-frozen-encoder","value":0.0},{"checkpoint_hash":"cbcf3bbcc97d5fa42be3663b207ae0933c09a5424ec75c3d1fa1b60e61b1aa7f","config_hash":"b3ce1aa02ac26ade217bec8badcd0d5515d605c10205f1cf3dd3452e927efc8f","control_role":"fork-a-frozen-encoder","metric":"effective_rank","notes":"Fork-A frozen-encoder safe-degrade baseline: constant 0 deg frame-drift and constant effective rank, the structural lower bound on divergence. (round-0 global-representation effective rank); checkpoint revision 148e4217fc25ef8a29f2a8bbc4a344c7a07b8a7d of abdelstark/lensemble-phase3-consortium-fork-a","planner_budget":{"action_dim":0,"eval_episodes":0,"horizon":0,"notes":"No latent-MPC planner is executed for the local consortium lifecycle smoke; task-scale planner eval is blocked separately.","planner":"not_applicable","planner_iterations":0,"planning_samples":0},"row_id":"fork-a-frozen-encoder.effective-rank","run_manifest_hash":"37504acd556d6fc45b32044152cde86d22ca98bdbcaf2a3c03e14984dcebf03c","seed":0,"source_report_sha256":"bec998d6be48f8e16e76e6abef6519b8d53cf579b78741587b9a6fec49f6fe14","task_env_id":"phase3://consortium-control-fork-a-frozen-encoder","value":2.387617588043213},{"checkpoint_hash":"2146dffe1e69866b64352e42c46706035f86d9d8065cde7a444c7fc57b99bdfc","config_hash":"2146dffe1e69866b64352e42c46706035f86d9d8065cde7a444c7fc57b99bdfc","control_role":"local-only","metric":"latent_frame_drift_deg","notes":"No-aggregation local-only control: silos train healthily but diverge maximally; bound to the published report sha256 and revision, not a global checkpoint. (inter-participant latent frame-drift across isolated silos); checkpoint revision a696da17ac2044914b91a69c5d748c79628c06b0 of abdelstark/lensemble-phase3-consortium-local-only","planner_budget":{"action_dim":0,"eval_episodes":0,"horizon":0,"notes":"No latent-MPC planner is executed for the local consortium lifecycle smoke; task-scale planner eval is blocked separately.","planner":"not_applicable","planner_iterations":0,"planning_samples":0},"row_id":"local-only.latent-frame-drift-deg","run_manifest_hash":"2146dffe1e69866b64352e42c46706035f86d9d8065cde7a444c7fc57b99bdfc","seed":0,"source_report_sha256":"2146dffe1e69866b64352e42c46706035f86d9d8065cde7a444c7fc57b99bdfc","task_env_id":"phase3://consortium-control-local-only","value":180.0},{"checkpoint_hash":"2146dffe1e69866b64352e42c46706035f86d9d8065cde7a444c7fc57b99bdfc","config_hash":"2146dffe1e69866b64352e42c46706035f86d9d8065cde7a444c7fc57b99bdfc","control_role":"local-only","metric":"effective_rank","notes":"No-aggregation local-only control: silos train healthily but diverge maximally; bound to the published report sha256 and revision, not a global checkpoint. (mean per-participant held-out effective rank (healthy local training)); checkpoint revision a696da17ac2044914b91a69c5d748c79628c06b0 of abdelstark/lensemble-phase3-consortium-local-only","planner_budget":{"action_dim":0,"eval_episodes":0,"horizon":0,"notes":"No latent-MPC planner is executed for the local consortium lifecycle smoke; task-scale planner eval is blocked separately.","planner":"not_applicable","planner_iterations":0,"planning_samples":0},"row_id":"local-only.effective-rank","run_manifest_hash":"2146dffe1e69866b64352e42c46706035f86d9d8065cde7a444c7fc57b99bdfc","seed":0,"source_report_sha256":"2146dffe1e69866b64352e42c46706035f86d9d8065cde7a444c7fc57b99bdfc","task_env_id":"phase3://consortium-control-local-only","value":120.31616020202637}],"model_card_eval_text":"Historical evidence status: these training and gauge rows predate the outer-update direction correction and the public-probe-v2 target-binding contract. They are retained for audit history only and do not validate the corrected runtime; rerun tracking is recorded in GitHub issue #335. Phase 3 evaluation evidence covers the local deterministic consortium-runtime smoke (participant-agent updates, 10 closed rounds, post-commit sum cross-check reporting, and one-round deterministic mechanism-accounting snapshots). Matched gauge controls are included only when their immutable reports and run manifests are supplied to the producer. Hash-bound historical gauge finding: anchored round-0 frame drift 48.9695 deg versus naive-FedAvg 180 deg; Fork-A 0 deg; local-only drift 180 deg with mean effective rank 120.316. These representation metrics are not robotics task-performance results. Historical limitation: at the old default outer step (outer_lr=0.7), the federated global representation collapsed over rounds (effective_rank -> 1). That trajectory also predates the outer-update correction, so it is failure-analysis history rather than a clean anchored-vs-naive result. Closed-loop task-scale SO-100 evaluation remains unrun; the published recorded splits cannot apply arbitrary planner actions to a live environment. Completed matched controls bound to published run hashes: anchored-federation, naive-fedavg, fork-a-frozen-encoder, local-only. These are representation-gauge controls and must not be described as completed robotics performance comparisons.","schema_version":2,"source_artifacts":[{"label":"Phase 3 long-run orchestration report","schema_name":"phase3_long_run_report","schema_version":2,"sha256":"573c4248cabafe994478ae716f39fadd6d10ba009f64fa489add940f9abe77a0","uri":"docs/evidence/phase3_long_run_smoke_report.json"},{"label":"Phase 3 anchored-federation control run report","schema_name":"phase3_long_run_report","schema_version":1,"sha256":"35e32ab2e0f2a7c382ad612e402bf2108d4b06dc7aeff3fb30c460541b62491f","uri":"abdelstark/lensemble-phase3-consortium-anchored-probe"},{"label":"Phase 3 naive-fedavg control run report","schema_name":"phase3_long_run_report","schema_version":1,"sha256":"a9f93f8adb8d1fdaf6dc80997e055cba2c8a478f50e89673f104515d24ee29a7","uri":"abdelstark/lensemble-phase3-consortium-naive-fedavg"},{"label":"Phase 3 fork-a-frozen-encoder control run report","schema_name":"phase3_long_run_report","schema_version":1,"sha256":"bec998d6be48f8e16e76e6abef6519b8d53cf579b78741587b9a6fec49f6fe14","uri":"abdelstark/lensemble-phase3-consortium-fork-a"},{"label":"Phase 3 local-only (no-aggregation) control report","schema_name":"phase3_local_only_report","schema_version":1,"sha256":"2146dffe1e69866b64352e42c46706035f86d9d8065cde7a444c7fc57b99bdfc","uri":"abdelstark/lensemble-phase3-consortium-local-only"}],"superseded_reason":"This run predates the correction of the outer-update direction and the public-probe-v2 target-binding contract. Its training and gauge metrics are retained for audit history only and do not validate the corrected runtime; rerun tracking is recorded in GitHub issue #335."}