Catalog review: 2026-10-09#
This refresh reviewed every entry in the previous catalog for relevance, wording, canonical destination and artifact type. Repository metadata and publisher documentation were checked; selected license files and reproduction instructions were inspected. The review adds missing practitioner workflows and removes overlapping or unsupported recommendations.
The review used AI-assisted research and drafting, followed by editorial primary-source checks. It did not execute the listed benchmarks, validate their results independently, audit their security or establish broad adoption. A working link and a published experiment are different kinds of evidence.
Gaps addressed#
| Gap | Additions | What they let a reader inspect |
|---|---|---|
| Adversarial ML beyond language-model jailbreaks | Adversarial Robustness Toolbox | Evasion, poisoning, extraction and inference attacks, defenses and evaluation interfaces. |
| Runnable evaluations and agent threats | Inspect Evals, PyRIT, AgentDojo, AgentHarm | Task implementations, attack/scorer interfaces, indirect injection and harmful multi-step requests. AgentHarm's purpose restriction is explicit. |
| Behavioral auditing and control | Inspect Petri, Petri Bloom, ControlArena, Monitorability Evals | Simulated audits, seeded behavior tests, control protocols and public monitoring-evaluation splits. These do not establish natural prevalence of misalignment. |
| Cyber and scientific capabilities | CyberGym, Cybench, LABBench2 | Real-software PoC reproduction, CTF environments and scientific research tasks. Keep knowledge, simulated actions and operational outcomes distinct. |
| Interpretation and interventions | circuit-tracer, Gemma Scope 2, OpenUnlearning | Attribution graphs, SAE/transcoder artifacts and comparisons of approximate unlearning methods. |
| Supply-chain and formal properties | Model Transparency, Fickling, Marabou, RFC 9334 | Signed model artifacts, serialized-file inspection, bounded neural-network queries and attestation roles. |
| Governance and assurance | NIST GenAI Profile, SP 800-218A, GPAI Code, OWASP LLM/Agentic guidance, agent security cheat sheet, AISI safety-case template | Official guidance, application threat models and explicit assurance arguments. |
| Research counterexamples and oversight | AI Control, empirical debate, Emergent Misalignment, Unlearning or Obfuscating?, SAEBench, AuditBench, reward-seeker report | Methods and limits beyond refusal scores and preference fit. |
Source and terms checks for new code#
These source snapshots support functionality and reproduction checks. The stated license refers to the inspected project grant, not every third-party dataset, model or asset. Later versions may change terms or interfaces.
| Resource | Source snapshot | Inspected project terms |
|---|---|---|
| Adversarial Robustness Toolbox | README | MIT; model and data terms remain separate. |
| Inspect Evals | README | MIT; per-task terms can differ. |
| PyRIT | README | MIT. |
| AgentDojo | README | MIT. |
| ControlArena | README | MIT. |
| Inspect Petri | README | MIT; current internal APIs differ from Petri 2.0. |
| Petri Bloom | Repository | MIT; replaces the frozen standalone Bloom repository. |
| CyberGym | README | Apache-2.0; target software has separate terms. |
| Cybench | README | Apache-2.0; task assets have separate terms. |
| LABBench2 | README | CC BY-SA 4.0; check underlying materials. |
| circuit-tracer | README | MIT; compatible transcoder weights required. |
| OpenUnlearning | Repository | MIT; dataset and model terms vary. |
| Monitorability Evals | README | Apache-2.0 with upstream dataset attribution; private data omitted. |
| Model Transparency | Repository | Apache-2.0. |
| Fickling | Repository | LGPL-3.0. |
| Marabou | COPYING | Modified BSD terms; inspect the complete grant. |
Gemma Scope 2's model card states CC BY 4.0 for the interpretability artifacts. It is a landing page linking model-specific weights. The base models have separate terms. AgentHarm's documentation specifies MIT plus a safety/security purpose restriction, so it is labeled restricted rather than unrestricted OSS.
Corrections and consolidation#
- Training versus safeguards: moved the Alignment Handbook to post-training. Removed unsupported standard/primary-library language and scale claims from TRL, OpenRLHF and TransformerLens. Removed output quality-guarantee wording from Guardrails AI.
- Evaluation semantics: StrongREJECT assesses refusal and harmful-answer usefulness, not just refusal frequency. Anthropic Evals is historical model-written data, not a current dangerous-capability suite. WMDP is a knowledge proxy. Vectara's leaderboard is source-conditioned summarization consistency with a particular judge.
- Canonical sources: updated Petri's destination and added Meridian's maintained Petri Bloom. Linked VNN-COMP's competition site rather than only its 2024 repository. Replaced the third-party AI Act overview with the Commission page. Used the published OWASP 2026 editions.
- Research framing: corrected Concrete Problems' five topics, including side effects. Scoped power-seeking claims to the paper's assumptions. Bounded Sleeper Agents, Alignment Faking and reward-tampering descriptions to their setups. Moved Claude's Constitution to behavioral specifications.
- Duplicates: replaced the broad PurpleLlama entry with its distinct Llama Guard and CyberSecEval destinations. Consolidated auxiliary activation/SAE libraries into current analysis interfaces. Removed the duplicate Anthropic research feed.
- Navigation: removed unverified location columns, linked direct curricula, and replaced the undocumented eight-week BlueDot duration with a curriculum description.
Removed or held recommendations#
Removal is an editorial scope or evidence decision. It does not imply that the resource is malicious or useless.
| Resource | Decision and primary evidence |
|---|---|
| LLM Guard | Removed from current application recommendations. Its README states that the project is no longer maintained. |
| METR Task Standard and Vivaria | Removed from new-project recommendations. Both Task Standard and Vivaria are archived and direct new work toward Inspect. Public tasks remain separately listed. |
| Asqav | Excluded under the code-license criterion. The SDK README identifies Elastic License 2.0 and hosted, server-side cryptography. Public source is not the same as unrestricted OSS. |
| EZKL | Held as an OSS recommendation. No project reuse grant was located in the inspected current tree. Its README describes inference circuits and a no-warranty notice; that notice alone is not a license. Removed the unsupported maturity ranking. |
| Randomized Smoothing implementation | Replaced with the paper. The repository preserves the experiment but no project license was located in the inspected tree. |
| EU AI Act Toolkit | Held pending legal-data refresh. Its timeline table conflicts with the Commission's current implementation guidance. The catalog maintainer owns this toolkit; the same criteria apply. |
| Credo AI and Holistic AI | Removed commercial product-directory entries. The catalog prioritizes public methods, artifacts and official guidance. |
| Future AGI platform, Agent Learning Kit and traceAI | Removed overlapping agent-engineering and telemetry entries whose distinct safety role was not established by the catalog description. |
| DeepSpeed-Chat, Baukit, repeng and early OpenAI SAE implementation | Consolidated overlapping training or analysis interfaces; historical experiments remain discoverable through the retained research routes. |
| RISC Zero and archived awesome-zkml | Removed generic proof infrastructure and a secondary curated list; retained concrete inference research artifacts with scope labels. |
| Generic RL course, Bitter Lesson, general AI wiki, Meta FAIR and Frontier Model Forum | Removed peripheral or overlapping reading/directory routes. The old RL course destination redirected to a generic homepage. |
SAEBench and AuditBench enter as papers: their abstracts support the research descriptions, but a project reuse license was not located in the inspected code roots, READMEs or package metadata: SAEBench at 8042bb3 and auditing-agents at e908591. No claim of unrestricted code reuse is made. HELM entered maintenance mode; its safety limitations remain useful methodological reading, without an active-leaderboard recommendation. BountyBench is held for a future dated-artifact review; its linked project site did not resolve during this audit.
Follow-up#
The hosted link sweep returned HTTP 403 for OpenAI's safety page on two attempts, while the primary page remained reachable and was manually read on 2026-10-09. .lychee.toml exempts that exact URL only. This is a manually verified exception, not a successful automated request. Recheck it during the next editorial review.
The next review should revisit held licenses and changed benchmark/scorer releases, check newly published control and monitoring methods, and refresh official governance sources. No general reliability or regulatory-compliance result is claimed by this catalog.