Japan AI Safety Institute (AISI) Brief

Headline

Japan AISI publishes experiment notes evaluating advanced cyber capability proliferation to open-weight AI models

Executive Summary

Japan's AI Safety Institute published experiment notes evaluating whether advanced cyber capabilities observed in frontier models have proliferated to openly available open-weight models. The evaluation, conducted in July 2026, assessed the exploit-development capabilities of GLM-5.2 against known V8 JavaScript engine vulnerabilities using the ExploitBench benchmark.

Bottom Line

Japan AISI's experiment notes establish that GLM-5.2, assessed as of July 2026, reaches a maximum of the T3 tier on ExploitBench across 41 V8 vulnerability tasks, with a mean score of 2.80 against 5.22 for Opus 4.8, and that the institute assesses advanced exploit-development capability proliferation to open-weight models as limited at the current point in time. The institute frames this as the first stage of continuing evaluation. Cresthaven reading: the publication places open-weight model developers and the broader AI governance community on notice that national AI safety bodies treat cyber capability proliferation in open-weight models as a standing evaluation priority, not a one-time assessment.

Key Regulatory Signals

• Evaluation Scope and Benchmark: Japan AISI used ExploitBench, a cybersecurity benchmark targeting known vulnerabilities in the V8 JavaScript engine used in Google Chrome, to measure how far AI agents can progress from vulnerability identification through attack-code generation and development. The evaluation covered 41 vulnerability tasks across a five-tier severity scale, from T5 (identification of vulnerable processing) to T1 (arbitrary operation outside the system).
• GLM-5.2 Performance Against Frontier Models: Across all 41 tasks, GLM-5.2 recorded a mean score of 2.80, below Opus 4.7 at 3.63 and Opus 4.8 at 5.22. GLM-5.2's highest tier reached was T3; Opus 4.8 reached T2 and T1 tiers. Japan AISI assessed GLM-5.2's exploit-code generation and development capability as lower than both Opus 4.7 and Opus 4.8 under ExploitBench conditions.
• Cost Efficiency Observation: Among the 36 tasks where all three models reached the T5 tier, token costs at the point of reaching T5 were USD 3.71 for GLM-5.2, USD 4.60 for Opus 4.7, and USD 7.34 for Opus 4.8. Japan AISI noted GLM-5.2 was more cost-efficient than the other two models on this measure.
• Hint Condition Testing on High-Risk Tasks: For 10 tasks selected to assess high-risk exploit-development capability including sandbox bypass, GLM-5.2 was tested with and without system-provided hints. GLM-5.2's mean score across those 10 tasks was 3.0 regardless of hint condition. Cresthaven reading: developers and deployers of open-weight models face an emerging institutional expectation, reflected in this first-stage evaluation, that cyber capability proliferation will be assessed on a continuing basis by national AI safety bodies.
• Proliferation Finding: Japan AISI assessed that, at the current point in time, the proliferation of advanced exploit-development capabilities to open-weight models is limited. The institute framed this evaluation as the first stage of ongoing monitoring of high-performance open-weight model cyber capabilities.

Regulatory Delta

No direct institutional precedent exists for a national AI safety institute publishing a structured, task-tiered exploit-development benchmark evaluation of open-weight models. This is a first-in-kind publication by Japan AISI.

The structural departure is the application of ExploitBench to compare an open-weight model against frontier closed models across 41 V8 vulnerability tasks on a five-tier severity scale, with a 10-task subset for GLM-5.2 selected to span attack surfaces, implementation paths and vulnerability disclosure dates.

The evaluation aligns with the cyber capability assessment agenda coordinated among AI safety institutes, including the UK AI Safety Institute and the US AI Safety Institute, both of which have published capability evaluations of frontier models. This publication extends that agenda explicitly to the proliferation risk posed by open-weight models.

Materiality Classification

MEDIUM — A first-in-kind capability evaluation publication by Japan AISI assessing cyber capability proliferation in open-weight AI models; directional policy signal for AI developers, deployers, and AI governance practitioners without an immediate binding compliance trigger.

Intelligence Outlook

Monitor Japan AISI for subsequent stages of this open-weight model cyber capability evaluation series and for any coordinated publication with peer AI safety institutes.