The National AI Governance Playbook · Part IV · Chapter 10
The safety and security capability
Every chapter so far leans on one asset: the capability that evaluates models and accredits assurance. This chapter builds it: the institutional forms in public practice, the mandate and its triggers, independence protected structurally rather than by convention, and the three inputs of talent, access and funding.
By Myriam Ayada · MindXO · July 2026
In brief
The safety and security regime of Chapter 4 runs on one asset: a national capability that evaluates models, tests their security and accredits the assurance market. The capability works as an engine. Three inputs feed it: talent, access and funding. Two products leave it: evidence and accredited assurance. One property is non-negotiable: independence from those it measures and from those who act on its findings. Four institutional forms recur in public practice, and the choice among them, the triggers written into the mandate, and the sourcing of the three inputs are the design decisions this chapter works through.
The engine of the horizontal regime
Every earlier chapter leans on the same asset without having built it. The standards of Chapter 3 bind only where a body can test models against them. The safety and security regime of Chapter 4, the horizontal one, exists as working machinery only where evaluation and accreditation operate. The chain of Chapter 5 closes only where link six produces evidence a qualified assessor has verified. This chapter builds the asset behind all three: the safety and security capability, the national function that evaluates models, tests their security and accredits the assurance market.
In Chapter 3's vocabulary, the capability combines the evaluation and testing function with the public half of assurance. Evaluation and testing measures model behaviour against the adopted reference points; Chapter 3 described it as the youngest of the four functions and the most demanding to build. Accreditation is the state's share of the assurance function: by accrediting private assessors against published schemes, among them certification to ISO/IEC 42001, the state lets an assurance market exist, and verification scales through firms instead of through a public counter.
Two earlier decisions already commit a government to this build. The sequencing of Chapter 7 places the capability ahead of the obligations that will rely on it, since a rulebook that cites standards presumes somebody can test against them, and an assurance obligation presumes accredited assessors. Chapter 3's cost analysis marked its funding profile as the sharpest of the four: capital- and talent-intensive, with the cost incurred upfront, before any fee or accreditation charge can offset it. Both findings point the same way. The capability is paid for years before it pays back.
Sheet 10 of 13, the engine: three inputs, two products, one property. Talent, access and funding enter from the left and feed a single capability block holding three functions: evaluation, security testing and accreditation. Two products leave to the right: evidence, flowing to the rulebooks, the design stage and the public record, links five and six of Chapter 5, and accredited assurance, flowing to the market, the assurance function of Chapter 3. Under the block runs the independence bar: findings cannot be ordered, and the producer is not the actor.
Institutional form
Four institutional forms recur in public practice. Each buys pace, depth or economy at a different price, and the price is usually paid in independence, in salaries or in time.
Form A. An institute inside a ministry
The United Kingdom created its institute inside government in 2023; it operates today as the AI Security Institute. Strengths: fast to stand up, since no statute is required; close to policy, so evidence reaches decisions while they are still open; able to draw on the centre of government for access negotiations. Limits: independence rests on convention. The proximity that moves findings into policy also exposes them to it, and the protection lasts as long as the convention does. Fits when: the jurisdiction wants a working capability within a year, hosts significant development or deployment, and is prepared to write structural protections into the mandate from the start.
Form B. A designated national centre
Singapore designated its Digital Trust Centre as the national AI safety institute in 2024. Strengths: builds on research capacity that already exists: staff, methods and university partnerships arrive with the designation instead of being hired one by one. Limits: university pace and pay scales. Academic calendars set the tempo, and recruitment competes for industry-priced engineers on public research terms. Fits when: the jurisdiction holds a strong research base and prefers to designate and fund an existing institution over chartering a new one.
Form C. A distributed model
NIST anchors the methods in the United States, its Center for AI Standards and Innovation carries the mandate, and a consortium of external labs supplies capacity. Strengths: leverages an ecosystem the state never has to build: specialist labs, universities and firms contribute methods, staff and compute under a coordinating mandate. Limits: coordination becomes the product. The central body's work shifts from testing models to aligning testers, and coherence depends on its convening authority holding. Fits when: a deep domestic evaluation ecosystem already exists, together with a standards body credible enough to convene it.
Form D. Participation without a domestic build
The small-state option: methods and results arrive through the international network of AI safety institutes, and a small domestic function receives them. Strengths: the cheapest form: shared methods, shared evaluations and pooled exercises, with a receiving function to staff and no laboratory to run. Limits: it presumes the network's outputs fit domestic priorities. The members who test set the agenda, and a jurisdiction that only receives does not choose what gets tested, in which languages, or against which deployment contexts. Fits when: small states without frontier development, where competent uptake of the network's evidence serves better than a thin domestic laboratory.
The forms combine in practice. A ministry institute contracts external labs for specialist tests; a designated centre carries the receiving function for the network; a distributed model still requires one body whose signature accredits. What no combination removes is the design decision underneath: which entity owns the findings, and what protects them.
Mandate and triggers
A workable mandate answers three questions: what the capability tests, when a test is triggered, and what happens to the result. Chapter 3 set out the triggers, and three recur. Pre-deployment evaluation applies where rules require it, as Regulation (EU) 2024/1689 does for the systems and general-purpose models it places in scope. Post-incident examination follows a failure in deployment, where the question is what the model did and under which conditions. Periodic re-evaluation follows from the fact that models change: weights update, fine-tunes ship, and an evaluation from eighteen months ago describes a system that no longer runs.
The mandate covers security as well as safety, and the two share access, tooling and much of the talent. Security testing measures adversarial robustness, how a model behaves under deliberately hostile inputs, and misuse potential, what a capable actor could extract from it. The United Kingdom's renaming of its institute to the AI Security Institute in 2025 records where that emphasis settled.
Publication policy belongs in the mandate as a design decision, settled once and applied per finding. Chapter 3 identified what publication does: it converts evidence into market pressure, since a published evaluation moves procurement, insurance and reputation without any legal instrument. Selective publication has equally concrete grounds: some security findings would instruct the misuse they describe, and some access agreements hold only because commercial confidence is protected. The design work is a published disclosure rule: what is released, at what level of detail, on what timetable, and who decides the contested cases.
Independence, protected structurally
Chapter 4 named the failure mode: the entity that produces the evidence must not be the actor that decides on it, or every finding doubles as a decision and both lose credibility. For the capability, the requirement runs in two directions. Independence from developers keeps the measurement honest; independence from regulators and ministries keeps the evidence usable by all of them. Convention can protect this for years. Structure protects it on the day a finding is contested. Four separations do the work.
- A mandate that names what cannot be ordered. The founding instrument states that findings belong to the capability: an evaluation can be commissioned, and its result cannot be directed. Written down, the protection survives a change of minister.
- A governing body where regulators sit as recipients. Regulators and ministries take seats as recipients of evidence with a voice on priorities, and the chair sits elsewhere. A board its recipients control turns the capability into an arm of the strongest member.
- Staff who do not rotate directly into the entities they measured. Cooling-off periods for evaluators mirror those applied to supervisors, since an evaluator negotiating employment with the developer under test carries a conflict no methodology can correct.
- A budget line that does not pass through a supervised program. Funding routed through a policy program hands the program's owner a lever over the findings. A direct appropriation with a multi-year horizon removes the lever.
Talent, access, and money
Three inputs feed the engine, and each is scarce in its own way. Talent is priced by industry, and a public capability competes for it all the same. The instruments that work are structural: secondments from industry with a fixed return date, joint appointments that let a researcher hold a university post and an institute post at once, and staff exchanges through the international network of AI safety institutes, which give a small team access to depth it cannot hire.
Access determines whether the capability evaluates or comments. Meaningful testing needs the model, at a depth the test dictates, and some security work needs compute at scale. Neither arrives by default. Access agreements with developers set depth, conditions and confidentiality; participation in the network's shared evaluations spreads both the burden and the methods; and where rules require pre-deployment evaluation, access becomes a condition of market entry rather than a negotiated favour.
Funding comes first from public money, on a multi-year horizon, or the build does not happen. The cost profile from Chapter 3 applies at full scale: the spend is upfront, in salaries, compute and facilities, while the offsetting revenues, accreditation charges and scheme fees, arrive only once an assurance market exists to pay them. Fees can follow; they cannot lead. The workable pattern is a direct appropriation sized for years one to three, with fee income treated as a later offset. Chapter 12 supplies the measurement discipline for a build whose outputs lag its spending by years.
Common failure mode. An institute in name. A capability is announced, branded and admitted to the international network, with no compute, no access agreements and three staff. It attends the summits of Chapter 1 and produces no evidence, and the rulebooks that planned to cite its findings have nothing to cite. The opposite arrangement fails more quietly: a well-funded capability placed inside a regulator, where the first contested finding tests an independence that was never structural.
The engine is now specified: three inputs sourced, two products flowing, independence written into structure. The capability's security work opens onto the hardest deployment surface a government supervises. Chapter 11 takes the regime into critical infrastructure.
Three questions for every government
- Which form fits the jurisdiction's scale and research base, and what does the choice cost in independence or pace?
- What triggers an evaluation, what is published, and who decides?
- Where do talent, access and funding come from in years one to three, before fees or obligations can carry any of it?
Selected public sources
- AI Security Institute (formerly AI Safety Institute), United Kingdom, renamed 2025
- Designation of the Digital Trust Centre as Singapore's AI safety institute, IMDA, 2024
- International network of AI safety institutes, inaugural convening 2024
- AI Risk Management Framework and Generative AI Profile, NIST, 2023 and 2024
- Regulation (EU) 2024/1689 (AI Act), European Union, 2024
- ISO/IEC 42001, AI management systems, ISO/IEC, 2023