An artificial intelligence (AI) system may be assessed on how well it prioritises insurance claims for review, then given a different task: recommending whether to accept or refuse them. Evidence that it can prioritise claims reliably does not establish whether its recommendations to accept or refuse claims are sound.

Mark Leiser, Associate Professor of Digital Regulation and Disruptive Technology at Latvia’s Riga Graduate School of Law, used the example at the World AI Regulation Summit, hosted by TAB Global in London on 6 November 2026, to illustrate why assessments need to reflect the task entrusted to AI.

The panel “Trust, Verify, Comply: What AI Assurance Really Demands” brought Leiser together with Ani Gevorkian, Responsible AI Public Policy at Microsoft; Ansgar Koene, Global AI Ethics and Regulatory Leader at EY; and Bojana Bellamy, President of the Centre for Information Policy Leadership. They were joined by Chung Sang Hao, Deputy Director of AI Governance and Safety at Singapore’s Infocomm Media Development Authority (IMDA).

The discussion highlighted a gap between confidence in a model’s capabilities and evidence that it can be used reliably in practice. Closing that gap requires testing how AI performs its assigned tasks, giving assessors access to relevant information and reviewing earlier findings when the system or its use changes.

Governance credentials do not establish application performance

Gevorkian distinguished organisational audits from technical evaluations in frontier model development. An organisational audit examines whether a company has a governance framework and follows it, while a technical evaluation assesses the model’s capabilities and the effectiveness of its safeguards.

Koene warned that this distinction can be lost when findings are interpreted. “There is a huge challenge and a huge risk that AI assessments and audits get misinterpreted as having tested something that they did not test,” he said. An audit confirming that a company follows its governance procedures could, for example, be mistaken for confirmation that its AI system performs reliably.

Koene noted that evaluations by model providers do not necessarily cover how an organisation uses its own data or adapts the model to a particular sector. The deploying organisation still needs to examine how failures will be detected and remedied in its operations.

Asked whether extra testing could slow adoption, Koene said boards had “an obligation to understand” the risks of using AI in their own operations, even when the model had performed well in testing.

Assurance requirements need a defined purpose

Gevorkian urged policymakers to define what an assessment must demonstrate before making it mandatory: whether a developer follows its governance commitments, a model’s safeguards work or an application performs reliably in its intended setting.

“It’s really important as we think about building regulation to be crisp about the type of assurance that we will intend to mandate in given circumstances,” she said.

Speaking from Microsoft’s perspective on frontier model development, she cautioned against concentrating requirements on testing models before deployment. Once incorporated into an agent, a model can gain access to tools and infrastructure under permissions that materially change its capabilities and risks.

“The value of assurance can’t be limited to a particular point in time or a particular actor on the value chain,” she said.

Credible assessors need relevant expertise and access

Leiser identified a problem for organisations hiring external AI assessors: they can struggle to judge the quality of the assessment they are commissioning. Buyers lack a common basis for comparing providers, while companies being assessed may be reluctant to disclose information needed for a thorough examination.

Chung’s preceding presentation described how Singapore is developing testing methods and helping buyers identify competent providers.

“AI assurance requires more than good frameworks,” he said, pointing to the need for practical methodologies, capable providers and mechanisms for establishing testing quality.

Singapore’s Global AI Assurance Sandbox connects organisations deploying AI with testing firms to identify relevant risks, design and conduct tests and interpret results. Working with real applications allows providers to refine their methods, while published case studies share methodologies and lessons without disclosing detailed findings that could expose commercially sensitive information.

Those engagements also inform the AI Verify Foundation’s AI Tester Accreditation Programme (AI TAP). Chung explained that experience with testing firms helps identify the technical competencies and operational capabilities needed to deliver credible assessments.

Accreditation examines whether providers can perform the specific services they claim to offer, alongside their track record, operational readiness and financial capacity to sustain testing. Recognition applies to a defined scope: competence to assess certain risks or types of system does not establish that a firm can assess every AI risk.

During the panel, Chung described this as “a learning exercise” for deployers, testing firms and accreditors alike. He said accreditation had not yet extended to agentic AI testing because the issues remained fluid, including how to assess actions outside an agent’s permissions or instructions.

Koene emphasised that credible assessment also requires independence and an investigative mindset. Reports must make clear what was examined, allowing buyers to understand the limits of the findings rather than infer assurance about risks or performance that were never tested.

Tools and permissions change the risks being tested

Gevorkian explained that giving a model access to tools and infrastructure under particular permissions changes what the resulting agent can do. “The risk surface and the nature of the risks change materially,” she said, distinguishing evaluation of the underlying model from evaluation of the agent and its deployment.

Chung described five baseline generative AI risks: hallucinations, undesirable content, data leakage, bias in decision-making and vulnerability to adversarial prompting. Applications also face risks specific to their sector and use, while agents require examination of behaviour: whether they act outside permissions or authorisation, or take actions that were neither intended nor instructed.

Leiser proposed examining the objectives agents are designed to pursue, including when several agents operate together. “We audit the optimisation strategy,” he said, explaining that assessors should compare those objectives with the system’s outputs and look for unintended consequences.

Gevorkian cautioned that reliable tests and specialist expertise are still developing for risks including loss of control, harmful manipulation, and chemical, biological, radiological and nuclear threats. She suggested that AI safety institutes specialise in different areas to strengthen assessment methods.

Supplier changes can leave earlier evidence behind

Even when an organisation keeps an application’s task unchanged, a supplier can alter the model, prompts or underlying data while it remains in use. Leiser noted that the buyer can have the least access to the evidence needed to understand those changes.

“The evidence expires before the contract does,” he said.

An update need not invalidate earlier findings, but it can change the conditions under which the organisation approved the system. Leiser proposed assigning responsibility for comparing errors, complaints and human overrides with original test results, looking for problems that aggregate performance figures can conceal.

Observed mistakes should inform the review schedule and subsequent checks, while changes should trigger further scrutiny. He also urged organisations to agree when operation should be paused, including when a system that passed testing begins acting outside the parameters examined.

Accountability includes preparedness for failure

An organisation can follow its governance procedures and still be unprepared when AI causes harm. Bellamy treated incident response, complaints and redress as part of accountability, alongside the controls, training and testing intended to prevent failures.

She urged companies to assess harm to others alongside their own exposure to non-compliance. “It’s the risk to society, consumers, service providers,” she said. Responsibilities differ across developers, deployers, integrators and users, while the harms examined depend on the application. A service for children, for example, needs to consider their best interests.

Bellamy pointed to AI tabletop exercises, borrowing from cybersecurity practice, as a way for boards and senior managers to work through a failure scenario and examine how their response arrangements would function.

Leiser also pressed for clear authority to halt operation. “And when everything goes wrong, who can switch it off?”

For organisations putting AI to work, assurance must support the responsibilities given to the system. That means testing its actual use, understanding the limits of assessors’ findings and reviewing changes after deployment—with clear authority to intervene when performance falls short.