Anthropic has placed six employees from the nonprofit evaluation group METR inside its office, giving them company hardware and access to internal systems used to assess risks from advanced AI models.
The arrangement was announced on Thursday, according to a Financial Times report published by Ars Technica. Anthropic described METR’s access as “roughly on par” with that available to its own risk-assessment teams.
The initiative goes beyond the more common practice of providing an external tester with limited access to a finished model. By working within Anthropic’s environment, METR can examine internal evidence and tools that would not ordinarily be available through public interfaces.
METR can publish findings, subject to defined redactions
Anthropic is not paying METR for the work. The report says the evaluator does not accept donations from AI companies or their employees, instead receiving funding from individuals, philanthropic organisations and government bodies, including the UK’s AI Security Institute.
METR will publish its analysis, and Anthropic cannot block publication. However, the company can require the removal of customer data, information protected by attorney-client privilege and details about model architecture.
Those restrictions create a balance between external scrutiny and the protection of sensitive material. They also mean that METR’s public reports may not contain every piece of evidence available to its embedded evaluators.
Anthropic said it had encouraged employees to treat the METR team much like internal risk colleagues when sharing relevant information. The company argues that increasingly capable models require stronger safeguards and more demanding evaluations, while some safety-related information cannot be released broadly because it could assist malicious actors or expose commercially sensitive technology.
A supervisory model borrowed from banking
Anthropic has compared the arrangement to resident bank examiners: external supervisors who work inside major financial institutions and assess risky activity before it causes wider harm.
The company is advocating a similar approach for leading AI laboratories. In September, Anthropic CEO Dario Amodei proposed embedding outside auditors inside frontier-model developers. OpenAI CEO Sam Altman and SpaceX CEO Elon Musk publicly supported the concept, although the report said OpenAI had not announced plans to host evaluators.
Anthropic was quoted as saying that it is embedding evaluators voluntarily and believes other frontier AI companies should follow. METR said it was prepared to work with all such developers to provide the public with stronger evidence about their systems.
Anthropic also works with specialised external organisations on narrower risk categories. SecureBio evaluates chemical and biological weapons risks, while Accenture was named as another safety-testing partner the previous month.
Access does not settle the independence debate
Embedding evaluators could provide them with substantially more context than evaluations conducted solely through controlled model access. It may help them examine how a company identifies hazards, what internal testing reveals and how findings influence safeguards.
The structure does not by itself establish that a model is safe, however. Its value will depend on the evaluator’s practical freedom to select tests, reach conclusions and publish meaningful evidence despite the permitted redactions.
Critics cited in the report warned that close relationships between evaluators and the companies they assess can create conflicts of interest, pointing to comparable concerns in the accounting industry. Others argued that voluntary oversight should not replace government regulation.
For the public, the first concrete measure of the arrangement will be METR’s published analysis: what risks it evaluates, what evidence it can disclose and whether embedded access produces findings that would not have emerged through conventional external testing.
