Anthropic brought independent evaluators inside the company. They are to assess AI model safety

Anthropic announced on 15 October an expansion of its cooperation with independent AI-safety evaluators. Six employees of the nonprofit METR received work equipment, office space and access to internal tools at a level roughly comparable to Anthropic’s own risk-evaluation teams. The aim is to reduce the information imbalance between the model developer and outside experts. The supplier knows development details, internal test results and limitations of safety measures, while the public and customers usually see only published summaries. METR is expected to use the information it receives for independent evaluation of risks and safety practices. Anthropic cannot unilaterally block publication of the conclusions. It can, however, request removal of predefined confidential information such as customer data or privileged legal documents. Evaluators can publicly state whether they consider such restrictions justified. This strengthens their independence, even though access remains constrained by contractual and security requirements. Anthropic compares the approach with banking supervision. Large banks know more about their own risks than the public, so supervisory institutions can use experts working directly inside the organization. Anthropic wants to apply a similar principle voluntarily to advanced AI models. The company sets four main conditions for external evaluation: expertise, independence, sufficient access to information and the ability to publish the evaluator’s own conclusions. Anthropic is also creating three levels of access. At the lowest level, evaluators perform narrowly defined checks and publication of results is subject to approval. Higher levels provide broader access and greater publication independence. The highest level corresponds to the current arrangement with METR. Anthropic also works with SecureBio, which assesses biological and chemical risks, and with Accenture. These relationships currently have different scopes of access and publication rights. The commercial significance lies in the credibility of safety claims. Enterprise customers today often have to rely on the supplier’s own documentation. An independent assessment can provide an additional basis for model selection, risk management and decisions about use in sensitive processes. Such an evaluation is not automatically a legal audit or confirmation of compliance with the European AI Act. The scope of risks assessed, access to information and methodology used must be clear. For regulated industries, it may be useful to include independent safety evaluations among the criteria for purchasing AI. Alongside price and performance, companies should assess the quality of published safety evidence and the supplier’s approach to incident handling. KEY TERMS Independent evaluation: Assessment of a technology by experts who are not part of its regular development team. Model safety measures: Technical and organizational mechanisms that limit dangerous or unauthorized AI behavior. Conflict of interest: A situation in which economic or other ties may affect an evaluator’s independence.

Anthropic brought independent evaluators inside the company. They are to assess AI model safety

The full article is available to users registered at Hard Skills