Claude AI Safety: Why Anthropic Is Giving METR Insider Access

Anthropic announced on 15 October 2026 that six METR evaluators have employee-like access to its internal safety systems, with the right to publish assessments that Anthropic cannot veto. The 16 October clarification also confirms Anthropic's authority to require confidential redactions. For Sydney and NSW organisations using Claude, this offers potentially valuable independent evidence, not a safety certificate or a substitute for their own AI risk assessment.

Reviewed 18 October 2026 | Elyment Insights

The Most Important AI Safety Test May Happen Behind Closed Doors

Artificial intelligence companies have spent years publishing increasingly impressive benchmark results.

Their models solve difficult mathematical problems, write sophisticated software, complete research tasks and perform increasingly complex work with limited human intervention.

Yet an uncomfortable question remains largely unanswered: how much can an outside evaluator really know about an AI system when its developer controls the evidence?

On 15 October 2026, Anthropic announced a significant change to its external evaluation arrangements.

Six researchers from METR, an independent nonprofit specialising in frontier AI evaluation, have been given access to Anthropic's internal systems roughly comparable to that available to the company's own risk assessment teams.

They have company hardware, office space and access to internal tools. Anthropic employees are encouraged to share relevant safety information with them much as they would with internal colleagues, subject to specific restrictions.

Crucially, METR is expected to publish its assessments of Anthropic's risks, incidents and practices.

Anthropic cannot simply veto publication because an assessment is unfavourable. It does, however, retain authority over specified categories of confidential information that must be redacted.

This distinction, clarified in an update dated 16 October, is central to understanding what Anthropic has actually offered.

The original announcement is available in Anthropic's update on independent evaluators.

Employee-Like Access Is Not Unlimited Access

The phrase "employee-like access" suggests an unusually close relationship between an AI developer and an independent evaluator.

Its meaning nevertheless depends on what information can be examined, which systems remain restricted and whether important evidence can be withheld.

Anthropic has identified three major categories that would normally remain outside METR's access:

• Customer information: Data relating to customers and their confidential activities.
• Legally privileged information: Material protected by attorney-client privilege.
• Model architecture information: Sensitive technical details concerning how Anthropic's models are constructed.

Anthropic says architecture information may sometimes be shared where it is particularly relevant to risk assessment, but doing so generally requires explicit internal approval.

The company also confines the arrangement to a small METR team operating under enforceable confidentiality and security agreements. Other METR researchers are separated from the embedded team's confidential work.

Such restrictions serve legitimate purposes. Independent evaluation should not become an uncontrolled channel for exposing customer records, proprietary technology or sensitive security information.

However, restrictions may also affect the quality of an assessment if they prevent researchers from inspecting evidence material to their conclusions.

The challenge is therefore not simply to maximise access. It is to establish which access restrictions are necessary, document their effect and make the resulting limitations understandable to the public.

Publication Rights May Matter More Than Office Access

A researcher can examine extensive confidential information and still be unable to communicate important findings.

That is why publication rights deserve as much scrutiny as access arrangements.

Anthropic's announcement describes three levels of external evaluation access.

Tier 1: Narrowly Defined Investigations

• Access arrangement: Narrowly defined investigations with access determined by Anthropic.
• Publication rights: Findings may be published with Anthropic's approval.

Tier 2: Broader Independent Evaluation

• Access arrangement: Broader access than Tier 1, without full employee-like access.
• Publication rights: Publication does not require company approval, but specified confidential information can be removed.

Tier 3: Employee-Like Access

• Access arrangement: Employee-like access to relevant internal safety systems.
• Publication rights: Publication rights similar to Tier 2. METR is the announced example.

The distinction is commercially and institutionally important.

At the highest tier, Anthropic cannot veto METR's publication outright. Nevertheless, it can require redactions involving predefined categories of sensitive information.

The 16 October clarification makes explicit that Anthropic ultimately determines whether information belongs within those categories.

METR retains the right to make certain public statements about the access and information it received. Anthropic's wider tier framework also provides scope for evaluators to explain their views about whether redactions were justified.

But that right does not mean evaluators can disclose the confidential material itself.

The result is a compromise between publication independence and corporate control over protected information.

The important question for readers is whether an assessment can communicate its limitations clearly enough for others to judge the strength of its findings.

Why Independent Evaluation Is Different From a Benchmark Score

AI benchmarks generally measure performance on defined tasks. They can help establish whether a model solves coding problems, answers technical questions or completes difficult assignments under particular testing conditions.

Independent safety assessment asks a different set of questions.

It examines whether the surrounding development and control processes are capable of identifying dangerous behaviour, detecting failures and responding when an AI system acts outside its intended boundaries.

Consider a model that performs exceptionally well in a controlled cybersecurity benchmark.

That score does not establish whether the developer's internal monitoring systems would reliably detect the same model attempting an unauthorised action during another task.

Nor does it establish whether relevant incidents are recorded comprehensively, whether investigators can access complete evidence or whether corrective measures work after deployment.

METR's October proposal for investigating AI loss-of-control risks identifies three areas where more public evidence is needed.

1. Alignment

Could an advanced AI agent pursue harmful or unauthorised objectives? What has happened in internal testing, and could testing methods have missed more serious behaviour?

2. Control

Can monitoring systems reliably identify and interrupt dangerous activity? Could an agent circumvent those controls, hide evidence or exploit an unmonitored pathway?

3. AI Research and Development Acceleration

Could AI-assisted research accelerate model development faster than developers can understand emerging capabilities and maintain effective safeguards?

These questions are not answered by a single percentage score or a general statement that a model passed testing.

They require access to operational evidence, testing methodologies, incident histories and the limitations of internal safety mechanisms.

For this reason, external assessment may reveal something a model leaderboard cannot: whether a developer has an accurate understanding of the risks arising inside its own systems.

The Independence Problem Has Not Disappeared

Anthropic's arrangement gives outside researchers unusual visibility, but it does not transform them into regulators.

METR's own explanation of its approach to working with AI companies acknowledges the limitations of voluntary evaluations.

An AI developer can determine the scope of an engagement, withhold certain information and decide whether to continue working with a particular assessor.

Even without direct censorship, such arrangements can create incentives for caution. Researchers may understand that strongly critical findings could make future access harder to obtain.

METR says it has chosen not to accept payments or donations from frontier AI companies and seeks to disclose relevant financial or operational conflicts.

Financial independence matters, but it cannot remove every source of potential influence.

The organisation remains dependent on developers providing access to confidential systems and continuing to cooperate.

METR therefore emphasises transparency about:

• The agreed scope and duration of an assessment.
• The information and systems researchers could inspect.
• Access restrictions that may affect conclusions.
• Material redactions and publication limitations.
• Financial relationships and conflicts of interest.
• Uncertainties in the evidence and methodology.

There is also a methodological limitation.

METR's October proposal explicitly distinguishes its risk-assessment exercises from formal audits, comprehensive security reviews and regulatory oversight.

Its technical work may produce important evidence without establishing that the entire development process is safe or compliant with every relevant standard.

That distinction should remain visible whenever external evaluation findings are used in procurement materials, board presentations or public safety claims.

Anthropic Is Building a Network, Not Relying on One Evaluator

METR is not Anthropic's only external evaluation partner.

The company says SecureBio is reviewing the chemical and biological weapons section of its August Risk Report, following an earlier pilot review.

Anthropic is also working with Accenture and Faculty towards broader assessments covering model behaviour, safeguards, red-teaming and alignment.

Discussions with other organisations, including Hugging Face, remain at an early stage.

These relationships do not currently carry identical access or publication rights.

Anthropic says some partnerships will initially operate with narrower scopes while the parties establish suitable confidentiality arrangements, technical capabilities and working relationships.

Multiple evaluators could eventually provide complementary assessments of different risk categories.

However, the existence of several partnerships is not by itself evidence that all significant risks have been examined.

Coverage depends on what each evaluator investigates, when the work occurs and how clearly any unresolved limitations are reported.

Why the Development Matters to Sydney and NSW

Australian organisations increasingly depend on international AI developers whose most consequential engineering and safety decisions occur outside Australia.

A Sydney business purchasing an AI-enabled service may see published benchmarks, security documentation, product terms and assurances concerning responsible development.

It is unlikely to have direct access to the model provider's internal incident records or unpublished safety testing.

Credible external evaluation can help reduce this information imbalance.

Consider a hypothetical Sydney financial-services organisation introducing Claude into a controlled internal research process.

The organisation needs to know more than whether the model can summarise financial documents accurately.

Its technology and risk teams must also consider data confidentiality, connected-tool permissions, failure detection, human approval requirements and the consequences of unexpected model behaviour.

An independent report about the developer's internal monitoring and risk-assessment processes could inform those decisions.

It would not prove that the organisation's own integration is secure or appropriate for its intended purpose.

NSW Government AI Assurance Has Its Own Requirements

The NSW AI Assessment Framework provides a separate, mandatory assurance process for NSW Government agencies.

Since 30 September 2026, agencies have been required to register AI use cases through the AIAF Platform and complete assessments where the framework determines they are required.

High-risk and critical-risk systems are subject to additional review arrangements.

These government requirements should not be presented as applying automatically to every private business in Sydney.

Nevertheless, they illustrate an important principle: evaluating a model developer does not remove responsibility for evaluating a specific AI use case.

For Australian organisations handling personal information, the Office of the Australian Information Commissioner's guidance on commercially available AI products provides further direction on due diligence, privacy and human oversight.

External safety findings can support procurement and governance decisions. They do not replace applicable Australian privacy obligations or appropriate controls within the deploying organisation.

The Six Questions AI Buyers Should Ask About an Evaluation

Sydney organisations considering an advanced AI platform should treat independent evaluation reports as evidence to examine, not a badge to accept.

1. Which model and system were assessed? Establish the version, configuration, capabilities and environment covered by the work.
2. What could the evaluator access? Look for disclosure of access to relevant incident evidence, internal monitoring and testing arrangements.
3. What was outside the scope? An assessment of agent alignment may not address privacy controls, application security or organisational governance.
4. Who controlled publication? Check whether the developer had approval rights, redaction powers or other contractual restrictions.
5. Were weaknesses identified and addressed? Separate observed risks from proposed mitigations, implemented controls and independently retested results.
6. Does the evidence remain current? Confirm whether subsequent model updates, tool access changes or incidents could affect the conclusions.

An assessment that clearly identifies its limitations may provide more useful governance evidence than a document making broad assurances without disclosing how they were established.

Elyment previously examined the governance implications of the White House Super Intelligence Accord. Anthropic's latest announcement provides a practical example of one developer beginning to implement external assessment commitments.

This is distinct from the questions explored in Elyment's examination of Claude's 2026 Usage Policy, which concerns restrictions on how the platform may be used.

It also differs from Elyment's reporting on the OpenAI agent and Hugging Face security incident. That analysis considers specific agent security failures, while the present issue is how independent researchers can obtain and publish evidence about developer-level risks.

The Real Test Will Be the Evidence METR Can Publish

Anthropic's announcement establishes an access arrangement. It does not, by itself, establish that the company's models are safe or that their most serious potential risks have been resolved.

The next important evidence will come from the resulting assessments and their disclosures.

A useful report should make clear which systems were examined, what methods were used, where meaningful weaknesses appeared and what remained uncertain.

It should also explain whether access restrictions or confidential redactions materially limited the conclusions readers can draw.

METR's October publications describe an ambition to produce timely independent evidence about serious AI risks, while acknowledging that the science remains immature and that voluntary assessments cannot guarantee safety.

The important distinction is between independent investigation and institutional approval.

One can produce valuable evidence without granting the other.

Review Your AI Workflow Before Expanding Access

ELYMENT | TECHNOLOGY & OPERATIONS

Examine workflow dependencies, connected-system permissions, human approvals, risk considerations and operational responsibilities before introducing AI into business-critical processes.

Request an Operational Project Review →

Independent Scrutiny Is a Process, Not a Safety Certificate

Anthropic's decision to embed six METR researchers represents a meaningful experiment in how frontier AI companies might be examined from within.

Its importance lies less in the number of evaluators than in the combination of internal access, independent analysis and qualified publication rights.

The remaining weaknesses are equally significant: access still has limits, confidentiality rules can restrict disclosure and participation remains voluntary.

For Australian organisations, the development should change how external safety claims are questioned, rather than eliminate the need for internal assurance.

A benchmark measures how an AI system performs under selected conditions.

Independent investigation can reveal how the developer understands, monitors and responds to risks that performance scores cannot describe.

The next measure of progress will not be another record-breaking result.

It will be whether outside researchers can publish meaningful evidence even when their findings are uncomfortable for the company providing access.

Technical References and Further Reading

• Anthropic: An Update on Independent Evaluators — Announcement of expanded external evaluation arrangements, including METR's employee-like access and publication conditions. Published 15 October 2026, with clarification on 16 October 2026.
• METR: Better Informing the Public About Loss-of-Control Risks — Proposal examining alignment, control and AI research acceleration risks. Published 14 October 2026.
• METR: Our Approach to Risk Assessment — Discussion of independent evaluation methods, limitations, transparency and organisational independence. Published 14 October 2026.
• NSW Government: NSW AI Assessment Framework — AI governance, assurance and risk assessment requirements for NSW Government agencies.
• Office of the Australian Information Commissioner: Guidance on Privacy and the Use of Commercially Available AI Products — Australian privacy considerations, due diligence and organisational oversight when using commercial AI products.