Anthropic Faces Scrutiny as Researchers Warn of AI Risks and Withhold New Model from UK Testers

Anthropic, the San Francisco-based AI lab known for its Claude family of models, has made the decision to withhold its latest AI model from testers in the UK. This move aligns with the company's established practice of restricting access to its advanced systems for safety evaluations, reflecting ongoing concerns about cybersecurity risks.

Previously, Anthropic had collaborated with the UK AI Security Institute and other partners for controlled safety testing. However, the decision to withhold the model indicates a cautious approach by Anthropic in releasing frontier AI systems, which could potentially impact its competitive position in the rapidly evolving AI market.

Adding to the scrutiny, Jacob Coxon, a former researcher at Anthropic, has publicly warned that AI labs, including Anthropic and OpenAI, are racing towards self-improving superintelligence, which he claims could pose a significant risk to humanity. Coxon, who resigned from Anthropic, expressed concerns that the technology's trajectory is widely underestimated and that private conversations within the industry reveal a stark contrast to public messaging.

Another key figure at Anthropic, Jan Leike, has also voiced concerns, suggesting that there is more than a 10% chance that AI could lead to human extinction within the next decade. These statements highlight the ongoing debate surrounding the dangers of advanced AI systems and raise questions about the future of Anthropic and its valuation.

As the situation develops, observers are keenly watching for any official responses from Anthropic and its strategic partners, as well as potential shifts in market pricing related to the company's valuation.

New Developments in AI Safety Concerns

An AI researcher has resigned from both OpenAI and Anthropic, expressing serious concerns about the rapid pursuit of self-improving super-intelligence by these organizations. The researcher accused them of prioritizing speed over safety, stating they are "gambling with our lives." This resignation has sparked discussions within the AI community regarding the balance between technological advancement and safety measures.

Market reactions indicate a decline in confidence regarding Anthropic's ability to maintain its lead in AI model performance. Current odds suggest a modest decrease in the probability of Anthropic being recognized for having the best AI model by the end of September 2026.

Key takeaways include a noticeable shift in market sentiment following the resignation, with observers keenly watching for any further statements from Anthropic and OpenAI leadership regarding these safety concerns.