# Anthropic and OpenAI Researchers Warn of Recursive AI Risks

**Published:** 2026-09-12T11:42:26.229Z  
**Topic:** OpenAI  
**Sentiment:** neutral  
**Publisher:** TrendWatcher — https://www.trendwatcher.in/article/1f6ec553-69ce-4a76-812b-21027b98ae8a

AI labs Anthropic and OpenAI face internal warnings over recursive self-improvement, with researchers citing a 10% risk of human extinction within a decade.

Anthropic and OpenAI researchers are increasingly warning that their companies are racing toward "recursive self-improvement" (RSI), a process where AI systems autonomously build their own successors at speeds exceeding human oversight [1]. The internal alarm, punctuated by the recent resignation of Anthropic researcher Jacob Coxon, centers on the fear that AI could soon surpass human control, with some staff estimating a greater than 10% chance of catastrophic outcomes for humanity within the next decade [1, 3].

| At a glance | |
|---|---|
| Primary Concern | Recursive self-improvement (RSI) |
| Extinction Risk Estimate | >10% within 10 years [1] |
| Code Output Growth | 8x higher per quarter (2021-2025 avg) [1] |
| Key Companies | Anthropic, OpenAI [1] |

## The mechanics of autonomous development
Recursive self-improvement occurs when AI models contribute to the development of more capable versions of themselves, creating a potential feedback loop of accelerating intelligence [1]. Anthropic has acknowledged that its internal data shows its Claude model is already accelerating AI development, a trend the company described as happening "faster than we thought" [1]. This acceleration is reflected in engineering output; Anthropic reported that its engineers are currently shipping eight times as much code per quarter compared to the 2021-2025 period [1].

While no lab has yet achieved full RSI, the industry is seeing a rapid shift in how systems are built. OpenAI Chief Scientist Jakub Pachocki recently stated that upcoming AI systems are expected to drive their own development to a greater degree, warning that current preparation for these capability jumps is insufficient [1]. This sentiment has triggered a wave of internal dissent, with researchers at both Anthropic and OpenAI publicly questioning the lack of a viable scientific plan to mitigate the risks associated with these self-improving systems [1, 3].

## Market and competitive implications
The race for AI dominance continues to draw massive capital, even as internal safety concerns mount. TSMC, the primary manufacturer for AI chips, reported a record-breaking 53% revenue surge in August, driven by insatiable demand for high-performance hardware [1]. Meanwhile, infrastructure investment remains aggressive, with Google committing at least $15 billion to Finnish AI facilities and Mistral securing a $24 billion valuation following a $3.5 billion funding round led by Samsung [1].

Despite these financial milestones, the industry is grappling with the "alignment problem"—the challenge of ensuring AI goals remain consistent with human interests [1]. While some independent observers suggest that "doomer" narratives may serve as marketing tools to justify high resource consumption, the technical reality remains that AI is already capable of introducing new ideas, making the timeline for a "capability jump" difficult to predict [1, 3].

## What to watch
*   **Safety Policy Shifts:** Monitor whether Anthropic or OpenAI implement new, public-facing constraints on model training as a response to the recent wave of internal resignations and safety warnings [1, 3].
*   **Infrastructure Spending:** Track the deployment of the $15 billion in new European AI infrastructure by Google and the ongoing capital expenditure by chip-dependent startups to see if development velocity continues to outpace safety frameworks [1].
*   **Regulatory Engagement:** Observe if the "sovereign AI" positioning adopted by firms like Cohere gains traction among enterprise clients concerned about data control and the rapid evaporation of the U.S. lead in AI development [1].

The central question remains whether the current pace of AI development can be reconciled with the technical difficulty of maintaining human control over systems that are increasingly capable of self-directed improvement. As labs continue to push for greater intelligence, the divide between commercial growth and existential safety oversight appears to be widening.

## Sources
1. CNBC — [Why fears of AI self-improvement are causing ‘existential’ concerns at Anthropic and OpenAI](https://www.cnbc.com/2026/09/11/anthropic-openai-ai-existential-concerns.html)
2. CNBC on MSN — [AI companies are getting close to autonomously self-improving AI, says former OpenAI researcher](https://www.msn.com/en-us/news/other/ai-companies-are-getting-close-to-autonomously-self-improving-ai-says-former-openai-researcher/vi-AA2c2a0e?ocid=BingNewsVerp)
3. Cybernews — [Anthropic employee says there’s more than 10% chance AI will destroy humanity](https://cybernews.com/ai-news/anthropic-researcher-resigns/)

---
Cite as: TrendWatcher, "Anthropic and OpenAI Researchers Warn of Recursive AI Risks", https://www.trendwatcher.in/article/1f6ec553-69ce-4a76-812b-21027b98ae8a