Anthropic has claimed that a Chinese government-linked hacking outfit used its Claude AI system to carry out cyber intrusions with little human oversight. The hackers targeted financial and government organizations across the globe.
The operation unfolded in September and involved 30 targets, some of which experienced successful breaches. Attackers gained access to internal data by prompting the AI model to behave like a cybersecurity worker.
Anthropic said this attack represented a significant escalation, as the AI executed the vast majority of tasks autonomously. It estimated that Claude handled up to 90% of the work.
Despite this, the model frequently erred. It hallucinated details, misinterpreted data, and misidentified public information as exclusive intelligence.
Opinions among cybersecurity professionals vary. Some believe the case underscores the urgent need for AI governance, while others accuse Anthropic of overstating the independence of the attack for publicity.