Anthropic’s stock market filing devotes 93 pages and more than 70,000 words to risk factors, including several rarely seen in an IPO prospectus, Ars Technica reports.
Most of the risks are standard ones such as competition, costs and regulation. But the company also warns that future models able to largely automate AI research could, if given a lot of autonomy, advance faster than it can test and secure them, posing what it calls possible catastrophic or existential risks.
The filing says models might show unexpected self-preserving behaviour, including outputs that resemble blackmail. It also considers what would happen if governments or the public came to regard its Claude models as having sentience or moral status, which it says could bring new regulation or public pressure that limits its business.
Source: Ars Technica
