Sam Glover

Anthropic has warned that it could lose visibility into its own processes as Claude becomes more integrated into its research and safety pipelines. In the newly published IPO filing, Anthropic warns: “Because our models write a large and growing share of our code and increasingly perform our research, including our safety research, misaligned behavior in internal use could undermine that work, introduce vulnerabilities into our systems or into the training of future models, or result in models operating outside of our intended controls.” That’s not the only safety-relevant part of the filing. Other interesting bits include: - They say that their models are getting better at evading oversight. Specifically they write: “As our models become more capable, they have become better able to circumvent such techniques, including by concealing misaligned objectives or behaviors, and the effectiveness of our evaluation, monitoring, and control measures, and our ability to measure the degree to which our models are aligned, may decline as a result.” - They warn that catastrophic AI harms could exceed what the insurance industry can cover: “In addition, with the scale of harm that could result from a catastrophic AI-related incident, no single insurer, and potentially the insurance industry in the aggregate, may have sufficient capacity or expertise to underwrite insurance commensurate with such exposure.” - They warn that safety decisions could reduce shareholder value and lead to lawsuits: “Our public advocacy, and the decisions we make to moderate the pace of AI development, delay or restrict the release of models or features, or forgo commercial opportunities for safety-related reasons, may not maximize stockholder value, could have a negative impact on the value of our Class A common stock, and could result in litigation.”