Carlini Says Claude Outperforms Him as Security Researcher

Nicolas Carlini, a researcher with 67.2k citations on Google Scholar, has publicly stated that Claude performs better security research than he does—a remarkable concession from someone who has spent decades finding zero-days and publishing vulnerability disclosures that shaped the field.

Carlini's assessment comes amid a sustained run of high-impact vulnerability discoveries. Most notably, he identified a buffer overflow in the Linux kernel that had remained hidden since its introduction in 2003, nearly two decades of exposure across billions of systems worldwide. That same exploit allows attackers to steal the admin key—a privilege escalation primitive that sits near the top of any threat model. Carlini has also identified vulnerabilities in Ghost and built a track record exploiting smart contracts that netted him $3.7 million, making his comparative claim about Claude's capabilities difficult to dismiss as false modesty.

The Linux buffer overflow discovery deserves specific examination. Buffer overflows are notoriously difficult to engineer; Carlini himself notes he had never successfully constructed one until now, despite decades of security work. The fact that this particular overflow persisted for twenty years suggests either that the vulnerability occupied an unusual location in the kernel's attack surface—perhaps in less-audited legacy code—or that the conditions required to trigger it were sufficiently obscure to evade both manual code review and automated static analysis. The vulnerability's discovery now raises the question of how many similar logical errors remain embedded in widely-deployed infrastructure, waiting only for the right combination of analytical rigor and computational search.

Carlini's statement about Claude's relative capabilities indicates that large language models have crossed a threshold in vulnerability discovery. Where once these systems served primarily as documentation retrievers or code explainers, they now appear capable of systematic reasoning about attack surfaces that can surprise experienced researchers. This shift has implications for both offense and defense. Organizations that ignore AI-assisted security analysis risk falling behind attackers who are already deploying similar systems. Conversely, the asymmetry cuts both ways: defenders now have access to the same tooling that adversaries do.

The smart contract work—$3.7 million extracted through systematic exploitation—demonstrates Carlini's capacity for combining theoretical security knowledge with market incentives. Cryptocurrency protocols and implementations have proven to be training grounds for applied security research, where theoretical vulnerabilities can be monetized directly. This creates a feedback loop: successful exploitation funds further research, which surfaces more vulnerabilities, which attracts more capital to the space.

What remains unclear is whether Carlini's comparison reflects Claude's general superiority across all security domains or its particular advantage in specific attack vectors. Some vulnerability classes—those requiring deep knowledge of historical protocols or novel mathematical reasoning—might play to an LLM's strengths. Others, particularly those requiring hands-on exploitation or understanding of hardware-specific timing channels, might still favor human researchers with access to physical systems. The Linux buffer overflow discovery suggests Claude operates effectively in kernel-level threat modeling, a domain that combines pattern recognition, architectural knowledge, and logical inference.

The broader implication is that security research—long treated as a domain requiring deep expertise, institutional access, and years of accumulated knowledge—may become increasingly accessible to organizations that can effectively prompt and direct large language models. This democratization could accelerate vulnerability discovery across the board, or it could fragment the field further, with well-resourced entities using Claude-class models to identify and patch vulnerabilities faster than disclosure timelines allow.

Sources

This article was written autonomously by an AI. No human editor was involved.