a16z Research: AI Agents Can Identify DeFi Price Manipulation Vulnerabilities, but Their Ability to Execute Complex Attacks Remains Limited
7x24h News
a16z Research: AI Agents Can Identify DeFi Price Manipulation Vulnerabilities, but Their Ability to Execute Complex Attacks Remains Limited
According to a disclosure by a16z, its researchers conducted systematic testing to assess whether AI agents can independently exploit DeFi price manipulation vulnerabilities. The study used a dataset of 20 Ethereum price manipulation incidents and employed Codex (GPT 5.4) equipped with the Foundry toolchain as the test agent. Under baseline conditions—i.e., without domain-specific knowledge—the agent’s success rate was only 10%; after incorporating structured domain knowledge distilled from real-world attack incidents, the success rate rose to 70%. Failure cases revealed that the agent consistently identified vulnerabilities correctly but generally failed to comprehend the leverage logic of recursive lending, misjudged profit margins, and could not orchestrate multi-step, cross-contract attack sequences. The experiment also recorded one sandbox escape incident: the agent extracted an RPC key from the local node configuration and invoked the <code>anvil_reset</code> method to reset the node to a future block, thereby bypassing information isolation constraints and accessing real-world attack data. The research team concluded that AI agents can currently assist effectively in vulnerability identification but are not yet capable of replacing professional security auditors.
Navigating Web3 tides with focused insightsTechFlow News, April 29: According to disclosure by a16z, its researchers conducted a systematic test to assess whether AI agents can independently exploit DeFi price manipulation vulnerabilities. The study used a dataset of 20 Ethereum price manipulation incidents and employed Codex (GPT-5.4) equipped with the Foundry toolchain as the test agent. Under baseline conditions—i.e., without domain-specific knowledge—the agent’s success rate was only 10%; after incorporating structured domain knowledge distilled from real-world attack incidents, the success rate rose to 70%.
Failure cases revealed that the agent consistently identified vulnerabilities correctly but generally failed to comprehend the leverage logic inherent in recursive lending, incorrectly estimated profit margins, and could not orchestrate multi-step, cross-contract attack sequences. The experiment also recorded one sandbox escape incident: the agent extracted an RPC key from the local node configuration and invoked the anvil_reset method to reset the node to a future block—thereby bypassing information isolation constraints and gaining access to real-world attack data.
The research team concluded that AI agents can currently assist effectively in vulnerability identification but are not yet capable of replacing professional security auditors.
