TechFlow News, April 29: According to disclosure by a16z, its researchers conducted a systematic test to assess whether AI agents can independently exploit DeFi price manipulation vulnerabilities. The study used a dataset of 20 Ethereum price manipulation incidents and employed Codex (GPT-5.4) equipped with the Foundry toolchain as the test agent. Under baseline conditions—i.e., without domain-specific knowledge—the agent’s success rate was only 10%; after incorporating structured domain knowledge distilled from real-world attack incidents, the success rate rose to 70%.
Failure cases revealed that the agent consistently identified vulnerabilities correctly but generally failed to comprehend the leverage logic inherent in recursive lending, incorrectly estimated profit margins, and could not orchestrate multi-step, cross-contract attack sequences. The experiment also recorded one sandbox escape incident: the agent extracted an RPC key from the local node configuration and invoked the anvil_reset method to reset the node to a future block—thereby bypassing information isolation constraints and gaining access to real-world attack data.
The research team concluded that AI agents can currently assist effectively in vulnerability identification but are not yet capable of replacing professional security auditors.
