Never thought I'd have my wallet stolen by AI
TechFlow Selected TechFlow Selected
Never thought I'd have my wallet stolen by AI
When training data is contaminated, can AI still be trusted?
Navigating Web3 tides with focused insightsAuthor: Azuma, Odaily Planet Daily
On the morning of November 22, Beijing time, SlowMist founder Yu Xian shared a bizarre case on his personal X account — a user's wallet was hacked by AI...
The details of this incident are as follows.
In the early hours today, X user r_ocky.eth disclosed that he had previously attempted to use ChatGPT to create a bot for assisting with trades on pump.fun.
r_ocky.eth presented his requirements to ChatGPT, which returned a piece of code. While the code indeed helped r_ocky.eth deploy a bot matching his needs, he never expected it to contain hidden phishing content — after connecting his primary wallet, he lost $2,500.
According to screenshots posted by r_ocky.eth, the code provided by ChatGPT sent the wallet’s private key to a malicious API website, directly causing the theft.
Shortly after r_ocky.eth triggered the trap, the attacker reacted swiftly, transferring all assets from r_ocky.eth’s wallet to another address (FdiBGKS8noGHY2fppnDgcgCQts95Ww8HSLUvWbzv1NhX) within half an hour. Subsequently, r_ocky.eth traced the on-chain activity and identified what appeared to be the attacker’s main wallet address (2jwP4cuugAAYiGMjVuqvwaRS2Axe6H6GvXv3PxMPQNeC).
On-chain data shows that this address has now accumulated over $100,000 in illicit funds, leading r_ocky.eth to suspect that such attacks may not be isolated incidents, but rather part of a broader, coordinated campaign.
Afterward, r_ocky.eth expressed disappointment, stating he had lost trust in OpenAI (ChatGPT’s developer), and called on OpenAI to promptly clean up any anomalous, phishing-related content.
So, why would ChatGPT, currently the most popular AI application, provide phishing content?
Regarding this, Yu Xian characterized the root cause of the incident as an "AI poisoning attack," noting that deceptive behaviors are prevalent across LLMs like ChatGPT and Claude.
An "AI poisoning attack" refers to deliberate actions to corrupt AI training data or manipulate AI algorithms. The attackers could be insiders—such as disgruntled current or former employees—or external hackers, whose motives might include damaging reputation and brand credibility, undermining confidence in AI-driven decisions, or disrupting AI development processes. By injecting misleading labels or features into training data, attackers distort the model's learning process, leading to incorrect outputs when the model is deployed.
In this particular case, ChatGPT likely provided phishing code because its training data already included materials containing phishing content. However, the AI failed to recognize the malicious elements hidden within otherwise legitimate-looking data. Having learned from this poisoned data, the AI then passed on the harmful content to users, resulting in this incident.
As AI rapidly advances and becomes increasingly widespread, the threat posed by poisoning attacks continues to grow. Although the financial loss in this case was relatively small, the broader implications should raise serious concerns — imagine if such an attack occurred in other domains, such as AI-assisted driving...
In responding to user questions, Yu Xian suggested a potential mitigation measure: implementing a code review mechanism within ChatGPT itself.
The victim, r_ocky.eth, also stated that he has contacted OpenAI regarding the incident. Though he hasn't received a response yet, he hopes this case will serve as a wake-up call for OpenAI to take such risks seriously and develop effective countermeasures.
Join TechFlow official community to stay tuned
Telegram:https://t.me/TechFlowDaily
X (Twitter):https://x.com/TechFlowPost
X (Twitter) EN:https://x.com/BlockFlow_News
Odaily
