Hackers have been hijacking Instagram accounts at scale by exploiting Meta's AI support chatbot. And, as if that weren't bad enough, the technique required no technical skill whatsoever.
When we think about accounts being taken over, we often imagine something of at least some level of sophistication: a credential stuffing attack, a phishing campaign, a SIM-swap, or call center workers being bribed.
However, the breach of several high-profile Instagram accounts involved none of those things. Attackers simply asked an AI chatbot to hand over access to accounts that didn't belong to them - and it did.
Last weekend, reports emerged of a wave of hijacked Instagram accounts. Amongst the victims were beauty retailer Sephora (whose account was defaced with nude photos) and a dormant Obama White House account (which claimed to now be under Shiite control).
Ordinary users with much-coveted short usernames were also targeted, with their handles quickly advertised for resale in Telegram groups.
What made the attack so significant, however, was its simplicity.
The attackers (it almost feels wrong to describe them as hackers, as the breach appears to have required so little technical knowledge) exploited a weakness in Meta's AI-powered support chatbot.
The chatbot, which had been introduced in March to resolve user issues "from start to finish" included the ability to "reset your password securely".
This feature, it transpired, could be manipulated into linking a targeted account to any email address under the control of the attacker.
So how did it work? The attacker connected to Instagram via a VPN set to the targeted account's country. They would then initiate a password reset and ask the AI support bot to switch the recovery email address. Astonishingly, the chatbot would send an eight-digit verification code to the attacker's email address rather than the legitimate owner's address.
Once the "verification" code was received, a simple password reset gave the attacker full control over the targeted account.
According to a report by 404 Media, the vulnerability has been present since at least the end of March, which means it may have existed almost from the moment Meta's AI support feature launched.
On Monday, Meta spokesperson Andy Stone posted that "the issue that did happen has already been fixed." However, by the following day new victims were still coming forward, and discussions on the Telegram channel suggested the technique was still working.
Instagram has since reached out to affected users warning them of "suspicious activity" and confirming that steps have been taken to secure their accounts. Meta has not said how many accounts were impacted.
There is increasing concern in the security community about the security risks involved in deploying AI agents without proper guardrails and governance. The simple truth is that a human support operator is unlikely to have made such a basic error as that made by Meta's AI support agent.
A human support operator would have hesitated. They would have known not to send a verification code to the attacker, but instead send it to the email address already on record for the account.
The AI didn't hesitate. It just did what it was told. Because that is what it was built to do.
As more and more firms rush into integrating AI there is a danger that they will not do in a secure fashion. In April it was revealed that Meta was cutting around 10% of its human workforce (roughly 8,000 employees) in order to lean more heavily into AI.
Editor’s Note: The opinions expressed in this and other guest author articles are solely those of the contributor and do not necessarily reflect those of Fortra.
Cybercrime Intelligence Shouldn't Be Siloed
Fortra® experts are dedicated to protecting organizations and the public by delivering the latest insights, data, and defenses to strengthen security against emerging cyber threats.
