Conversation

Notices

1. Embed this notice
   Zack Whittaker (zackwhittaker@mastodon.social)'s status on Saturday, 06-Jun-2026 22:45:55 JST Zack Whittaker

   New, by me: Meta has filed a data breach notice confirming that *thousands* of people had their Instagram accounts hacked as part of a months-long campaign abusing its Meta AI chatbot.

   Meta's breach notice shows the hacks were far more widespread than first thought.

   More: https://this.weekinsecurity.com/meta-confirms-thousands-of-instagram-accounts-were-hacked-by-abusing-its-ai-chatbot/

   Sign up/RSS for my free weekly newsletter: https://this.weekinsecurity.com/

   • James Morris likes this.
   • Rich Felker repeated this.
   • Embed this notice
     JWcph, Radicalized By Decency (jwcph@helvede.net)'s status on Sunday, 07-Jun-2026 03:28:20 JST JWcph, Radicalized By Decency

     in reply to

     @zackwhittaker Correction: It's not a bug & the AI wasn't "tricked". It worked exactly like it was supposed to - Meta just didn't care to consider that anyone could ask it to do so.

     It's a tech bug in the same sense that a person plowing a pickup truck through a kindergarten is the car's fault.

     - which also means the "bug" is "fixed" until somebody stumbles upon the next vulnerability, because the underlying issue, as the book said, is carelessness.

     Rich Felker repeated this.
   • Embed this notice
     Stian (sab@hostux.social)'s status on Sunday, 07-Jun-2026 03:28:25 JST Stian

     in reply to

     • JWcph, Radicalized By Decency

     Meta literally claims that "The tool itself worked properly and functioned as intended". Incredible.

     If your tool allows for people to hack your users by simply asking politely for it, reasonable people might argue that this tool is not, in fact, "working as intended".
     @jwcph @zackwhittaker

   • Embed this notice
     Rich Felker (dalias@hachyderm.io)'s status on Sunday, 07-Jun-2026 03:34:58 JST Rich Felker

     in reply to

     • Aubrey Clark

     @aubreyclark @zackwhittaker In this case, "the AI being able to be tricked into doing things it wasn't supposed to" isn't the problem. The problem is that it was given permission the same wrong permissions that human support staff were wrongly given, to bypass access controls on user accounts.

     If this kind of access exists at all, it should require escalation to approval by multiple parties, long mandatory waiting periods for the account owner to see it's happening if they still have access, and something to impose financial and/or legal risk on the party requesting access if it turns out to be fraudulent. Not something human or slopbot support agent can do unilaterally.

   • Embed this notice
     Aubrey Clark (aubreyclark@mastodon.social)'s status on Sunday, 07-Jun-2026 03:35:00 JST Aubrey Clark

     in reply to

     @zackwhittaker This reminds me of a friend who tests AI systems for a living. Her job is basically to see whether an AI can be tricked into doing things it wasn't supposed to do.

     Is this the same general idea, or is it a completely different kind of vulnerability? 😲

     Ódio e Apatia :Ryyca: repeated this.
   • Embed this notice
     ⏩︎VOTE⏪︎ ᶜʸⁿⁱᶜⁱˢᵐ⁼ᵃᶜᶜᵉᵖᵗᵃⁿᶜᵉ (Ben Royce) (benroyce@mastodon.social)'s status on Tuesday, 09-Jun-2026 00:14:08 JST ⏩︎VOTE⏪︎ ᶜʸⁿⁱᶜⁱˢᵐ⁼ᵃᶜᶜᵉᵖᵗᵃⁿᶜᵉ (Ben Royce)

     in reply to

     • Aubrey Clark

     @aubreyclark @zackwhittaker

     it's really, really stupid

     it's based on "AI is magic, yay! just turn it on, no problems, yay!":

     "hackers abused a flaw in Meta's chatbot that allowed anyone to reset the password of any account that did not have two-factor authentication switched on. The bug tricked the chatbot into sending a verification code to an email address controlled by the hacker, rather than the account holder's email address on file, simply by asking it. The chatbot complied anyway"