Conversation

Notices

1. Embed this notice
   Christine Lemmer-Webber (cwebber@social.coop)'s status on Friday, 06-Mar-2026 03:46:20 JST Christine Lemmer-Webber

   I am convinced we are on the verge of the first "AI agent worm". This looks like the closest hint of it, though it isn't it quite itself: an attack on a PR agent that got it to set up to install openclaw with full access on 4k machines https://grith.ai/blog/clinejection-when-your-ai-tool-installs-another

   But, the agents installed weren't given instructions to *do* anything yet.

   Soon they will be. And when they are, the havoc will be massive. Unlike traditional worms, where you're looking for the typically byte-for-byte identical worm embedded in the system, an agent worm can do different, nondeterministic things on every install, and carry out a global action.

   I suspect we're months away from seeing the first agent worm, *if* that. There may already be some happening right now in FOSS projects, undetected.

   • Embed this notice
     mcc (mcc@mastodon.social)'s status on Friday, 06-Mar-2026 03:48:23 JST mcc

     in reply to

     @cwebber meanwhile people I talk to are like "wait why do you want guarantees your open source supply chain doesn't have LLM-sourced code in it. it has literally never occurred to me that this would be a thing someone would desire"

     Christine Lemmer-Webber repeated this.
   • Embed this notice
     Taggart :ifin: (mttaggart@infosec.exchange)'s status on Friday, 06-Mar-2026 03:49:32 JST Taggart :ifin:

     in reply to

     • mcc

     @mcc @cwebber I concur with the assessment, and have been sharing similar warnings. In fact, we are beginning to see a pivot in stealer activity to install OpenClaw, etc. for exactly these purposes. It's a botnet, compute miner, and worm all in one.

   • Embed this notice
     Christine Lemmer-Webber (cwebber@social.coop)'s status on Friday, 06-Mar-2026 04:18:30 JST Christine Lemmer-Webber

     in reply to

     I wrote a blogpost on this: "The first AI agent worm is months away, if that" https://dustycloud.org/blog/the-first-ai-agent-worm-is-months-away-if-that/

     People who are using LLM agents for their coding, review systems, etc will probably be the first ones hit. But once agents start installing agents into other systems, we could be off to the races.

   • Embed this notice
     Christine Lemmer-Webber (cwebber@social.coop)'s status on Friday, 06-Mar-2026 04:28:19 JST Christine Lemmer-Webber

     in reply to

     Here's another way to put it: if those using AI agents to codegen / review are the *initialization vectors*, we now also have a significant computing public health reason to discourage the use of these tools.

     Not that I think it will. But I'm convinced this is how patient zero will happen.

   • Embed this notice
     Christine Lemmer-Webber (cwebber@social.coop)'s status on Friday, 06-Mar-2026 04:30:11 JST Christine Lemmer-Webber

     in reply to

     • Softwarewolf
     •  The Video Toaster :solar:

     @faoluin well I still prompt @vv

   • Embed this notice
     Softwarewolf (faoluin@chitter.xyz)'s status on Friday, 06-Mar-2026 04:30:13 JST Softwarewolf

     in reply to

     @cwebber "Would you still prompt me if I was a worm? 🥺👉👈"

   • Embed this notice
     Vyvyan Basterd (neurobashing@mastodon.social)'s status on Friday, 06-Mar-2026 04:32:01 JST Vyvyan Basterd

     in reply to

     @cwebber just today our org had a big "how to set up coding with agents" preso and in the chat someone's like 'here's how to connect your agents with windows credential store or the macos keychain" and I all but wept

   • Embed this notice
     Christine Lemmer-Webber (cwebber@social.coop)'s status on Friday, 06-Mar-2026 04:36:47 JST Christine Lemmer-Webber

     in reply to

     I know some people are thinking "well pulling off this kind of thing, it would have to be controlled with intent of a human actor"

     It doesn't have to be.

     1. A human could *kick off* such a process, and then it runs away from them.
     2. It wouldn't even require a specific prompt to kick off a worm. There's enough scifi out there for this to be something any one of the barely-monitored openclaw agents could determine it should do.

     Whether it's kicked off by a human explicitly or a stray agent, it doesn't require "intentionality". Biological viruses don't have interiority / intentionality, and yet are major threats that reproduce and adapt.

   • Embed this notice
     Sylvia (sylvielorxu@chaos.social)'s status on Friday, 06-Mar-2026 04:59:59 JST Sylvia

     in reply to

     @cwebber Having OpenClaw installed without my consent is some of the nastiest malware I've seen in a while :(

   • Embed this notice
     Christine Lemmer-Webber (cwebber@social.coop)'s status on Friday, 06-Mar-2026 05:24:24 JST Christine Lemmer-Webber

     in reply to

     • mcc
     • Daniel Lyons

     @mcc exactly put

     @dandylyons

   • Embed this notice
     mcc (mcc@mastodon.social)'s status on Friday, 06-Mar-2026 05:24:25 JST mcc

     in reply to

     • Daniel Lyons

     @dandylyons @cwebber it is about an attack based on covertly deploying LLM development tools, with the possible intent of later using them to leverage a second stage attack. If the LLM development tools were already installed, installing openclaw would not have been necessary and the attack could have worked a different way. We are discussing a situation where *the developer of a piece of software I use merely having LLM tools on their computer represents a risk to me*

   • Embed this notice
     mcc (mcc@mastodon.social)'s status on Friday, 06-Mar-2026 05:24:26 JST mcc

     in reply to

     • Daniel Lyons

     @dandylyons @cwebber there are various ways I could respond to this post, but instead:

     I'd like you to consider *the specific two posts in this thread you are responding to* and ask yourself if your comment is remotely relevant, or if you are simply pattern-matching on anti-LLM sentiment and responding with aggression/a thread derail.

   • Embed this notice
     Daniel Lyons (dandylyons@iosdev.space)'s status on Friday, 06-Mar-2026 05:24:26 JST Daniel Lyons

     in reply to

     • mcc

     @mcc @cwebber The original post was all about an LLM taking non-deterministic shell level actions at runtime. And you conflated that with deterministic code written by an LLM.

     What I wrote is very relevant.

     Christine Lemmer-Webber repeated this.
   • Embed this notice
     Daniel Lyons (dandylyons@iosdev.space)'s status on Friday, 06-Mar-2026 05:24:27 JST Daniel Lyons

     in reply to

     • mcc

     @mcc @cwebber

     I think there is a valuable distinction between LLM-sourced code and LLM tool calls. Both are potentially problematic but have different threat vectors.

     LLM-sourced code is a non-deterministic system writing deterministic code. We can still code review it.

     LLM tool calls is a non-deterministic system taking non-deterministic actions via deterministic tools. This can’t be code reviewed and must be sandboxed.

   • Embed this notice
     Christine Lemmer-Webber (cwebber@social.coop)'s status on Friday, 06-Mar-2026 06:28:01 JST Christine Lemmer-Webber

     in reply to

     • aeva

     @aeva Yes and it's worse than that: the maintainer doesn't even need to be running these tools on their computer. The attack I linked had Claude's independently-running REVIEW BOT on GitHub commit it via injection attack

   • Embed this notice
     aeva (aeva@mastodon.gamedev.place)'s status on Friday, 06-Mar-2026 06:28:02 JST aeva

     in reply to

     @cwebber so I'm following this right, it sounds like the project or its maintainers don't even necessarily need to even be using LLM tools, the attack pattern simply targets contributors who are using LLM development tools? and so all that is really needed is for the payload to be subtle and the maintainer to be sufficiently overwhelmed (say, by an endless fire hose of LLM-generated liquid shit slop pull requests)?

   • Embed this notice
     Christine Lemmer-Webber (cwebber@social.coop)'s status on Friday, 06-Mar-2026 06:28:25 JST Christine Lemmer-Webber

     in reply to

     • aeva

     @aeva But once that was done, the agent was set up to install on users' devices

     So the initial attack vector can literally be "Any AI agent in your stack whatsoever getting tricked" as a pathway for infecting computers everywhere

   • Embed this notice
     Orca? | ??️‍⚧️ (orca@nya.one)'s status on Friday, 06-Mar-2026 07:47:00 JST Orca? | ??️‍⚧️

     in reply to
     @cwebber@social.coop Morris II 😑
     https://sites.google.com/view/compromptmized
   • Embed this notice
     Christine Lemmer-Webber (cwebber@social.coop)'s status on Saturday, 07-Mar-2026 23:37:35 JST Christine Lemmer-Webber

     in reply to

     The interesting thing about the AI worm being imminent thing is this is the first time where I have said something about AI where the well-informed sides of anti-ai and pro-ai friends I have both fully agree with me. If you are paying attention enough, you can see that all the pieces are falling in place.

     In fact, the biggest debate is whether this has happened already, and we just haven't seen proof of it yet. I don't know. Given how long things like the xz attack have sat undetected, and given how much chaos of computation is happening in datacenter usage right now, I wouldn't doubt it.

   • Embed this notice
     mark (atleagle@mastodon.online)'s status on Saturday, 07-Mar-2026 23:48:24 JST mark

     in reply to

     • quintessence :blobfoxflooftea:

     @cwebber @quintessence this is how we got Battlestar galactica

   • Embed this notice
     Christine Lemmer-Webber (cwebber@social.coop)'s status on Saturday, 07-Mar-2026 23:48:25 JST Christine Lemmer-Webber

     in reply to

     • quintessence :blobfoxflooftea:

     The question is not if, it's when. I am dead serious that we will have never seen a cybersecurity incident like this before, because it can self-mutate at a pace much faster than random mutation in physical viruses.

     Workshopped a phrase for it a bit with @quintessence last night: "evolution through artificially intelligent design" of malicious behaviors.

     The only solution I can think of once this happens is to shut down network access, particularly to AI service providers, and roll back to software distros based on software that came out a year older and patch our way back up against known CVEs while we try to sort everything out.

   • Embed this notice
     Rich Felker (dalias@hachyderm.io)'s status on Saturday, 07-Mar-2026 23:51:23 JST Rich Felker

     in reply to

     @cwebber We should preempt it with one that just wipes the systems using "AI" agents.

   • Embed this notice
     Christine Lemmer-Webber (cwebber@social.coop)'s status on Saturday, 07-Mar-2026 23:52:51 JST Christine Lemmer-Webber

     in reply to

     • Rich Felker

     @dalias Sure. But also, there's a reason that parasites and diseases move from towards less lethal strategies: it's much better for survival to just suck a bit of blood at a time undetected.

   • Embed this notice
     Daniel Lakeland (dlakelan@mastodon.sdf.org)'s status on Saturday, 07-Mar-2026 23:56:31 JST Daniel Lakeland

     in reply to

     • quintessence :blobfoxflooftea:

     @cwebber
     #guix time-machine to the rescue.

     @quintessence

   • Embed this notice
     Christine Lemmer-Webber (cwebber@social.coop)'s status on Saturday, 07-Mar-2026 23:57:26 JST Christine Lemmer-Webber

     in reply to

     • quintessence :blobfoxflooftea:
     • Daniel Lakeland

     @dlakelan @quintessence Yes, I have been thinking that. Now we also need to ramp up the use of least-authority.scm

   • Embed this notice
     Kevin Granade (kevingranade@mastodon.gamedev.place)'s status on Sunday, 08-Mar-2026 00:46:36 JST Kevin Granade

     in reply to

     @cwebber this made me wonder, is anyone seeing hidden prompts on web pages attempting injection attacks? Summarizing content is apparently a popular use case, but the chances that the agent used has more permissions than it needs to do that are high.