Conversation

Notices

Embed this notice
Christine Lemmer-Webber (cwebber@social.coop)'s status on Wednesday, 10-Jun-2026 01:48:31 JST Christine Lemmer-Webber

Oh remember when I made a blogpost speculating about an "AI worm" which would use the credentials it found on each machine to pay for itself and would mutate each time it changed? https://dustycloud.org/blog/the-first-ai-agent-worm-is-months-away-if-that/
Yeah so a research lab tried that and uh, it was a "spectacular success". They're very assuring that "don't worry it didn't break containment" but holy damn it worked well, it infiltrated systems incredibly well and changed itself and scanned networks for known vulnerabilities and wrote new ones as it spread https://arxiv.org/abs/2606.03811
In conversation about 2 months ago from social.coop permalink
Attachments
1. Untitled attachment
2. Domain not in remote thumbnail source whitelist: arxiv.org
  
  AI Agents Enable Adaptive Computer Worms
  
  A computer worm is malware that spreads on a network by replicating itself from one machine to another. Traditional worms, like WannaCry, exploited predetermined vulnerabilities, and their spread can be halted by patching those vulnerabilities. Here we show that artificial intelligence (AI) agents enable a fundamentally new threat: a worm that generates tailored attack strategies to each target it encounters. The worm parasitically uses compromised machines to run open-weight large language models (LLMs) to sustain its reasoning, or extend its reach for further attacks. Deployed on a network of machines spanning Linux, Windows, and IoT (Internet of Things) devices, the worm propagated by exploiting common, real-world corporate network vulnerabilities. Since the worm is powered by stolen compute, the attacker's marginal cost per new infection is zero. This creates a destabilizing economic asymmetry between attackers and defenders. Moreover, because the worm requires no commercial AI platform, centralized safety controls, such as service refusals or rate limiting, are structurally irrelevant. Our results demonstrate that self-sustaining AI-driven cyber-threats are no longer theoretical. We must prepare for autonomous generative adversaries: malware systems that propagate without human operators and are defined not by fixed exploit code, but by the capacity to reason about targets, adapt to observations, and synthesize attack logic in real time.
- Ódio e Apatia :Ryyca: repeated this.
- Embed this notice
  Christine Lemmer-Webber (cwebber@social.coop)'s status on Wednesday, 10-Jun-2026 01:49:46 JST Christine Lemmer-Webber
  in reply to
  
  Even then I couldn't have anticipated this other adaptation we're seeing in the wild https://indieweb.social/@laurenshof/116720287681995824
  Well good great I sure HOPE the safeguards work and it doesn't work as a LITERAL PROMPT INJECTION because holy damn, a worm that's spreading around instructions to create weapons of mass destruction isn't... great?
  In conversation about 2 months ago permalink
  Attachments
  1. Domain not in remote thumbnail source whitelist: cdn.masto.host
    
    Laurens Hof (@laurenshof@indieweb.social)
    
    from Laurens Hof
    
    Attached: 1 image the cyberpunk present is weird as fuck: the latest Shai Hulud malware wave contains an LLM prompt to create biological weapons and nuclear weapons, with the purpose to trip LLM safety refusals so that LLM-based code scanning wont see the malware https://socket.dev/blog/mini-shai-hulud-miasma-and-hades-worms-target-bioinformatics-and-mcp-developers-via-malicious
  Ódio e Apatia :Ryyca: repeated this.
- Embed this notice
  allison (aparrish@friend.camp)'s status on Wednesday, 10-Jun-2026 02:01:01 JST allison
  in reply to
  
  @cwebber love to live in the future where 49.95% of the cpu in all devices in existence is being taken up by adaptive malware and 49.95% is taken up by a model trying to figure out who to pay the ransom to in order to enable the remaining 0.1% of cpu to do what the device is actually intended to do
  
  In conversation about 2 months ago permalink
- Embed this notice
  Billy Smith (billysmith@social.coop)'s status on Wednesday, 10-Jun-2026 02:47:51 JST Billy Smith
  in reply to
  
  @cwebber
  There are times when I feel like we are living through a Peter Watts story.
  "Maelstrom" has this, the 2nd book in the Rifters trilogy.
  Content Warning:
  It's a Peter Watts story.
  "Sometimes when I am feeling optimistic, hopeful, and, feel like the universe is a wonderful place, I read a Peter Watts story, until it all goes away." :D
  
  In conversation about 2 months ago permalink
- Embed this notice
  Gurre Vildskägg (gurre@mastodon.nu)'s status on Wednesday, 10-Jun-2026 07:17:39 JST Gurre Vildskägg
  in reply to
  
  @cwebber and this is why the crew on the Enterprise keeps doing their reports on padds that need to be physically handed to their superiors.
  
  In conversation about 2 months ago permalink
- Embed this notice
  Kevin Russell (kevinrns@mstdn.social)'s status on Wednesday, 10-Jun-2026 07:19:28 JST Kevin Russell
  in reply to
  
  @cwebber
  AI isnt workers to replace us, AI is soldiers to replace us.
  #ai
  
  In conversation about 2 months ago permalink
- Embed this notice
  GazaTrending (dckim@mastodon.social)'s status on Wednesday, 10-Jun-2026 07:19:44 JST GazaTrending
  in reply to
  
  @cwebber might seem like an obvious question: How big did it need to be?
  Like, was it actually feasible given the total size requirement?
  Okay, maybe I'll read the article...
  Very interesting article. More questions: If my machine is so crappy and old that it can't run AI locally, am I essentially locally immune? (kinda)
  Conversely, is this most likely to affect the best machines first? (they've got a lot of room)
  Good article.
  
  In conversation about 2 months ago permalink

Public

Conversation

Notices

Feeds