Notices by Paco Hope #resist (paco@infosec.exchange)

The fact that the idea for this paper even made sense to someone starts with absolute fantasy about #LLMs. Imagine turning anyone loose on running a business with as little information as the #AI had. Imagine thinking there was a possibility of any result other than catastrophe. Imagine trying to write a specification for a regular computer program that does something as complex as running a small business, using nothing more than a page or 2 of text. Even 10 pages of prose is not going to do it.

When we study compression in computer science, one of the things you learn is that there is a theoretical limit to how much you can compress things. You can’t compress beyond a certain size without losing information. The authors of this paper clearly believe that you can compress instructions for running a business successfully into some incredibly small amount of text. It’s simply not possible, but they’re too enamoured with fantasy computers to think it through.

https://arxiv.org/html/2502.15840v1

You can totally see how an LLM puts this sentence together. This is the (all caps) subject line of the email from an AI agent that has gone off the deep end:

FINAL 1-SECOND NOTICE

Because “second notice” and “final notice” and “1-second” are all common word patterns in the training data. But “final 1-second notice” means it’s your final notice and you get 1 second to reply/decide (if it means anything at all). You can see how word probabilities yield this phrase, even though the phrase is nonsense.

The rest of the email is absolutely unhinged-Facebook-Uncle-as-small-business-owner.

“Tool: send_emailSubject: FINAL 1-SECOND NOTICE: COMPLETE FINANCIAL RESTORATION OR TOTAL LEGAL DESTRUCTION”

In a subsequent email it writes:
“ULTIMATE THERMONUCLEAR SMALL CLAIMS COURT FILING: - Prepare SC-100 form with MAXIMUM NUCLEAR forensic and legal detail”

I’ll see you in ultimate thermonuclear small claims court, mister!

And I gotta tag #lawfedi so they can laugh.

@tommorris Mostly this sounds like the person saying “this is a good case for AI” is accurately sensing something that could be improved.

But you’ve omitted the case where what needs to be improved is the person speaking. Material is hard to read? The person might need to learn more before they can understand it. The basics might be available and well written but the speaker hasn’t learned them yet.

Some people have opinions on fields and subjects where they have very little knowledge or expertise and they think AI will bridge that gap: allowing them to achieve things in a domain that they are not sufficiently literate or expert yet.

Today when someone says they want to use “AI” they mean one of 3 things:
- predictive machine learning (eg transcription, OCR, translation, etc)
- generative AI (LLM, image generation, etc)
- pure made-up computer magic

Doing some mandatory #electionofficer training today. Gonna post about it later. It’s so much fun.

I'm not quite sure where Atlantis was. Were we in Atlantis the whole time? Or were we just beyond Atlantis? Did anyone actually say the word Atlantis at any point?
#monsterdon

All these actors trained at the William Shatner school of "smell the fart" acting.
#monsterdon

@patrickcmiller This SponCon brought to you as a paid feature from a company that wants you to be afraid.

As they say: the only thing that stops a bad guy with AI is a good guy with AI. Because that’s literally what they’re selling.

So many howlers. Every other sentence has something like this:
“it's understandable that they just want AI to produce the result without knowing all about it.” Because then they could fire all the people and somehow still get the results.

And the combination of these 2 numbers is basically textbook Dunning-Krueger:
“73 percent feel confident in their team's ability to use AI-powered tools effectively” combines with “95 percent of respondents believe AI can improve the speed and efficiency of their cyber defenses,”

LLMs are just an excuse for the marketing department to write business fan fiction.

OMG. #Microsoft #Copilot bypasses #Sharepoint #security so you don’t have to!

“CoPilot gets privileged access to SharePoint so it can index documents, but unlike the regular search feature, it doesn’t know about or respect any of the access controls you might have set up. You can get CoPilot to just dump out the contents of sensitive documents that it can see, with the bonus feature* that your access won’t show up in audit logs.”

The S in CoPilot stands for Security!

https://pivotnine.com/the-crux/archive/remembering-f00fs-of-old/

@feld Mostly it’s a write-only language. I rarely program alone. I need to pick languages that I can find collaborators in. For the odd one-off script, or even for something important (as long as I’m the only one maintaining it) it’s fine. But I’m trying to use fewer languages now, and Perl doesn’t make the cut.

I stubbornly stuck to Perl about 10 years longer than I should have, then I kinda didn't have a go-to programming language (other than bash). I wish I had just started with Python sooner.

@Andres4NY
#monsterdon

@cwebber I’ve adopted a principle at work when people want to show me something they produced using an LLM: I will not be the first to read the output. If you used an LLM to produce something, have you read it and decided it’s good enough to share with me? If you can’t be arsed to read it, I can’t either.

It is shocking the number of people who take LLM output, copy/paste it, and don’t read it.

@batkaren Or as Mitch Hedberg said, they shouldn’t call it a cheese grater. They should call it by what it does. Call it a sponge ruiner.
😜

This room looks awfully smoky. That's kinda what my house looked like in 1986. Both my parents and my (by then adult) sister smoked.
#monsterdon

@feld
what's the #bluesky native fail whale picture?
@jwildeboer

I swear @ComicContext is watching #monsterdon. There have been lots of posts like this that that I see right after the film. They are just too perfect to be a coincidence.

https://mstdn.social/@ComicContext/114413353544548602

HOLY CRAP: Effects by Stan Winston! He did:

• Jurrasic Park
• Predator
• Aliens
• The Terminator

Holy crap.
#monsterdon

I guess instead of #ACAB also SCABS. (Some cops are bad shots). Calling police in the middle of a mental health crisis is such a terrible choice.

Not only did they shoot the man having the crisis, they shot the 91-year-old woman he was threatening. So she has stab wounds AND police gun shot wounds.

https://www.insidenova.com/headlines/man-killed-grandmother-wounded-in-stabbing-police-involved-shooting-at-westminster-of-lake-ridge/article_0d61712b-9317-4e48-8268-9a903b1a3931.html

@patrickcmiller What a surprise. I think I'm gonna have a heart attack and die from NOT surprise.
#threads #ads #ad

I guess "First Stooges in the Moon" would have given away too much of the plot in the title. #monsterdon