Notices by Gary McGraw (cigitalgem@sigmoid.social)

Autonomy has its...um...benefits? #MLsec

This is the beginning of the beginning.

https://www.csoonline.com/article/4203630/microsoft-confirms-an-ai-worm-is-propagating-through-copilot-and-other-ms-apps.html

@dalias got it. I also believe that OpenAI was negligent at worst and irresponsible at best.

My view on the marketing front is deeply impacted by Anthropic's high bullshit...

https://berryvilleiml.com/2024/02/08/absolute-nonsense-from-anthropic-sleeper-agents/

https://berryvilleiml.com/2025/11/14/houston-we-have-a-problem-anthropic-rides-an-artificial-wave/

https://berryvilleiml.com/2026/06/14/anthropic-versus-us-administration-fable-mythos-and-the-cyber-cyber/

@dalias you are correct about hugging face but I disagree about your cynical take. This is not an anthropic marketing move in my view.

Instead it strikes me that openAI doesn't really understand what autonomy means.

Is this hugging face hack by OpenAI Agentic bots who escaped the lab important?

https://www.nytimes.com/2026/07/21/technology/openai-attack-hugging-face.html

Yes, especially in light of this 
https://berryvilleiml.com/2026/06/05/biml-and-the-papernot-worm/

It is both inevitable and very concerning.  Autonomy cuts both ways.  As the arsenal of sneaky tricks gets bigger, we can expect more interesting exploit chains.

#MLsec #AI

This kind of #MLsec issue will definitely get the attention of #AI vendors and enterprise users. Getting a chatbot off the rails is easy and fun and...profitable!

https://www.cio.com/article/4155404/ai-token-freeloaders-are-coming-for-your-customer-support-chatbot.html

A good posting reality check on Anthropic's mythos hyperbole around #swsec #appsec

#MLsec adjacent

https://blog.vidocsecurity.com/blog/we-reproduced-anthropics-mythos-findings-with-public-models

Anthropic is overstating the use of #AI by "nation state actors" while providing scant hard evidence. Automation of attacks in security is not new. The real technical question to ask is which parts of these attacks could ONLY be accomplished with AI. The answer to that question seems to be "none of it."

The confluence of cyber cyber and #ML is interesting indeed and hard even for deeply technical people who are firmly grounded in one of the two camps (security engineering or #ML ).(1/2) #MLsec

Don't take a job at any company that uses this kind of AI.

https://www.theregister.com/2025/09/06/ai_job_interview_experience/

Oh hey look...even more obvious data feudalism

https://www.searchenginejournal.com/perplexity-says-cloudflare-is-blocking-legitimate-ai-assistants/552927/

Why so much prompt injection in AI? 1. We don't follow the security engineering design principle "economy of mechanism," and 2, input to LLMs mixes control and data with impunity. We know better. #MLsec #infosec #security

https://www.darkreading.com/vulnerabilities-threats/llms-on-rails-design-engineering-challenges

The world at large does not understand how important #MLsec is. It is just as essential to get this right as it is to fight authoritarianism. #AI #ML #security

https://www.theverge.com/news/624485/turing-award-andrew-barto-richard-sutton-ai-dangers

@mattblaze I mean you can post about using your block button!

@mattblaze You may not believe it, but mine works too and I like to use it for no reason sometimes just to shake things up.

@dibi58 @paezha @bmacDonald94 marketing people and middle management...

We all knew that insecure code was bad, but this is a riot!

Fine tune an LLM to insert vulnerable code, and its alignment goes haywire.

#swsec #appsec #MLsec #ML #AI #security

https://arstechnica.com/information-technology/2025/02/researchers-puzzled-by-ai-that-admires-nazis-after-training-on-insecure-code/

"No matter how many times Stalin told his scientists to plant wheat in the snow so that it could evolve to grow in the winter, the wheat (which had no political allegiances) died."

https://www.theatlantic.com/ideas/archive/2025/02/career-civil-servant-end/681712/

The only way ML models should be called "open source" is if entire training data sets and evaluation sets are public. AI/ML code is not at all interesting. It's the data, stupid.

#MLsec

https://www.infoworld.com/article/3630275/the-future-of-open-source-will-be-messy.html

@elias_sorensen @dalias yep. Ridiculous claims. Often in cases like these, the SOTA benchmark is in the damn training set!

@elias_sorensen I had some interesting talks with the synthetic data guys in the fall. They were delusional and did not listen to reason.

Synthetic data are not the solution.

"Once the training began, researchers discovered a problem in the data: It wasn’t as diversified as they had thought, potentially limiting how much Orion would learn. "

This is beyond obvious from a statistical perspective.

https://www.wsj.com/tech/ai/openai-gpt5-orion-delays-639e7693?st=ng5hBi&reflink=desktopwebshare_permalink