Conversation

Notices

1. Embed this notice
   Charlie Stross (cstross@wandering.shop)'s status on Monday, 10-Mar-2025 06:39:28 JST Charlie Stross

   It occurs to me that Ken Thompson's classic lecture "Reflections on Trusting Trust" has a LOT to say about code generation using LLMs.

   The Thompson self-propagating UNIX hack that he described was a lot like a prompt injection attack (the right parsed input could trigger an unexpected output via opaque-to-programmer intermediate steps).

   https://www.cs.cmu.edu/~rdriley/487/papers/Thompson_1984_ReflectionsonTrustingTrust.pdf
   https://mastodon.social/@gmh/114132067525074476

   • Embed this notice
     Charlie Stross (cstross@wandering.shop)'s status on Monday, 10-Mar-2025 06:39:25 JST Charlie Stross

     in reply to

     • Graydon
     • Poul-Henning Kamp

     @graydon @bsdphk I'd be a little less draconian, but make directors of limited companies personally liable for any lawbreaking and impose jail time on them at the highest management level involved in the infraction. Fines don't work on the rich. "If you cheat on taxes via an offshore wheeze, or do a rug-pull on your customers via contractual shenanigans, you WILL go to prison for 11 years" ought to concentrate their minds wonderfully.

   • Embed this notice
     Graydon (graydon@canada.masto.host)'s status on Monday, 10-Mar-2025 06:39:26 JST Graydon

     in reply to

     • Poul-Henning Kamp

     @bsdphk @cstross No feedback involving a tax is useful until the general problem of making corporates pay taxes has been solved.

     My general take on this is that the limited liability corporation is a tool of conquest and should be abolished as part of the alterations to the social machinery acknowledging we live on a finite planet and need to close all the loops to keep the "live" part functional and factual.

     (It's also the case that it's not clear ANY 3ʳᵈ party data retention is legitimate.)

     Rich Felker repeated this.
   • Embed this notice
     Poul-Henning Kamp (bsdphk@fosstodon.org)'s status on Monday, 10-Mar-2025 06:39:27 JST Poul-Henning Kamp

     in reply to

     • Graydon

     @graydon @cstross

     I am increasingly convinced we need a tax on stored data.

     Even a little as 1$/MB/year would seriously change decisions about what is stored and how long it is stored.

     It would also bring into light who holds how much data.

   • Embed this notice
     Graydon (graydon@canada.masto.host)'s status on Monday, 10-Mar-2025 06:39:28 JST Graydon

     in reply to

     @cstross I think that's certainly an LLM hazard.

     The core hazard as I see it is not a single specific "this is a bad idea" but that the LLM push is a continuation of the American imperial value extractor built around IP controls. No amount of negative consequences to other people will cause or permit not using LLMs. They're going to get made compulsory unless or until there's a collapse of the empire or they somehow start having intolerable costs to Apple, Google, and Microsoft.