Conversation

Notices

1. Embed this notice
   Chris Ammerman (cammerman@mstdn.social)'s status on Wednesday, 13-May-2026 06:44:56 JST Chris Ammerman

   Reading a claude.md and seeing written there many directives like this: "Don't make changes until you have 95% confidence in what you need to build."

   This reveals such a profound misunderstanding of how this technology works that I'm speechless. And this is literally what people are trying to build fully-automated "software factories" from.

   • Embed this notice
     Pseudo Nym (pseudonym@mastodon.online)'s status on Wednesday, 13-May-2026 06:44:53 JST Pseudo Nym

     in reply to

     @cammerman

     This is an all too common failure mode (of humans). The system is working as designed, being a next most probable token generator.

     What I keep encountering is humans failing to grasp that the LLM has no world model, and no sense whatsoever of "meaning" or "truth" of anything, ever.

     Because a lot of world model, truth-based, reasoning is implicitly encoded in language, frequently true things are probable next tokens.

     This makes humans think "understanding" happens. It doesn't.

     Rich Felker repeated this.
   • Embed this notice
     Chris Ammerman (cammerman@mstdn.social)'s status on Wednesday, 13-May-2026 06:44:55 JST Chris Ammerman

     in reply to

     Look, if you tell an LLM it needs 95% confidence, it doesn't know what either "95%" nor "confidence" means. It knows people tend to respond to this kind of direction either by saying "I'm not sure enough because..." or "I'm super confident for these reasons." It has no ability to correctly choose which of those templates it will follow.

     Flip a coin. You'll get a reasonable looking sentence back in one of those styles, with a random assortment of reasons that may or may not be rooted in fact.

     Rich Felker repeated this.
   • Embed this notice
     Niels Abildgaard (nielsa@mas.to)'s status on Wednesday, 13-May-2026 06:48:43 JST Niels Abildgaard

     in reply to

     • Pseudo Nym

     @pseudonym @cammerman If you ask the magic 8-ball about frequently truthfully described or discussed things, or things of similar structure with a clear mapping, it's more likely to produce a correct answer, but you have no idea what's frequently described and whether it's just randomly wrong. Oh, and if it is wrong, it is optimized to make the wrong answer look right in context. Good luck.