Conversation

Notices

1. Embed this notice
   Cassandra is only carbon now (xgranade@wandering.shop)'s status on Tuesday, 10-Jun-2025 06:51:07 JST Cassandra is only carbon now

   This isn't really a critique of AI products, per se, but it does make the computer scientist in me slightly sad how much everything has moved over to throwing massive amounts of data at learning problems and being clever about training infrastructure, rather than being clever about defining the problem space in some way that reflects the structure of the learning problem.

   • Rich Felker repeated this.
   • Embed this notice
     Cassandra is only carbon now (xgranade@wandering.shop)'s status on Tuesday, 10-Jun-2025 06:51:44 JST Cassandra is only carbon now

     in reply to

     Like, that the approach to building in memory is to just prepend a transcript of all previous interactions with an "agent." LLMs are fundamentally Markov chains, which seem desperately ill-matched to the problems that people are applying LLMs to.

     Rather than resolving that mismatch, modern AI products overcome it by just throwing alarming amounts of data and processing power at it (and yes, using clever engineering to do so).

     Rich Felker repeated this.
   • Embed this notice
     Cassandra is only carbon now (xgranade@wandering.shop)'s status on Tuesday, 10-Jun-2025 06:51:55 JST Cassandra is only carbon now

     in reply to

     I was thinking about this when thinking about LISP the other day, how the whole idea of the language was trying to represent the solution space for self-modifying code. That ultimately didn't really quite wind up being AI, but we learned a *lot* about computer science as a result.

     You don't see that same kind of surprise come out training mismatched models through immense brute force.

     Rich Felker repeated this.
   • Embed this notice
     Glyph (glyph@mastodon.social)'s status on Tuesday, 10-Jun-2025 06:59:20 JST Glyph

     in reply to

     @xgranade I really feel this. For all their genuinely surprising properties, some of which may even translate to capabilities one day, transformer architectures seem like an evolutionary and intellectual dead end quite unlike any previous development in tech

   • Embed this notice
     Rich Felker (dalias@hachyderm.io)'s status on Tuesday, 10-Jun-2025 07:00:51 JST Rich Felker

     in reply to

     • Glyph

     @glyph I suspect part of the reason they're such a dead end lies in their being an immense cognitive hazard for the people working with them.

   • Embed this notice
     Jay Stephens (jaystephens@mastodon.social)'s status on Tuesday, 10-Jun-2025 08:34:07 JST Jay Stephens

     in reply to

     • Glyph
     • Rich Felker

     @dalias @glyph
     100% this.
     I have very smart friends convinced that by bolting together a few specialist systems plus an LLM trained on larger data sets than now, AGI is inevitable.
     The exposure is poisoning the brains.

   • Embed this notice
     silverwizard (silverwizard@convenient.email)'s status on Tuesday, 10-Jun-2025 21:00:37 JST silverwizard

     in reply to
     @xgranade I keep having conversations along the lines of "why would I want a bunch of data scraped from reddit to do customer service"