Conversation

Notices

1. Embed this notice
   katana crimson (katana@mas.to)'s status on Saturday, 06-Jan-2024 02:48:00 JST katana crimson

   • Thomas 🔭🕹️

   @thomasfuchs from the studies that I've been seeing roll out the last little while, it seems more that they're better at enabling Dunning-Kruger effects than anything.

   • Embed this notice
     Steve (kladni@newsie.social)'s status on Saturday, 06-Jan-2024 02:56:33 JST Steve

     • Thomas 🔭🕹️

     @thomasfuchs
     One thing I think about is separating the "knowledge" we expect from an LLM from the "language processing".
     I think the former will do as you say, but the latter can improve.
     So, things like the ability to translate from one language to another, or from speech to text, will get better.

   • Embed this notice
     Steve (kladni@newsie.social)'s status on Saturday, 06-Jan-2024 03:07:09 JST Steve

     • Thomas 🔭🕹️

     @thomasfuchs
     We all agree "garbage in, garbage out" here. I also agree there probably is a turning point in 2023 where the inputs can't be assumed to be human generated. I've been assuming the primary symptom will be content-based.

   • Embed this notice
     Chris Dillon 🌹 (squarism@hachyderm.io)'s status on Saturday, 06-Jan-2024 03:10:43 JST Chris Dillon 🌹

     • Thomas 🔭🕹️

     @thomasfuchs How do we know this? What starts the garbage out feedback loop if we have an evaluation system that works? Do we have an evaluation system that works? Do we have an eval system. What is ML eval.

     Do people know what ML eval is.

   • Embed this notice
     Urzl (gooba42@mastodon.social)'s status on Saturday, 06-Jan-2024 03:49:30 JST Urzl

     • Thomas 🔭🕹️

     @thomasfuchs The architects of these systems no doubt know this about them. I'm curious if there's not an ulterior motive behind the big push to install the LLMs everywhere.

     Is this effectively a second phase of training? We taught it how we write with the initial canon and now we're teaching it to parse natural language engrams through user interaction with that canon.

     It won't get better at generating accurate outputs but it might get better at understanding the prompts.

   • Embed this notice
     Larry O'Brien (lobrien@hachyderm.io)'s status on Saturday, 06-Jan-2024 03:49:38 JST Larry O'Brien

     • Thomas 🔭🕹️
     • Lisa Melton

     @thomasfuchs @lisamelton Citation needed. The differences between GPT generations have been qualitative so far. More blocks and larger context windows mean more abstract features and more state. Don’t know how you conclude that won’t improve quality. The “tech bro” conceit is “scaling is _all_ you need,” following “The Bitter Lesson” argument: meta-methods that can find and capture complexity > methods inserted manually. Don’t know any studies that strongly refute this.

     https://www.cs.utexas.edu/~eunsol/courses/data/bitter_lesson.pdf

   • Embed this notice
     Mathew Attlee (codeinabox@hachyderm.io)'s status on Saturday, 06-Jan-2024 04:45:50 JST Mathew Attlee

     • Thomas 🔭🕹️

     @thomasfuchs it's already been shown they degenerate from consuming their own output, they are much the victim of their own success https://paperswithcode.com/paper/large-language-models-suffer-from-their-own