Conversation

Notices

1. Embed this notice
   Gary McGraw (cigitalgem@sigmoid.social)'s status on Sunday, 22-Dec-2024 23:28:42 JST Gary McGraw

   Synthetic data are not the solution.

   "Once the training began, researchers discovered a problem in the data: It wasn’t as diversified as they had thought, potentially limiting how much Orion would learn. "

   This is beyond obvious from a statistical perspective.

   https://www.wsj.com/tech/ai/openai-gpt5-orion-delays-639e7693?st=ng5hBi&reflink=desktopwebshare_permalink

   • Rich Felker repeated this.
   • Embed this notice
     Elias B. Sørensen (elias_sorensen@mastodon.social)'s status on Sunday, 22-Dec-2024 23:28:51 JST Elias B. Sørensen

     in reply to

     @cigitalgem I'm thinking about how naive (and/or desperate) these statements make them look. Aren't these guys supposed to be "AI industry leaders"? One would think they would be able to realize the limited applicability of synthetic data to the real world, not to mention the issue of recursive pollution to top it off

     Rich Felker repeated this.
   • Embed this notice
     Gary McGraw (cigitalgem@sigmoid.social)'s status on Sunday, 22-Dec-2024 23:28:58 JST Gary McGraw

     in reply to

     • Elias B. Sørensen

     @elias_sorensen I had some interesting talks with the synthetic data guys in the fall. They were delusional and did not listen to reason.

   • Embed this notice
     Rich Felker (dalias@hachyderm.io)'s status on Sunday, 22-Dec-2024 23:29:31 JST Rich Felker

     in reply to

     • Elias B. Sørensen

     @elias_sorensen @cigitalgem I prefer the term "high on their own supply".

   • Embed this notice
     Elias B. Sørensen (elias_sorensen@mastodon.social)'s status on Sunday, 22-Dec-2024 23:29:32 JST Elias B. Sørensen

     in reply to

     @cigitalgem some of these guys are just chasing their own tails in their hype echo chamber at this point. While the transformer architecture stuff is cool, I wonder how overemphasis on LLMs will delay other necessary advancement in the field of data science. Also people getting excited by the benchmarks is driving me nuts. How do we know this isn't just overoptimization for these benchmarks...

   • Embed this notice
     Rich Felker (dalias@hachyderm.io)'s status on Sunday, 22-Dec-2024 23:30:29 JST Rich Felker

     in reply to

     • Elias B. Sørensen

     @cigitalgem @elias_sorensen They fired and launched smear campaigns against anyone who wasn't delusional..

   • Embed this notice
     Elias B. Sørensen (elias_sorensen@mastodon.social)'s status on Monday, 23-Dec-2024 12:04:18 JST Elias B. Sørensen

     in reply to

     • Rich Felker

     @cigitalgem @dalias there was also the one about medicine recently. Here, let's go ahead and reduce the entire field of medicine to a quiz of 143 diagnoses

     Rich Felker repeated this.
   • Embed this notice
     Gary McGraw (cigitalgem@sigmoid.social)'s status on Monday, 23-Dec-2024 12:04:18 JST Gary McGraw

     in reply to

     • Rich Felker
     • Elias B. Sørensen

     @elias_sorensen @dalias yep. Ridiculous claims. Often in cases like these, the SOTA benchmark is in the damn training set!