Conversation

Notices

1. Embed this notice
   Dana Fried (tess@mastodon.social)'s status on Thursday, 28-Nov-2024 03:10:44 JST Dana Fried

   tfw u have been doing computers for a very, very long time

   • Embed this notice
     Eaton (eaton@phire.place)'s status on Thursday, 28-Nov-2024 03:10:44 JST Eaton

     in reply to

     @tess did a big comparison between document categorization approaches with dozens of models and techniques. without doing any particular tailoring or optimization, simple kmeans proximity with embeddings scored within 10% of the best prompt based llm approaches. Orders of magnitude faster and more energy efficient, too. It’s hard to imagine why one wouldn’t start by optimizing that to improve results, rather than endless prompt fiddling.. but admittedly it took relearning some tangly math

   • Embed this notice
     Dana Fried (tess@mastodon.social)'s status on Thursday, 28-Nov-2024 03:20:13 JST Dana Fried

     in reply to

     • Eaton

     @eaton I wonder if you can just throw the LLM at it in the small percentage of cases where the strength/confidence of the output was low? Basically use the cheap classifier as a preprocessing step?

   • Embed this notice
     Eaton (eaton@phire.place)'s status on Thursday, 28-Nov-2024 03:20:13 JST Eaton

     in reply to

     @tess that’s one of the ideas in my to do queue; more NLP preprocssing with explicit high-leverage rules, etc. one use of LLMs so far has been automating the creation of cluster-descriptions when using vector search to unearth patterns in the content, but it still takes some chewing

   • Embed this notice
     Eaton (eaton@phire.place)'s status on Thursday, 28-Nov-2024 03:22:08 JST Eaton

     in reply to

     @tess i will say, the increasing shift in energy towards smaller tuned models for specific tasks feels a lot more encouraging than the race to create the biggest, baddest general purpose “ask me anything” llms