Conversation

Notices

1. Embed this notice
   Jorge Arango (jarango@mastodon.social)'s status on Tuesday, 30-Jul-2024 09:20:29 JST Jorge Arango

   I've started using Llama 3.1 8B with some command line AI workflows. It's pretty good! Slower than GPT 4o and not as 'smart' — but it's *free* and *local*. (I.e., all data stays in my computer.) Amazing. (Also a reminder that commercial model providers don't have much of a moat — unless they build services around these things.)

   • Embed this notice
     Eaton (eaton@phire.place)'s status on Tuesday, 30-Jul-2024 09:20:28 JST Eaton

     in reply to

     @jarango I’ve been pretty impressed with Llama 3.1; obviously nowhere near as polished at stuff like “recognizing and explaining the structure of a JPEG meme” but extremely capable of the fine grain associative/summary/transformer stuff I like to tinker with at scales that make gpt4 cost-prohibitive…

   • Embed this notice
     Jorge Arango (jarango@mastodon.social)'s status on Tuesday, 30-Jul-2024 09:32:33 JST Jorge Arango

     in reply to

     • Eaton

     @eaton that's how I've been using it too. (GPT 4o Mini is also doing great in the 'good enough but cheap’ dept.)

   • Embed this notice
     Eaton (eaton@phire.place)'s status on Tuesday, 30-Jul-2024 09:32:33 JST Eaton

     in reply to

     @jarango I’ll be honest: I genuinely believe that the future for LLM-as-a-service style APIs is very limited once the heat of VC fervor dies down and the costs of training and running the models at scale are passed on to end users.

     That said, for projects at the scale we’re talking about (and the average enterprise web site is well within that range) I think locally hosted models of increasing efficiency will become a standard part of the kit for analysis and experimentation.