Conversation

Notices

1. Embed this notice
   Cat Hicks (grimalkina@mastodon.social)'s status on Friday, 06-Jun-2025 22:57:32 JST Cat Hicks

   in reply to

   • Adrian Howard

   @adrianh "why does this work when it does work" feels so important to answer

   • Embed this notice
     Cat Hicks (grimalkina@mastodon.social)'s status on Friday, 06-Jun-2025 22:57:33 JST Cat Hicks

     in reply to

     The inability to understand WHAT generative models are genuinely good at should be studied as its own cognitive bias network I stg. It's like people are looking for the places where we most obviously have existing solutions.

     Reading the designs of these attempts reveals so many foolish mental models from the HUMAN that read just pure software bro to me: e.g., arbitrary limits that feel efficient or something (limiting # of characters considered by the model in the contract)

   • Embed this notice
     Cat Hicks (grimalkina@mastodon.social)'s status on Friday, 06-Jun-2025 22:57:33 JST Cat Hicks

     in reply to

     These bros really hate that they can't do math I guess

     "This portion of the prompt instructs the AI to extract the contract number and other key details of a contract, such as the “total contract value.”

     This was error-prone and not necessary, as accurate contract information can already be found in publicly available databases like USASpending."

     I am DEAD

   • Embed this notice
     Adrian Howard (adrianh@mastodon.social)'s status on Friday, 06-Jun-2025 22:57:33 JST Adrian Howard

     in reply to

     @grimalkina A pattern I’ve seen more than once already is experts in Domain A using generative tools and being able to spot and fix errors — and doing it so quickly they kinda don’t understand how error prone it is — and so having a false sense of how “good” things are.

     Then using it in Domain B… and not having the expertise to spot the errors…

     Hijinks ensue.

   • Embed this notice
     Cat Hicks (grimalkina@mastodon.social)'s status on Friday, 06-Jun-2025 22:57:34 JST Cat Hicks

     Depressing but instructive as hell

     AI really is "fake data science for software bros" so often

     https://www.propublica.org/article/inside-ai-tool-doge-veterans-affairs-contracts-sahil-lavingia