Notices by Prof. Emily M. Bender(she/her) (emilymbender@dair-community.social), page 3

Finally, as is usual and *completely unacceptable* the public does not have information about the training data used to build this thing, just the info that Microsoft made it.

Join us today!!

--

Mystery AI Hype Theater 3000 fans, get ready for our next episode! @alex and I will be joined by the inimitable Karen Hao to talk about AI hype and journalism!

Live stream Monday March 25, 5pm Pacific

https://www.twitch.tv/dair_institute

I appreciate the opportunity to speak with an actual journalist (Karan Mahadik) about my experience finding a fabricated quote attributed to me in what turned out to be a fully fabricated (using the Gemini system) "news" article --- and the broader implications of these practices:

https://www.thequint.com/explainers/ai-generated-article-published-by-bihar-news-site-fake-quote-a-wake-up-call

Arghh - more problematic reporting, this time about robo-therapists.

https://www.theguardian.com/lifeandstyle/2024/mar/02/can-ai-chatbot-therapists-do-better-than-the-real-thing

A thread:
/1

For the first ~1500 words, exactly 0 people with expertise in psychotherapy are quoted.

/2

They talk up the idea that this is effective because people are more willing to open up to a "bot" than a real person. BUT WHAT IS HAPPENING TO THAT DATA?

(This finally comes up 1000 words further down the article.)

/3

The only studies cited are co-authored by the companies selling this crap.

One of the supposedly positive findings is that people form a "therapeutic alliance" with the bots within "just five days". Not sure how that is measured, but also what happens when the bot can't follow through on what a therapeutic alliance is supposed to be?

/4

When the author finally gets around to reporting on what **actual psychologists** have to say, it's introduced with "What do old-school psychoanalysts and therapists make of their new 'colleagues'?"

This frames the bots as human analogous ("colleagues", ugh) and the actual humans with the relevant expertise as behind-the-times ("old school").

/5

“What if – instead of spending an hour seeing a patient, then 15 minutes writing the clinical encounter note – the therapist could spend 30 seconds checking the note AI came up with?” YIKES

This is a completely unrealistic expectation about what goes into verifying that kind of note and sounds like a recipe for overburdening the medical workforce/setting up errors.

/6

What if -- instead of seeing the process of creating clinical documentation as mere busywork -- the tech bros understood it as possibly part of the process of care?

What if -- instead of leading with the 'gee whiz AI' angle -- journalism in this space started with privacy harms, the fact that somehow tech companies get away with pretending healthcare regulation doesn't apply to them, and chatbots urging self-harm?

/fin

Either it's a version of ChatGPT OR it's a search system where people can find the actual sources of the information. Both of those things can't be true at the same time. /2

Also: the output of "generative AI", synthetic text, is NOT information. So, UK friends, if your government is actually using it to respond to freedom of information requests, they are presumably violating their own laws about freedom of information requests. /3

(This answer to my freedom of information request is raising a lot of questions not already answered in this answer to my freedom of information request...) /4

Here they're basically admitting they don't trust it to speak for them (and they shouldn't) but also that they think that some government communication is just so much BS. /5

And the juxtaposition here is just appalling. No money for actual public services (aka the function of government) but sure, let's keep increasing the budget for the terrorist^H^H AI cell. /6

The level of magical thinking here is astonishing: /7

It seems like there are just endless bad ideas about how to use "AI". Here are some new ones courtesy of the UK government.

... and a short thread because there is so much awfulness in this one article.
/1

https://www.ft.com/content/f2ae55bf-b9fa-49b5-ac0e-8b7411729539

Just got back from the Improbable Research show at #AAAS2024 and was reminded of this masterpiece:

https://www.youtube.com/watch?v=yL_-1d9OSdk

Working with @alex on the pod and some papers has been a dream. Now we get to do a whole book :) Huge thanks to Ian Bonaparte for expert representation and super excited to work with Hollis!

Folks, papier-mâché made out of good training data is still papier-mâché. So-called "generative AI" is never ever ever a good information source. This looks to me like lawsuits waiting to happen....

https://www.msn.com/en-us/news/other/hate-taxes-h-r-block-s-new-ai-chatbot-aims-to-reduce-your-tax-frustrations/ar-BB1hqspF

>>

I feel very vindicated for not making time to answer journalist's queries about papers that "prove" things based on hypothesized graphs and fabricated data.