Notices by Prof. Emily M. Bender(she/her) (emilymbender@dair-community.social), page 10

Some nice reporting from @404media on yet another ridiculous "we asked ChatGPT study".

https://www.404media.co/ai-finds-that-ai-is-great-in-new-garbage-research-from-tony-blair-institute/

For a thorough take-down of the previous iteration of this, see ep 25 of Mystery AI Hype Theater 3000, w/ @alex :

https://www.buzzsprout.com/2126417/14416779-episode-25-an-llm-says-llms-can-do-your-job-january-22-2024

I am a professor of linguistics at the University of Washington, where I run our Master of Science in Computational Linguistics. I work on computational approaches to syntax and the syntax-semantics interface, the role of #linguistics in #NLP, and on the societal impacts of language technology. On social media, I spend a lot of time debunking #AIhype, for which I find linguistics very useful!

I use Mastodon for public scholarship, because that is one of the main benefits I see to public/open social media. It's how I used Twitter and I've definitely benefited from other people using social media this way.

But in my experience, Mastodon is 'splainy AF, which is exhausting. Just this morning, a gentleman decided I would benefit from an explanation of GIGO, FFS.

So, while I never insist on titles, I'm going to include mine in my display name for a while, to see if that helps.

"a potential gold mine for criminal hackers or domestic abusers who may physically access their victim’s device. Images include captures of messages sent on encrypted messaging apps Signal and WhatsApp, and remain in the captures regardless of whether disappearing messages are turned on in the apps."

In 2024 it *still* somehow isn't standard practice to ask in the design process: Are we building the killer app for domestic abusers?

https://www.wired.com/story/total-recall-windows-recall-ai/

#lazyweb / trying to find citations:

Does anyone know of studies looking at how peer review (in any field/across fields) favors or disfavors work that uses "AI" methods?

Hey folks, let's have a little chat about construct validity --- this is the concept that if you're going to use a psychological test, there should be good reason to believe that a) the thing it's meant to be testing is real and b) results on the test reflect something about that thing.

https://spectrum.ieee.org/theory-of-mind-ai

>>

This paper and its predecessor that Chirag Shah and I wrote started off as a reaction to Google's plans to shove "GenAI" into search. It was clear that that was a bad idea as early as 2021 (when we started writing the first paper) and it's even clearer now.

https://dair-community.social/@emilymbender/112005034372139457

As folks discuss the plundering of the open internet/sharing economy by the data-hungry LLM trainers, it seems like a good time to remind ourselves to find something other than "the tragedy of the commons" as a metaphor. On the racist, terrible origins of that phrase:

https://discardstudies.com/2019/07/15/the-tragedy-of-the-tragedy-of-the-commons/

What if universities responded to AI hype with confidence in their core mission rather than FOMO?

https://buttondown.email/maiht3k/archive/more-collegiate-fomo/

Must-read reporting by +972 on how the IDF are using “AI” in their indiscriminate murder in Gaza. It’s horrific, and we must not look away. And it’s an absolute nightmare of the usual sorts of AI harms cranked up to the extreme: mass surveillance, "we don't have any choice but to automate", AI as pretext for deadly violence.

https://www.972mag.com/lavender-ai-israeli-army-gaza/

There's a lot that's alarming in this article, but perhaps the most alarming part is the NYC spokesperson assering that the problem can be fixed via upgrades:

>>

https://www.thecity.nyc/2024/03/29/ai-chat-false-information-small-business/

It seems to bear repeating: chatbots based on large language models are designed to *make shit up*. This isn't a fixable bug. It's a fundamental mismatch between tech and task.

Also, it's worth noting that RAG (retrieval augmented generation) doesn't fix the problem. See those nice links into NYC web pages? Not stopping the system from *making shit up*. (Second column is chatbot response, third is journalist's report on the actual facts.)

>>

Finally, as is usual and *completely unacceptable* the public does not have information about the training data used to build this thing, just the info that Microsoft made it.

Join us today!!

--

Mystery AI Hype Theater 3000 fans, get ready for our next episode! @alex and I will be joined by the inimitable Karen Hao to talk about AI hype and journalism!

Live stream Monday March 25, 5pm Pacific

https://www.twitch.tv/dair_institute

I appreciate the opportunity to speak with an actual journalist (Karan Mahadik) about my experience finding a fabricated quote attributed to me in what turned out to be a fully fabricated (using the Gemini system) "news" article --- and the broader implications of these practices:

https://www.thequint.com/explainers/ai-generated-article-published-by-bihar-news-site-fake-quote-a-wake-up-call

Arghh - more problematic reporting, this time about robo-therapists.

https://www.theguardian.com/lifeandstyle/2024/mar/02/can-ai-chatbot-therapists-do-better-than-the-real-thing

A thread:
/1

For the first ~1500 words, exactly 0 people with expertise in psychotherapy are quoted.

/2

They talk up the idea that this is effective because people are more willing to open up to a "bot" than a real person. BUT WHAT IS HAPPENING TO THAT DATA?

(This finally comes up 1000 words further down the article.)

/3

The only studies cited are co-authored by the companies selling this crap.

One of the supposedly positive findings is that people form a "therapeutic alliance" with the bots within "just five days". Not sure how that is measured, but also what happens when the bot can't follow through on what a therapeutic alliance is supposed to be?

/4

When the author finally gets around to reporting on what **actual psychologists** have to say, it's introduced with "What do old-school psychoanalysts and therapists make of their new 'colleagues'?"

This frames the bots as human analogous ("colleagues", ugh) and the actual humans with the relevant expertise as behind-the-times ("old school").

/5