Notices by Prof. Emily M. Bender(she/her) (emilymbender@dair-community.social), page 9

Sunday's thread on why chatbots & LLMs are a bad solution for information access, with replies to the most common types of counterarguments I encountered in my mentions.

https://buttondown.com/maiht3k/archive/information-literacy-and-chatbots-as-search/

Sunday's thread on why chatbots & LLMs are a bad solution for information access, with replies to the most common types of counterarguments I encountered in my mentions.

https://buttondown.com/maiht3k/archive/information-literacy-and-chatbots-as-search/

As OpenAI and Meta introduce LLM-driven searchbots, I'd like to once again remind people that neither LLMs nor chatbots are good technology for information access.

A thread, with links:

Chirag Shah and I wrote about this in two academic papers:
2022: https://dl.acm.org/doi/10.1145/3498366.3505816
2024: https://dl.acm.org/doi/10.1145/3649468

We also have an op-ed from Dec 2022:
https://iai.tv/articles/all-knowing-machines-are-a-fantasy-auid-2334

>>

Why are LLMs bad for search? Because LLMs are nothing more than statistical models of the distribution of word forms in text, set up to output plausible-sounding sequences of words.

https://www.youtube.com/watch?v=qpE40jwMilU

>>

If someone uses an LLM as a replacement for search, and the output they get is correct, this is just by chance.

Furthermore, a system that is right 95% of the time is arguably more dangerous tthan one that is right 50% of the time. People will be more likely to trust the output, and likely less able to fact check the 5%.

>>

But even if the chatbots on offer were built around something other than LLMs, something that could reliably get the right answer, they'd still be a terrible technology for information access.

Setting things up so that you get "the answer" to your question cuts off the user's ability to do the sense-making that is critical to information literacy.

>>

That sense-making includes refining the question, understanding how different sources speak to the question, and locating each source within the information landscape.

>>

Imagine putting a medical query into a standard search engine and receiving a list of links including one to a local university medical center, one to WebMD, one to Dr. Oz, and one to an active forum for people with similar medical issues.

If you have the underlying links, you have the opportunity to evaluate the reliability and relevance of the information for your current query --- and also to build up your understanding of those sources over time.

>>

If instead you get an answer from a chatbot, even if it is correct, you lose the opportunity for that growth in information literacy.

The case of the discussion forum has a further twist: Any given piece of information there is probably one you'd want to verify from other sources, but the opportunity to connect with people going through similar medical journeys is priceless.

>>

Finally, the chatbots-as-search paradigm encourages us to just accept answers as given, especially when they are stated in terms that are both friendly and authoritative.

But now more than ever we all need to level-up our information access practices and hold high expectations regarding provenance --- i.e. citing of sources.

The chatbot interface invites you to just sit back and take the prepackaged AI slop as if it were "information". Don't be that guy.

/fin

Mystery AI Hype Theater 3000 Episode 43: AI Companies Gamble with Everyone's Planet

https://www.buzzsprout.com/2126417/episodes/16022676-episode-43-ai-companies-gamble-with-everyone-s-planet-feat-paris-marx

in which @parismarx joins @alex and me for an update on techbro fantasies of AI solving climate change + the current reality of the climate impact of “AI”.

w/thx to Christie Taylor for production!

Also available as video on PeerTube:

https://peertube.dair-institute.org/w/reoPQ5YTzEQbFoaRcNwJ88

I just noticed that an up-coming online talk I'm giving is being advertised with the wrong title -- as in the host "edited" it before sending it out. Like he knows better than me what I'm going to talk about. Who does that??

"It is not the case that “AI gathers data from the Web and learns from it.” The reality is that AI companies gather data and then optimize models to reproduce representations of that data for profit."

"The productivity myth suggests that anything we spend time on is up for automation — that any time we spend can and should be freed up for the sake of having even more time for other activities or pursuits — which can also be automated."

https://www.techpolicy.press/challenging-the-myths-of-generative-ai/

Read the whole thing!

I talk to the press largely to *combat* AI hype. It's beyond frustrating to be misquoted in ways that contribute to it instead.

New newsletter post:

https://buttondown.com/maiht3k/archive/correcting-the-record/

Excellent new essay by Ted Chiang, who gets to the heart of why language and art are inherently about conveying meaning and experience, not just form:

https://www.newyorker.com/culture/the-weekend-essay/why-ai-isnt-going-to-make-art

I've seen a lot of awful and ridiculous AI hype in the past two years, but this weekend there was one that briefly took away my ability to even.

I recovered, and wrote up a newsletter post:

https://buttondown.email/maiht3k/archive/medical-science-shouldnt-platform-automating-end/

It's Agurketid for Mystery AI Hype Theater 3000 -- @alex and I will be taking on a very silly long-form interview with a CEO whose fantasies about AI have really gotten the better of him.

Join us Monday July 29, 1pm Pacific
twitch.tv/dair_institute

Does anyone have info about the context of this slide? I see it credited to an IBM presentation from 1979---but a presentation by whom, to whom about what?

Why can't more journalists reporting on "AI" straightforwardly say "This is bad, actually"? Case in point: In this article juxtaposing the grandiose claims of OpenAI et al with the massive environmental footprint of LLMs, Goldman still has to include this weird AI optimism:

https://fortune.com/2024/07/12/ai-bizarro-world-agi-electricity-carbon-emissions/

>>