Prof. Emily M. Bender(she/her)
Anthropic is out with new nonsense about Claude (or the collection of models called Claude) deserving moral consideration and/or being able to feel distress, so my inbox is filling up with journos asking me to comment.
I've also got a day job, so here is a brief set of public comments on this:
[Note to the mansplainers of Mastodon: This is a thread, so please read the whole thing before replying. Please also know that I am aware of things beyond what's in this thread...]
>>
Prof. Emily M. Bender(she/her)
In other words, it is the job of scientists to devise experiments that can test their claims, to always be cautious about what they are claiming, and to critically evaluate the claims of other scientists. By these well-established standards, no serious science is happening behind Anthropic's claims.
/fin
Prof. Emily M. Bender(she/her)
Thus, any evaluation of LLMs that involves looking at either language produced by some software, or language input into the system for that matter, needs to be done with extreme care to avoid projecting our interpretation of the language onto the system.
>>
Prof. Emily M. Bender(she/her)
I will point out, however, that serious scientists are cautious with their claims. Furthermore, as we wrote in 2021 (in "On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?🦜"), p.616:
https://dl.acm.org/doi/pdf/10.1145/3442188.3445922
>>
Prof. Emily M. Bender(she/her)
3. Okay, so what would convince you that a piece of software (often phrased here as "an AI", gah) is conscious/can feel pain/etc?
Honestly, I'm really tired of this extremely silly question, but here is my answer:
Extraordinary claims require extraordinary evidence, and I am not the one making the extraordinary claim, so it is not my job to specify what the evidence should be.
>>
Prof. Emily M. Bender(she/her)
2. But Anthropic researchers say they see patterns that suggest that Claude can.
The employees at Anthropic are telling themselves stories based on the words they input into the software and the words that come back out.
It's the job of the media to scrutinize such statements and decide whether there's a story here at all (maybe the story is Anthropic researchers have fallen into a fantasy-land and brought many policymakers with them), not to just platform them.
>>
Prof. Emily M. Bender(she/her)
1. Can a language model feel pain?
Clearly not. Large language models are computer software and they can no more suffer than any other human-created artifact can.
>>
S. John Ross
@emilymbender I do wonder, though, which is worse, and which is more true: that Anthropic's people are drinking the Kool-Aid, or that Anthropic's people are coming up with creepier ways to try and sell it.
S. John Ross
@emilymbender I often think of teddy bears.
It's not unusual for a human to love a teddy bear and to project onto a teddy bear and to imagine that the teddy bear loves them back.
But that's an act of fantasy and we generally agree that it's so. Adding a voice so a Teddy Ruxpin could talk didn't change it.
A teddy bear mathematically modeling how hurt feelings are expressed in words doesn't change it either.
It's disturbing that, for some, it seems to make the difference.
Rupert V/
@SJohnRoss @emilymbender My impression is that Anthropic is mainly cultists and OpenAI is mostly scammers, but they're both a mix.
GNU social JP is a social network, courtesy of GNU social JP管理人. It runs on GNU social, version 2.0.2-dev, available under the GNU Affero General Public License.
All GNU social JP content and data are available under the Creative Commons Attribution 3.0 license.