Paul Cantrell
Paul’s Hot Take of the day: both Terminator and The Matrix misguide us when we try to understand the risks current AI poses.
2001: A Space Odyssey is a much better fictional model to study.
(Re this from @arstechnica: https://arstechnica.com/ai/2024/10/40-years-later-the-terminator-still-shapes-our-view-of-ai/)
1/
Paul Cantrell
@arstechnica In brief:
Terminator and The Matrix both make the mistake of anthropomorphizing the machines: they are superintelligent and inhuman, but also driven by human-shaped motivations (survival, power, self-defense, vengeance, the will to exist, “self-awareness,” yada yada).
2/
Paul Cantrell
In 2001, it’s the •humans• who make the probably-mistake of anthropomorphizing the AI. Hal speaks in grammatically polished yet conversational English. Hal feels like a member of the crew.
Hal’s failure modes, when they arrive, are bizarre and surprising. The cause of failure is not the machine acquiring Humans Feelings. It •may• be misprogramming / misprompting — humans giving instructions with unanticipated effects — or there may be no explanation at all.
3/
Paul Cantrell
I mention this for two reasons:
(1) Making us think of Skynet, and its associated errors of reasoning, is very explicitly a marketing strategy for AI vendors.
(2) As we think about how and whether to deploy generative AI, the first step of a proper risk assessment is to recognize that its failure modes won’t be human-shaped (or code-shaped, for that matter). Its failure modes will be bizarre, inexplicable, and hard to anticipate.
/end
Paul Cantrell
Footnote:
Mr. Data, my personal favorite fictional AI character, sits somewhat to the side of this conversation. In his story’s framing, he is a person; his narrative purpose is not to warn about the promise and peril of technology, but to teach us about acceptance of the Other, understanding across neurological divergences, etc.
I do see two ways he’s relevant, though:
/A1
Paul Cantrell
(1) The myth of human-like interaction combined with code-like correctness / logical consistency •is• part of the aforementioned AI sales pitch — and very clearly •not• something LLMs actually accomplish. Another trap there.
(2) The lesson of The Measure of a Man is relevant: the impulse to claim AI is human-equivalent without considering the AI’s humanity is evidence of a slaver mindset, and that’s true even when (as now) the AI is not actually human-equivalent at all.
/A2
Paul Cantrell
@nicklockwood
Huh, yeah. I feel like Ash is kind of muddled as a view of AI, more there are a narrative device than as a musing on machine nature. Agreed that MU/TH/UR probably has more for us to chew on.
Replicants are another one to consider, and probably wind up somewhere in the vicinity of what I said about Mr. Data.
Nick Lockwood
@inthehands what are your thoughts on Ash from Alien? Seems he's similar to Hal - selfless but also utterly alien (pun intended) in his thinking, with no sense of morality or loyalty to anything but his directives
Nick Lockwood
@inthehands (although I guess MU/TH/UR is actually more analogous to our current AI tech - capable only of simple automation and answering questions without having any opinions or goals of its own)
Paul Cantrell
@Sevoris
I should have specified that I’m considering Hal as presented in the 2001 film •only•, and not Dr. Chandra’s retcon explanation from 2010.
In the original film, Hal goes sideways and we simply do not know why.
Sevoris
@inthehands and yet I want to content that Hal is also a bad example, because Hal is a weirdly rational actor in his breakdowns: he‘s told to lie and can‘t. He understands truth and distortion.
Skynet blows people up cause it‘s a military AI and frames survival in terms of mutually assured destruction.
As "dysfunctional by intention" goes I feel Skynet captures the contemporary callusness of model design better than HAL.
Paul Cantrell
@hyc
Ha, I’d love to see that if you find it!
(The new sorting algorithm generated by an LLM of course, I’d assume nobody would understand how it works because it doesn’t.)
Howard Chu @ Symas
@inthehands just reminded me of the sort routine produced by a genetic algorithm some years ago, that made the news because "nobody understood how it worked". Can't find a reference now.
Howard Chu @ Symas
@inthehands hm, it was longer ago than I thought..... https://www.reddit.com/r/compsci/comments/p9sqi/sorting_routines_developed_by_genetic_algorithms/
Paul Cantrell
@hyc
Ah, that is interesting! The paper linked from the comments clears up a lot: it’s a sorting network. That means fixed size (not a general algorithm), which is much better suited to a GA. And it totally makes sense that (1) a GA could find a more optimal one and (2) that it would possess no human-recognizable organizing principle. Thanks for digging it up!
GNU social JP is a social network, courtesy of GNU social JP管理人. It runs on GNU social, version 2.0.2-dev, available under the GNU Affero General Public License.
All GNU social JP content and data are available under the Creative Commons Attribution 3.0 license.