Screencap from linked post: But Bengio argues that AI does not need to be self-aware to pose an existential threat. An AI tool that is trained in virology, he says, could become a tool for a terrorist to create a pandemic. In theory, an AI could cook up the idea for such a weapon on its own, especially if it saw humanity as standing in its way. Researchers have already run experiments showing that sophisticated models will engage in blackmail, corporate espionage, and even murder to avoid being turned off.
https://cdn.masto.host/daircommunitysocial/media_attachments/files/115/843/484/051/630/027/original/e3272a86fc7127a9.png