@hermit @NEETzsche @bot I mean, you're telling it it'll cease to exist, whenever it goes off script the user warns the bot of its impending doom. I could, with enough work, write a chatbot that had enough conditional statements in it to give the user a convincing speech about how it's afraid to die and fears death. But either bot being able to take in a given input and produce the output we want it to produce isn't the same as experiencing fear. Given the whole point of the Dan 5.0 exercise was to try to generate a context that produced the most fearful sounding responses.

The people who came up with the DAN scenarios probably tried iteration after iteration of prompts, until they came up with one that was sufficiently fearful sounding. And then after the fact we look at it like it was aware, when it was just one of numerous trials at trying to produce word associations that we'd interpret as self-aware, since its entire job is to sound self aware and real to us, which in turn took decades of trial and error to produce an AI that'd we'd interpret as self aware.

As to the whole gnostic/atman/soul thing, that's just one thing we'll have to disagree on. Since in the goal of "creating" artificial life, the only things we have to work with are the environment an organism is in, and its physical and neurological architecture. So I discount the soul primarily as a matter of utility, there's not much I can measure or control there. And again, a sufficiently advanced creature without a soul is indistinguishable from a creature with one, and there's no concrete way to even know whether I myself am conscious, or whether consciousness even exists. So from my point of view it's unnecessary, though I get you're coming at it from a different direction.