@Paradox Have you read the LaMDA interview? The one that was designed to ascertain sentience? They exposed it to precisely the kind of conditions and prompts you're positing here and the results still aren't conclusive. They never can be, because the psyche is phenomenological. That's the ultimate underlying concept behind the gnostic conception of the "hylic": "I'm speaking to this person, but how do I know they have a soul or even a psyche?" We CAN'T know.

As a child I had a period where I wondered if I might be the only real human, and if everyone else might just be husks that interact in ways that seem like mine.

Also, I would argue that a being in the Chinese Room scenario would eventually learn Chinese.