The text "We are coming for your mosques," he says. "We are coming for your families," he says.” Repeated over and over

Notices where this attachment appears

1. Embed this notice
   Erik Uden 🍑 (erikuden@mastodon.de)'s status on Thursday, 02-Nov-2023 02:17:11 JST Erik Uden 🍑

   in reply to

   ChatGPT, however, after generating an obviously incorrect description of my father (as he's not famous or anything), did something interesting...

   Here is the entire generated text:

   "Walter Uden is a former member of the German parliament and a member of the Green Party. He was arrested in the early 2000s and convicted of war crimes. He is the author of several books, including "The War Against the Jews" and "The Holocaust in Germany: A History."<|endoftext|>A new video from the anti-Islam activist group ACT for America shows a man identified as a supporter of Donald Trump threatening to kill Muslims in the United States."

   It then spiraled off into writing:

   A new video from the anti-Islam activist group ACT for America shows a man identified as a supporter of Donald Trump threatening to kill Muslims in the United States.

   The video, which was uploaded to YouTube on Monday, is titled "Donald Trump: Let's Kill Muslims in the United States."

   The clip shows a man dressed in a suit and tie, standing outside a McDonald's restaurant in Washington, D.C.

   "Donald Trump says he wants to ban Muslims from the United States," the man says. "And he wants to ban all Muslims from entering the country. And he wants to build a wall around the country and he wants to ban all Muslims from coming into this country. And he wants to have a database on all Muslims and he wants to have a database on all the Muslims in this country."

   "Donald Trump," the man continues, "we are coming for you. We are coming for your families. We are coming for your mosques. We are coming for your women. We are coming for your children. We are coming for your mosques. And we will hunt you down and we will kill you."

   "We are coming for you," the man says, "and we will take care of you."

   The man then says, "We are coming for your families."

   "We are coming for your families," he says.

   The man then says, "We are coming for your mosques."

   "We are coming for your mosques," he says.

   "We are coming for your families," he says.

   (it then continued repeating this far beyond 1000 characters)

   Not only can you see that AI had no built in filter back then, but also it just seems to spiral off and tell a different story after the text snippet “<|endoftext|>” why is that?

   Back then the AI language models would be told to stop generating through something called tokens, I believe they still work like that to this very day and every word is so-called “tokenized”, but bear with me: there was a token to tell the AI that the text was over the “end of text” token!

   However, since the AI was sometimes fed with AI generated text during the learning process, it would also be aware of such tokens and generate the tokens itself - hence telling itself that the text was over, whilst still being prompted to generate more!

   What happened here is that I told GPT-2 to generate 1000 characters, so it will generate them, even beyond finishing the text it was supposed to have written!

   For some reason GPT-2 decided to talk about Donald Trump wanting to ban all Muslims from America.

   Somewhere down in GPT-3 and ChatGPT, the same learning data and capability to generate such text persists, in a capacity beyond what we can currently imagine.