What people don't realize about genAI is that their creators are just as suprised as you are that it kind of works, having no clue themselves regarding why this is the case, and also no clue regarding why it often doesn't work, so that they can't fix it so that it never fails.
All a genAI chatbot does is to predict, probabilistically, what the words you write in the prompt would be followed up by, given what people have said on the internet previously, including books they published, nonsense they wrote on X, and much more.
It is a big autocompletion machine. It may feel like it is doing something intelligent, but they are just regurgitating what people have already said, without any attention to reasoning or logic, emotional intelligence, and so on.
Just as people say nonsense everywhere in the internet, so does genAI because of its training.
GenAI can't, by definition, come up with original ideas. It can only regurgitate what people already thought and wrote about.
Sometimes this can be useful (and even give a Nobel Prize). But don't confuse this with intelligence.
AI stands for Autocompletion Industry, which doesn't involve any kind of intelligence - just probabilistic prediction based on what people already came up with (good or bad).