The philosopher Harry Frankfurt defined bullshit as speech intended to persuade without regard for the truth. By this measure, OpenAI’s new chatbot ChatGPT is the greatest bullshitter ever. Large Language Models (LLMs) are trained to produce plausible text, not true statements. So using ChatGPT in its current form would be a bad idea for applications like education or answering health questions.
Despite this, there are three areas where LLMs can be extremely useful: https://aisnakeoil.substack.com/p/chatgpt-is-a-bullshit-generator-but