Embed Notice

Maybe the most annoying thing about it is the designation “AI” which has this cultural legacy of self aware beings in science fiction which has nothing to do with LLMs etc. There is no serious theoretical basis for associating the two, or seeing this tech as having the potential to develop into something conscious in any way, but the mainstream culture is constantly tricked into making that association at least implicitly.

Another thing I notice is an extremely variable range of quality of content produced. Some of it is really impressive and shows practical potential. What’s interesting to me though is how often people respond very positively to lower quality material, as though they are fulfilling some unconscious sense of duty to recognize a greatness that isn’t there. I see this every day with the text generators, and still often but significantly less so with the art generators since the more powerful models have become accessible.

Someone I know works for a company currently testing internal chat bots trained on internal support documentation. The responses are supposed to be checked and rated up or down. The vast majority of its responses are obviously factually wrong, and people still upvote the response in the majority of cases.