Embed Notice

@ryanjyoder
All models work fundamentally in the same way: predicting a series of output tokens based on a series of input tokens. If you don’t understand this basic implementation mechanism, the rest of this conversation is inaccessible.

It doesn’t store facts. It doesn’t have a representation of “true” or “false.” It isn’t a database. It splits written text into tokens and does colossally huge, environmentally damaging, and fabulously expensive “training” on that data using billions of parameters to arrive at a statistical model of tokens that follow other tokens. The model can then be queried to produce statistically likely replies to inputs.

Given an input like “tell me a lie about the capital of France” the most statistically improbable reply is “the capital of France is Paris.” Other replies like “wear a seatbelt” are also super improbable. The size of these models and the probabilities they work with are really difficult to get one’s head around. But it returned a statement that was a probabilistically likely reply to that input. That’s all it did.

When models make up legal cases that don’t exist, books that don’t exist, programming APIs that don’t exist, etc, they are simply outputting likely results. Text that fits the probability distribution of their input data. That’s why it is not a “bug” when an LLM bullshits. It’s not an error. It is working as designed.

There is nowhere to report to an LLM company the factually incorrect outputs its model produced because there is nothing they can do with that. It is working as designed.

@ralfmaximus @glc