Embed Notice

@thomasfuchs LLMs are stochastic (not deterministic) algorithms, they don't always produce the same next token for the same input (unless you set the temperature to 0).

Edit: but the stochastic bit is at the very end where it uses the temperature setting as a weight to determine which of the most likely tokens to output based on randomness.