Notices by pixxl (pixxl@mastodon.social)

@cthos @thomasfuchs Temperature itself has nothing to do with determinism either. It all depends on the RNG used by the control system which picks tokens. If it's a regular seeded RNG then using the same prompt and same seed will produce identical responses.

@cthos @thomasfuchs LLMs are deterministic, as they are just machine learning models. The way they're used (by the control system) could be non-deterministic. In the end it depends on the control system since they could work in any way, but using the same prompt and the same seed will always produce the same output with most systems. Arguably the control system (the thing that picks the next token from the distribution) is not part of the model itself.