Embed Notice

@dngrs @glyph Also note that I'm talking about systems that use RAG and large context windows to feed results directly into the prompt for summarization. I'm my experience, LLMs are much less likely to hallucinate in those cases, because they aren't reliant on latent information stored during training.

eg, If you ask an LLM, "tell me about the Floozbor API", most will happily make up bullshit for you. But if you take search results for [Floozbor API], feed them into your context window, and then ask for a summary, they'll do a decent job, because the key information is there in the input tokens instead of being extracted from the model weights.