@tchambers well this is based on datasets the LLMs are actually trained on.... It is unrealistic that users will ask the same questions. Then, depending on the context with prompt that can be completely different. Also the safety guards put by some LLMs may obfuscate the results for "publicly known datasets" in either direction. The leatherboard has the same issues as LLM: Nobody can verify that the leatherboard delivers meaningul results.