Notices by Jörn Franke (jornfranke@mastodon.online)

@Bellingcat I saw that you use Google Docs to provide an overview of Bellingcat's Online Investigation Tools, why not something like @cryptpad ? @digitalcourage hosts an instance (https://cryptpad.digitalcourage.de/) and it could be interesting for bellingcat to host an own one

@tchambers well this is based on datasets the LLMs are actually trained on.... It is unrealistic that users will ask the same questions. Then, depending on the context with prompt that can be completely different. Also the safety guards put by some LLMs may obfuscate the results for "publicly known datasets" in either direction. The leatherboard has the same issues as LLM: Nobody can verify that the leatherboard delivers meaningul results.

@tchambers there is no specific hallucination rate of a LLM. The errors of LLM change depending on the context and what you ask. It is impossible to measure a specific rate - it is a nonsense measure as it always depends on the context.