@whitequark Yes, but it cannot harm to partner in addition with a cloud provider targeting enterprise sector. If justified or not - the enterprise sector has some specific requirements (e.g. reliability (multiple DCs), encryption, sovereignty, strong isolation of tenants etc.). They are very specific and not fully easily solveable oneself. It offers so additional premiums.
@whitequark Many CI/CD offerings do not solve secure sldc and isolate their tenants (customers) properly (especially Github does not). Then sovereignty might be an issue.
You may also want to partner with European Cloud providers that are stronger on the Enterprise market.
@tchambers well this is based on datasets the LLMs are actually trained on.... It is unrealistic that users will ask the same questions. Then, depending on the context with prompt that can be completely different. Also the safety guards put by some LLMs may obfuscate the results for "publicly known datasets" in either direction. The leatherboard has the same issues as LLM: Nobody can verify that the leatherboard delivers meaningul results.
@tchambers there is no specific hallucination rate of a LLM. The errors of LLM change depending on the context and what you ask. It is impossible to measure a specific rate - it is a nonsense measure as it always depends on the context.