“Here we see the fundamental flaw of the system: ‘AI Overviews are built to only show information that is backed up by top web results.’ The design is based on the false assumption that Google's page-ranking algorithm favors accurate results and not SEO-gamed garbage. Google Search has been broken for some time, and now the company is relying on those gamed and spam-filled results to feed its new AI model.”
https://arstechnica.com/information-technology/2024/05/googles-ai-overview-is-flawed-by-design-and-a-new-company-blog-post-hints-at-why/