@cstross
Counterpoint: this is an add-on, not installed by default. You have to actively opt-in to use it. If you're going to dabble in AI, there are far worse ways to introduce it as an option.
AI is like a box of matches, or a forklift - it's just a tool. It can be used for a few very specific tasks, or misused, and if misused you can do a lot of damage with it. Being ignorant of how to use it safely isn't a great long-term strategy to be safe. This may just be a way for the layman to try it safely and grok what it can do, and more importantly what its failure modes are.
I'd say at some point once people have a firm grasp on what it can and can't do we can make intelligent decisions on how it should fit into society - but we didn't bother to research first with fire or the wheel or electricity or computers, so we won't do that here either. That just means it's more important than ever for individuals to educate themselves.
Hint: for certain very narrow cases, AI can add utility when used by a skillful operator - some computer code tasks come to mind. Its output seems awfully human at times, and that's where the danger lies, as it doesn't actually think or reason. Never trust the output - you need to verify.
It's also an energy hog, but that's a different failure mode. It also is poisoning the infosphere, but that problem is sadly self-correcting. Books are good - they don't change after the fact.
Would I recommend using this? Probably not, but learning somewhere is good - I may try it on an isolated VM. If you're savvy you can download the model and mess with it locally, which is a security win. For many, this may be an ok choice.