This morning, I requested a number of popular LLMs answer a simple legal question of the form "it is illegal in England and Wales to do [X] - please set out the legislation that makes it such".
The response discussed a piece of legislation that was repealed in 2005.
And another piece of legislation that applies in Scotland. The fact "(Scotland)" is in the short title might be some kind of clue.
It completely failed to state the current law.
Still, let's pump it into the nation's veins.