If there is a working design for a nuclear weapon on the internet, then LLMs will plagiarize it.
If there isn’t, they’re not going to come up with one.
This is just cheap journalist bait. Every fantastical headline-grabbing “OMG our imitation machine is too dangerous for humanity!!!! (attn: investors)” press release like this is a play for headlines.
In 1974 the supreme court in the US that the main Chicago broadsheet could in fact publish a 2 page spread of a suitcase size nuclear bomb.
A journo went to the public library and basically researched everything necessary to build such a device. So the design was based entirely on publicly available knowledge.
The Department of Energy (iirc) tried to stop publication, but because the design did not include details of the mere 3 actually classified top secret components (which I'm not going to mention) SCOTUS found that it was within the 1st amendment to publish the design.
So, on the internet? Probably not. On a back catalog microfiche somewhere ...
It turns out there's very little that's actually *secret* about nuclear bomb design.