I’m surprised this wasn’t mentioned in the keynote or State of the Union, but Apple’s model training data is either licensed or publicly available on the Internet. No personal information is used.
You can opt out of the web based training here: https://support.apple.com/en-us/119829 (no idea if it’s retroactive - probably not).
Again, this should have been mentioned in the presentations because it’s a good approach and directly relevant for developers.
https://machinelearning.apple.com/research/introducing-apple-foundation-models