For USB token auth, it's just an HID device that communicates using the CTAP protocol to serialize requests/responses, so there's not much capacity for it to talk to the outside world, unless you fabricate some RF-emitting component inside the token to transmit to some auxiliary wireless network to exfiltrate that information. It's just a very opinionated standard of public key authentication, anyone's free to implement hardware as they so choose.

My interest in it is solely for hardware-backed authentication, versus private keys that are resident within your filesystem or RAM (such as when a private key is unwrapped). You can also use a token for SSH public key auth for cheap.

Of course it still falls into a matter of trust of the hardware vendor, but that's also the same dilemma but on a much wider scale with most desktop computing hardware.

Nonetheless, as stated: my interest is for USB token authentication, used as a second-factor of authentication. I'm questionable in some areas, such as using a smartphone as a single-factor authenticator (regardless of whether it has it's own isolated hardware cryptographic component). I only advocate for it within the former profile. There's also the standard itself which is openly documented and inspectable (especially in the device communication), and if it starts to get shoved in the wrong usage, then of course that's time to raise hell if any of the larger orgs steer adoption in the wrong direction.