It's not as much about key length as it is about RSA in general.
There are, like, a HANDFUL of RSA implementations that are audited, trustworthy, and suitably protected against the insane variety of subtle mistakes that can completely wreck its security. Even using those implementations safely takes a lot of care and effort. Using other libraries is a gamble; rolling your own is the cryptographic equivalent of bungee jumping without bothering to check if your harness is attached.
Timing attacks, faulty prime selection, improper padding, poor public exponent selection, related message attacks, etc. Lots of it discovered 20+ years ago, yet still showing up in RSA software written today.
Heck, even partial key leakage is a shit show compared to more modern algos. If I accidentally expose a quarter of a SPHINCS+ key, the remaining bits should still need to be brute-forced. If I expose a quarter of the bits in a private RSA exponent, though, the game is up completely.
If you're gonna go classical, use ECC. If you're gonna go PQ, there are good options to choose from (and NIST standards coming out). Either way, deprecate RSA wherever possible.