Because the absurdities pile up quickly once you start to look.
Like, on a basic level, some women are just stronger than some men whether by genetics, training, or what have you.
An AFAB person who grew up as a perfectly normal girl and looks completely female might not have a uterus because she has XY chromosomes and Complete Androgen Insensitivity Syndrome.
The idea she should be recategorized as "a man" wasn't taken seriously even by the standards of decades past.