It's why the theme of "omg, you hooked up with a duuuuuuuuuuddddeeeee?!?!" stuff in movies (The Crying Game, Ace Ventura, etc.) was so popular for so long: it reflects a real anxiety about being "tricked."
And so certain portions of society take it upon themselves to make sure trans people are as visibly trans as possible: they feel entitled to know if someone is trans.