You could rebut that "anybody can wear a tee", and it's true. It comes down to how evident the statement is, and who is it addressed to. The pin was really subtle, and so addressed at "the marginalized in-crowd", like an obscure shibboleth. Some tees I've seen are really subtle about their trans flag, so you're almost left with something that says "You'll have to go through me" without context, and that's useless, too.