One reason we're not angrier is that we don't see all the abuse that marginalised people get. We don't see people's DMs, for example, and we don't see publicly visible abusive messages that someone else has already reported and had deleted by an admin. A troll can also use message visibility settings to get friends to pile on to a marginalised person while the rest of us don't see the abuse.
Because I'm not a target, I have no figures on how often this happens.