"Certainly down in the rural south where I was raised, there'd be trouble if you accientally called a man "ma'am"."
This, this right here. I'm glad to see someone else mention it, because I feel like it's a significant piece of the puzzle.
A lot of the reason people don't mention it, I think, is that they're worried it's going to sound like, "why should I have to change? I learned what I learned, and that's good enough". And I suppose that would be the case from some people. In my case though - and yours - it's acknowledging that there's some mental reprogramming at work, so we're fighting the existing programming.