I think it has more to do with the puritanical influence on US culture.
The toxicity is not a white thing, so much as a white American thing.
For example, men in Sweden or Finland are as involved in childcare as Black men in the US are. Men in Europe are much more likely to wear pink, eat gelato in public (gasp!), hug their friends, etc. as a result, European men are much less lonely than American men.
https://www.apa.org/news/press/releases/2024/03/middle-aged-americans-lonelier
So Black men are unique in this regard *In America*.