we do not have the perfect society where massive wealth cannot influence lots of people in lots of different ways. I'd be for ancapistan or communism if I thought people were interested or able to thrive in those situations too etc.
you should care whether it's a billionaire or 100k middle class dudes. because wanting to do evil is rarely the problem. it's not understanding other perspectives and not having to compromise. that's the danger of billionaires in a democracy.