There's a lot of layers to it. Yes, there's the power corrupts/only the deranged wish to control tons of people consideration. But even if they're normal, by being the final say over the experiences of tons of people, any minor flaw in judgment can really mess with the overall network experience. The larger size amplifies any issues. Likewise, large instances are intrinsically bad for the purpose and resilience of the fedi.

It also just so happens we've seen a repeated pattern of dysfunction from several prominent admins (Grad, P, now Moon, etc.) that often results in them picking fights with us and people we like.