Then there's reliability, if everyone goes onto one server it slows down and stops. If we are spread out and one server goes down, the others work fine.
And there are extra features on some servers (my personal account is on a server with a 1000 character limit), or even totally different feature sets.
But, the main thing in spreading out on to smaller servers is to keep the network as a whole safe. It's a civic duty to spread out. Larger servers are less safe.