I'm not a sysop, but think the most crucial thing with Mastodon is number of simultaneous processes server can cope with.
If too many requests come in (such as unusually large number of people joining all at once) then it starts to form a backlog and the server has to delay stuff so it can work through the backlog. This is why messages start showing up late in timelines for example.
If you're techy, official documentation on scaling a server might be useful?