I checked Sidekiq and my queues were hugely backed up, like nearly a thousand jobs, so I just emptied all of them. Hopefully that won't cause any issues with the site, but it certainly seemed to have made things more responsive.
So, assuming this wasn't a stupid thing to do for some reason I don't know about, I recommend doing that if anyone else on masto host are still seeing problems.