@radmin @moffintosh This is pretty accurate. Basically the problem is that as the database grows, performance gets worse. There's not a good explanation for it because it shouldn't be an issue from everything I've read but the fact remains that the database is ultra fast when it's nearly empty but slogs when it gets big.

I'm running high end nvme drives and it's still uncomfortably slow.

Another aspect is when you have all this data scrapers are merciless and it the performance problem multiplies until the the site literally goes down from load.

I wanted to keep this database as a living archive of the early Fediverse. It still will be to some extent but I can't keep all the data around forever anymore.