Conversation
Notices
-
Embed this notice
@1iceloops123 @moffintosh @radmin around 640GB
-
Embed this notice
@roboneko @moffintosh @radmin @1iceloops123 I mentioned that upthread that I know it shouldn't be that way so I don't have an explanation for it.
-
Embed this notice
@Moon @moffintosh @radmin @1iceloops123 out of curiosity I assume the entire thing is on nvme? it's weird because postgres gets used at _much_ larger sizes so you've got to wonder WTF is causing it
-
Embed this notice
@i @roboneko @moffintosh @radmin @1iceloops123 I understand why it takes so much space, I don't understand why it slows down.
-
Embed this notice
@roboneko @moffintosh @radmin @1iceloops123 @Moon it's just a complete mess of an implementation on the pleroma side, 30gb of activities needs 24gb of indexing, 20gb of objects needs 9gb of indexing (on this instance), not to mention all the wonky pgplsql functions making even fundamental things like the home timeline non deterministic
no one cares enough anymore fix any of this mess, and maybe that's for the better
-
Embed this notice
@graf @moffintosh @radmin @1iceloops123 I wrote that confusingly. they are two separate problems. someone was raping my server, every core was 100% PostgreSQL. This is while Pleroma itself is on a separate box still.
-
Embed this notice
@Moon @1iceloops123 @moffintosh @radmin ours is over 600GB and we don’t have this problem. i dont think it’s scraping as much as the rot meme. the added load might slow it down but it’s not the root cause
-
Embed this notice
@graf @moffintosh @radmin @1iceloops123 I don't like calling it rot but I admit it's a semantic distinction. I have worked with databases that are much larger than Pleroma's and don't have this slowdown problem. People keep saying "it's because JSON" but that just doesn't make any sense. I personally have a bigger DB than SPC at work that uses JSONB
-
Embed this notice
@feld @roboneko @i @moffintosh @radmin @1iceloops123 you're a three user instance.
-
Embed this notice
@i @roboneko @moffintosh @radmin @1iceloops123 @Moon
> no one cares enough anymore fix any of this mess, and maybe that's for the better
I've never experienced any of the weirdness SPC reports, personally, so it's hard to fix what you can't reproduce
-
Embed this notice
@birdulon @roboneko @feld @i @moffintosh @radmin @1iceloops123 There are also cases yeah where in bursts it is painfully slow for no reason.
-
Embed this notice
@Moon @roboneko @feld @i @moffintosh @radmin @1iceloops123 funnily enough this instance is similarly small yet sees awful performance issues from time to time
-
Embed this notice
Do you mind sharing how you're trimming the database? I've found that the activities table is the biggest beast.
-
Embed this notice
@Humpleupagus @moffintosh @radmin @graf @1iceloops123 I'm going to do a series of delete queries. Then when I am done I am going to do a full backup and then full restore.
-
Embed this notice
@Humpleupagus @1iceloops123 @graf @moffintosh @radmin if you don't do a delete and restore, it won't let go of the space.
-
Embed this notice
You can vacuum full, but you need enough space for the rebuilt db. That'll take a while given the size of your database.
You may be able to partition the tables based on date and then drop the partitions you don't want. That should be nearly instant. I wouldn't try this without a backup or without knowing exactly what you're doing. Often when I'm planning on doing something I'm not sure of in a live environment, I'll set up a test environment on a separate system so I fuck that up trying to figure out the correct process instead of the live server.