Conversation

Notices

1. Embed this notice
   Hailey (hailey@hails.org)'s status on Wednesday, 12-Nov-2025 22:29:35 JST Hailey

   • ✧✦Catherine✦✧

   @whitequark fun to see you use garbage collection for grebedoc. I also landed on garbage collection as the right solution when reworking gh pages to do its own replication rather than drbd back in the day. very different set of design constraints and different kind of gc, but interesting to see it recur as the right solution for this kind of thing

   • Embed this notice
     ✧✦Catherine✦✧ (whitequark@mastodon.social)'s status on Wednesday, 12-Nov-2025 22:29:34 JST ✧✦Catherine✦✧

     in reply to

     @hailey oh, nice to have the confirmation ^^
     any comments on the actual gc algo? i think it's ok but these things are really subtle

   • Embed this notice
     ✧✦Catherine✦✧ (whitequark@mastodon.social)'s status on Thursday, 13-Nov-2025 12:53:29 JST ✧✦Catherine✦✧

     in reply to

     @hailey thank you, makes perfect sense and yeah

   • Embed this notice
     Hailey (hailey@hails.org)'s status on Thursday, 13-Nov-2025 12:53:30 JST Hailey

     in reply to

     • ✧✦Catherine✦✧

     @whitequark no particular comments on the gc algo, you're using gc in a a very different way. we didn't attempt to do any object dedup for pages, whole site builds were just rsynced from build server to file servers. mysql was the source of truth for which file servers held site replicas, new builds were allocated to the N file servers with most free disk at time of build, and each file server ran GC to clean up dangling replicas at some point in time after the replica records were deleted from mysql. idea was we'd need some kind of gc anyway because we can't delete from mysql and the fileservers in one transaction, so skip synchronous fs deletions entirely and make gc load bearing thus continuously proving it works

   • Embed this notice
     Hailey (hailey@hails.org)'s status on Thursday, 13-Nov-2025 12:53:30 JST Hailey

     in reply to

     • ✧✦Catherine✦✧

     @whitequark no idea if it still works like this or whether it's all on azure object storage or whatever now, but i was pleased with how neat the solution ended up being once i had the brainwave to use gc, and i don't recall it ever having any major problems in production once shipped