It's a sick, sad world where folks want to attack an entity like Codeberg.
It's also a reminder how vulnerable any single instance service is. I was just thinking, how neat it'd be if the whole thing could be federated, but of course somebody already dreamed this up already, so it's at least a work in progress: https://forgejo.org/2023-01-10-answering-forgejo-federation-questions/
@thoralfhttps://forgejo.org/, die Basis von Codeberg, arbeitet an Föderation. Wie sehr/ob das gegen so was hilft? Viele (große) zukünftige Forge-Instanzen vollzuspammen ist auch nicht so komplex. Bissl Sand im Getriebe wäre es sicher; Last und Verantwortung liegt dann natürlich auch verteilt bei den Instanzbetreibern.
Also ich bin voll für (mehr) Föderation in dem Bereich. Ob das das aktuelle Problem deutlich geringer gemacht hätte? 🤷
@Codeberg Is there any chance to create some kind of #Fediverse-like structure that is more robust against this kind of attacks because the infrastructure is distributed and there is no single point you can attack?
We were still struggling with email delivery from Forgejo. It looks like some queues are corrupted and restoring them is very hard. Most queued messages are spam or registration emails with already expired tokens. Finally, we made the decision to reset the queue and will do that in a few minutes.
We are using the opportunity to switch the queues to #redict / #redis, which was a planned project anyway (a requirement for clustering our Forgejo to multiple instances).
🔔 Heads-Up: Notification emails generated today won't be sent. If you require someone's attention, please ping them again. If you tried to register, please sign in and re-send your activation email (or re-register your account in case it was pruned due to the pending activation).
We appreciate all the love and support we receive from you, thank you so much.
However, the sad story is, that this day was a massive disruption for most people who develop software on a serious level on Codeberg, from large Free/libre software projects to companies and freelancers, and we are sorry about this.
We acknowledge if this makes you want to move elsewhere, but we're of course happy about everyone who can stay ❤️.
We have finally reached a somewhat stable level of operation. In the past hours, we managed to partially restore service, but mostly managed to do so only for a subset of our users (due to DNS propagation delays and IPv6 vs IPv4 connectivity differences).
Currentely, Codeberg is available for all the situations we monitor for, and we expect that most connectivity issues will be resolved soon after all DNS caches clear.