@Infoseepage We can only agree with you. The massive delay is because of big API change at Wikimedia... and lots of consequences of it. We are still working to be back to normal.
If there is any silver lining to this, it is that we could see a few points of vulnerabilities as well as our ability to turn things around in a reasonably quick manner (here be kudos for the two heroes who manage our infra).
Learning were made, and we will see in the coming weeks/months how we can implement new safegards within our resource constraints.
Luckily we have mirrors and these were not affected. We grabbed a new machine somewhere else (Scaleway ; if we name-and-shame the one we might as well name-and-greet the other) and immediately started re-importing our data to our new Master server.
All in all, it still took about 48 hours to get these 8-ish TB back online.
Turns out that Hetzner has decided to cancel our account and terminate all servers. There was no warning (yes, we checked our spam folder), and nobody could be reached before Monday morning.
When reached, they could not explain the reason for the cancellation: Them: - We sent you an email. Us : -We did not receive it, can you please resend? Them: - We don't have it Us: ಠ_ಠ
Murphy's law states that if things can go wrong, they will. Ideally in the worst possible way.
For us, that meant having our servers disconnected at 00:00 on a Sunday.
Our main storage backend became entirely unreachable. For the average user that meant not being able to access the library and download files, and for us that meant not being able to connect to it and see what was wrong.
@Gargron Yes, but a special form, in comparison to a traditional Foundation or Association. It has more in common with a traditional company than usual in NGO field. It’s not a critic. Seems just challenging to face capital based takeover attempts as long as you have a stock based company. What would happen if key shareholder(s) would die? In a nustshell, maybe the current setup is not the most appropriate anymore to protect Mastodon achievements and secure further developments on longer term.
@Gargron Any plan to place #Mastodon IP, R&D and operarations (mastodon.social) under a formal non-profit ombrella organisation? Something similar to the move made by Jimmy Wales with the Wikimedia Foundation? This would allow to definitly protect our digital common of any capital based raid.
Governments are turning off the internet to silence dissenters at an ‘exponential’ rate—and threatening civil society, says Dan Keyserling, the head of Google’s Jigsaw project. We sat down with him to discuss the phenomenon. https://trib.al/W50SEPI
Web content for people without internet access. Go #offline and take Wikipedia, TED talks, the Gutenberg Project library & more with you! https://kiwix.org