Notices by Emelia 👸🏻 (thisismissem@hachyderm.io)

@mattly it's on the list, iirc. I think it's waiting for the FEP to finalise?

@mattly yup, they're applicable here, though most known for input constrained devices.

I'm planning to bring them to Mastodon this year if I can secure funding for that work.

@mattly perhaps also for that is using Device Code Grant Flow for input (UI) constrained devices or clients.

Your terminal app should probably be using this instead of asking for a password directly.

@badnetmask yeah, I'm trying to figure out if this occurred before last night.

@badnetmask so last 24 Hours but not earlier than that?

@badnetmask yeah, this was actually a bug. Do you know roughly when you noticed posts with missing images first?

@badnetmask I don't think it's egress, just cold storage. They could even move media not frequently accessed to glacier tier storage

@badnetmask @malte mmm, I've no idea about any of that 😅

I did use LUKS last time but managed to shutdown the machine & it became totally bricked needing OS reinstall, so I'm rolling out debian for it

@badnetmask @nevarro I'm pretty sure its just all the pipes in my head up there are f'ed — I've already had my nose completely rebuilt a few years ago

@badnetmask nah, we'd accidentally had pgbouncer configured for 2000-2500 connections, but the process had LimitNOFILESoft of 1024, so we were running out of sockets and accept() calls were failing when mastodon tried to connect to the database.

Issue only surfaced 2 days ago but we hadn't changed server configuration in 20 days, just upgraded system packages on non-database nodes in our infrastructure, so we thought we'd broken a dll linked library in ruby like libvips or libicu

we thought we'd fixed it yesterday, but had the wrong root cause. The error message was obtuse and the promtail log lines dropped the stack trace lines that followed, so we only had partial information & thought the error must've been from the upgrade, when in reality it was just a freak coincidence.

Finally figured it out earlier & deployed a fix. But it took a few tries to get it right.

We also now have alerts in place if pgbouncer errors again (we're in the process of rebuilding the primary database server & switching to pgcat which has much greater observerability)

The other option was that it was a networking error, but we could access the database node via the tailnet that connects our infrastructure.

@molly0xfff aha, this explains it, because there's absolutely no way the token has $20 billion backing it — even if all 77 million people who voted for trump bought it at 53$ it'd only be $4.08 billion

@mekkaokereke current estimates of yearly cost per account is $0.30 to $0.80 based on infrastructure, storage, etc. From what I've seen.

I'm pretty sure @esk or @dma worked out the numbers for Hachyderm too.

@skinnylatte another factor here is the oversupply of bootcamps and the differential between what they teach and what you'll actually need — I see a lot of these bootcamps in Germany which are really just a grift for government dollars (through providing "training"), they specifically target people who can't navigate the german system & also cannot challenge them within the german system.

I'm certain that also holds somewhat true for other countries, though maybe instead of a grift for government funding, it's a grift that saddles the person learning with bad debt or wage garnishments.

@hrefna @mxk yeah, we're doing review and all that before we make changes, but even then we can't sometimes be 100% sure everything is right, so we need to just double check everything's actually fine live.

@mekkaokereke this was added by Eugen and improved upon by various other people

@hrefna this wasn't about mastodon, but rather how developers work in general: the tech we use in the startup world is different to in enterprise world, which is different to that which a hobbyist might use.

@mekkaokereke I'd love to get some of those people involved in the ActivityPub Trust & Safety Taskforce, but as I don't have strong relationships with these people I'm not sure how best to invite them, and am also cognisant that standards work is an entire thing, and can be very laborious, so I don't wish to expect that of anyone from a marginalised background.

I'm currently making sure to cite articles by black & brown authors when I know of a relevant article to a specific work item.

@hrefna @DenialShown will be interesting to see what the ethics committee says about this

@molly0xfff do we even want to know what "fair democracy" is? It doesn't sound good as a company.

@mattly @graue "this has been rebased for nearly 2.6 years, that's kinda absurd" says it all really. I hope things improve soon, I really do.