Hour two of the Cloudflare outage, Mastodon retry queue at record length and failed attempts at highest amount on record too.
Conversation
Notices
-
Embed this notice
Kevin Beaumont (gossithedog@cyberplace.social)'s status on Tuesday, 18-Nov-2025 22:52:00 JST
Kevin Beaumont
-
Embed this notice
Andrew Golding (huronbikes@cyberplace.social)'s status on Tuesday, 18-Nov-2025 23:14:26 JST
Andrew Golding
@GossiTheDog incoming euro-pop song about Cloudflare?
-
Embed this notice
Kevin Beaumont (gossithedog@cyberplace.social)'s status on Tuesday, 18-Nov-2025 23:16:38 JST
Kevin Beaumont
Wild, entering hour three and still seeing sites down and about 40% failure rate on Mastodon Sidekiq queue requests.
Mx Jay Baker repeated this. -
Embed this notice
Kevin Beaumont (gossithedog@cyberplace.social)'s status on Tuesday, 18-Nov-2025 23:47:02 JST
Kevin Beaumont
Fun factor, if you google booter services (DDoS services), almost every major one is a Cloudflare customer. So currently, by anti-DDoS provider Cloudflare being offline, the world is a safer place.
Rich Felker repeated this. -
Embed this notice
Jérôme Meyer (jmeyer@infosec.exchange)'s status on Tuesday, 18-Nov-2025 23:49:22 JST
Jérôme Meyer
@GossiTheDog Sadly, already over, like a beautiful rainbow in the storm https://infosec.exchange/@jmeyer/115571243852832673
-
Embed this notice
Kevin Beaumont (gossithedog@cyberplace.social)'s status on Tuesday, 18-Nov-2025 23:54:26 JST
Kevin Beaumont
Cloudflare says it is resolved, and my Mastodon server queue is clearing now... so prepare for DDoS to resume, since their DDoS botnet customers are back online.
In conversation permalink Attachments
-
Embed this notice
Jérôme Meyer (jmeyer@infosec.exchange)'s status on Tuesday, 18-Nov-2025 23:55:36 JST
Jérôme Meyer
@GossiTheDog Might be location-dependent — haven't had an error message since
In conversation permalink -
Embed this notice
Kevin Beaumont (gossithedog@cyberplace.social)'s status on Tuesday, 18-Nov-2025 23:58:50 JST
Kevin Beaumont
Here's the 6 month Mastodon Sidekiq track, you'll see the highest amount of failures on the far right today in red - that was the Cloudflare outage.
Hopefully it encourages more #mastoadmin to consider if Cloudflare is a good choice.
In conversation permalink Attachments
-
Embed this notice
Kevin Beaumont (gossithedog@cyberplace.social)'s status on Wednesday, 19-Nov-2025 00:09:16 JST
Kevin Beaumont
17 hours from now, somebody will reply to this thread saying 'It's not down, works for me'. Always happens. I love the internet.
In conversation permalink -
Embed this notice
Paul Cantrell (inthehands@hachyderm.io)'s status on Wednesday, 19-Nov-2025 03:56:50 JST
Paul Cantrell
@GossiTheDog @tiotasram
“First we’ll poison you, then we’ll sell you the antidote”In conversation permalink -
Embed this notice
Marcus Adams (gerowen@mastodon.social)'s status on Wednesday, 19-Nov-2025 15:19:29 JST
Marcus Adams
@nopatience @GossiTheDog Cloudflare offers DDoS protection. Anubis is just a local reverse proxy so it'll stop scrapers and bots, but it won't stop somebody just flooding your connection with useless traffic until it stops responding. But this is what, the 3rd or 4th time in just the last year or so that large swaths of the internet have gone down because of Cloudflare, so one site here and there going down to DDoS is arguably preferable to everybody being centralized.
In conversation permalink -
Embed this notice
Fernando (fsjr@mastodon.social)'s status on Wednesday, 19-Nov-2025 15:19:29 JST
Fernando
@gerowen @nopatience @GossiTheDog Not nice when it's my website, and I would like it to stay up and also not have an insane bandwidth bill.
Everybody saying "JUST STOP USING CLOUDLFARE OMG LOL YOU IDIOT" isn't really considering the needs of folks running smaller sites, having less experience or a smaller budget trying to keep their little site online.
Major "why don't you just buy a huge operation to run your little website like the large company I work for does?" vibes. All of you.
In conversation permalink -
Embed this notice
Christoffer S. (nopatience@swecyb.com)'s status on Wednesday, 19-Nov-2025 15:19:31 JST
Christoffer S.
@GossiTheDog I'm seeing the same thing, several orders of magnitude more retries than usual.
Why? I mean... how could many small instances possibly need "protection" from Cloudflare? C'mon the fuck on.
Get something like Anubis (https://github.com/TecharoHQ/anubis) and be done with it.
Cloudlare? ...Shsszz...
In conversation permalink Attachments
-
Embed this notice
Kevin Beaumont (gossithedog@cyberplace.social)'s status on Wednesday, 19-Nov-2025 16:08:49 JST
Kevin Beaumont
Cloudflare RCA is out.
Basically a few things went wrong which then hit a coding error. https://blog.cloudflare.com/18-november-2025-outage/
In conversation permalink Attachments
-
Embed this notice
daniel:// stenberg:// (bagder@mastodon.social)'s status on Wednesday, 19-Nov-2025 16:13:08 JST
daniel:// stenberg://
@GossiTheDog but it was memory-safe! 😎
In conversation permalink -
Embed this notice
Dante Vortex 💾 (dantevortex@cyberplace.social)'s status on Wednesday, 19-Nov-2025 18:50:44 JST
Dante Vortex 💾
The biggest question I got is why they didn't find out during the standard OTAP procedure they "allegedly" are following like a professional company would.
In conversation permalink -
Embed this notice
𝙽𝙴𝚃𝚁𝙴𝚂𝙴𝙲 (netresec@infosec.exchange)'s status on Wednesday, 19-Nov-2025 21:20:59 JST
𝙽𝙴𝚃𝚁𝙴𝚂𝙴𝙲
@da_667 @GossiTheDog From https://urlhaus.abuse.ch/statistics/
In conversation permalink Attachments
-
Embed this notice
da_667 (da_667@infosec.exchange)'s status on Wednesday, 19-Nov-2025 21:21:00 JST
da_667
@GossiTheDog Many different types of malware campaigns use cloudflare. Many fake search engine ad campaigns, click-fix and various other malware campaigns have C2s or distribution servers on cloudflare
In conversation permalink Kevin Beaumont repeated this. -
Embed this notice
Dante Vortex 💾 (dantevortex@cyberplace.social)'s status on Thursday, 20-Nov-2025 02:25:06 JST
Dante Vortex 💾
Sorry that was a Dutch thing,
It's the "Development(Ontwikkel), Test. Acceptance, Production"
But I already learned this was a bug that's been in the code for a longer time and for some reason it just happened to show up right now.
In conversation permalink
-
Embed this notice