Conversation

Notices

1. Embed this notice
   pistolero (p@fsebugoutzone.org)'s status on Thursday, 27-Mar-2025 14:44:21 JST pistolero
   Either Wordpress fucked up a deployment or they've decided to stop troubling FediList: https://fedilist.com/instance/recent-changes
   • realcaseyrollins ✝️ repeated this.
   • Embed this notice
     pistolero (p@fsebugoutzone.org)'s status on Thursday, 27-Mar-2025 14:53:40 JST pistolero

     in reply to
     Nope, just FediList: it's giving an empty response to the crawler but if I check the nodeinfo URL myself, it's fine. It's kind of amazing to watch.
     
     Farewell, a1pesticides.wordpress.com, pureadultsocialcare.blog, balconybabies2.wordpress.com, quatroagreengeneralconstruction.wordpress.com, wellnessinsightsbackyard.wordpress.com, all these goddamn spam blogs.
     wordpressocaust.png
   • Embed this notice
     pistolero (p@fsebugoutzone.org)'s status on Thursday, 27-Mar-2025 15:28:20 JST pistolero

     in reply to
     This is kind of interesting: the queue-clearing bit, basically if enough instances suddenly go down, it stops crawling. This was written to handle the case where Fedilist itself was broken: it stops crawling when that happens. It used to just stop completely for ten minutes whenever the state changes were all from up to down-ish (nxdomain, etc.). Especially when it was using a byzantine load-balanced proxy setup, a proxy would sputter and Fedilist would start marking all the sites as dead.
     
     That was basically a hack, so it's a little smarter now: passing the threshold for instances that changed state and the proportion that went down triggers a check, and we see
     
     reluctantmediumatlarge.wordpress.com, bethanyhomesforsale.com, these aren't the kind of servers you think of when you think of fedi: they technically speak ActivityPub but it's effectively a broadcast channel. I was trying to think of how to stop burning resources checking and rechecking WordPress sites. I built in a bit that checks less frequently if the information changes less frequently, for example. I hated doing this, but I even special-cased WordPress itself so that FediList would only check them every 60 minutes at minimum. The minimum frequency is around 15 minutes, but not a lot of sites are checked that frequently: the frequency is cranked down if the site responds slowly (so as not to swamp busy sites), if there are a lot of overdue sites, and if the site information doesn't change frequently. Sites that are down are checked even less frequently. (One exception is sites whose domains don't resolve: DNS queries are cheap and don't cause any load on the server we're looking at.)
     
     Special-casing WordPress didn't sit well with me, and neither did dropping WordPress, but there were also :over9k: over 9,000 WordPress sites that had made there way into FediList and FediList was basically wasting its time looking at WordPress "instances" because no one cares. But WordPress basically solved the problem for me. 6.7.2-RC1-59780 must have special-cased FediList, because that is the version number of the 4,149...4,157 instances that FediList just marked as "down".
     wordpress278dead.png
     wordpressdyingmore.png
     wordpress274dead.png
     wordpress265dead.png
      and ĐØⱠⱠ like this.
   • Embed this notice
     ĐØⱠⱠ (doll@decayable.ink)'s status on Thursday, 27-Mar-2025 15:34:27 JST ĐØⱠⱠ

     in reply to

     ...I wasn't really paying attention, I hope there's not a test.

     pistolero likes this.
   • Embed this notice
     pistolero (p@fsebugoutzone.org)'s status on Thursday, 27-Mar-2025 15:37:53 JST pistolero

     in reply to
     > That was basically a hack, so it's a little smarter now: passing the threshold for instances that changed state and the proportion that went down triggers a check, and we see
     
     @p writes an entire post without leaving a paragraph unfinished challenge (IMPOSSIBLE).
     
     Anyway, it does a check of all the requests and there are some bits where it could look for patterns but I didn't write any of that.
     
     > :over9k:
     
     @p doesn't fuck up an emoji challenge. :over9000:
     ✙ dcc :pedomustdie: :phear_slackware: likes this.
   • Embed this notice
     pistolero (p@fsebugoutzone.org)'s status on Thursday, 27-Mar-2025 15:38:46 JST pistolero

     in reply to

     • ĐØⱠⱠ
     @Doll
     youarenotexpectedtounderstandthis.png
     ĐØⱠⱠ likes this.
   • Embed this notice
     ✙ dcc :pedomustdie: :phear_slackware: (dcc@annihilation.social)'s status on Thursday, 27-Mar-2025 15:40:42 JST ✙ dcc :pedomustdie: :phear_slackware:

     in reply to
     @p :circus_values:
     pistolero likes this.
   • Embed this notice
     pistolero (p@fsebugoutzone.org)'s status on Thursday, 27-Mar-2025 16:08:24 JST pistolero

     in reply to
     Okay, looks like this isn't a software change, but a deployment thing maybe. Of the WordPress instances that went down since 05:00 (UTC), the only epidemic of dead happened in WordPress's IP space. Most of the other WordPress instances are still alive.
     Chinese man ? #nobot repeated this.
   • Embed this notice
     pistolero (p@fsebugoutzone.org)'s status on Thursday, 27-Mar-2025 16:51:49 JST pistolero

     in reply to
     So, despite the 4,261 instances representing 13.63% of the instances on FediList, the total user count on the front page only went down by 0.052% (9,791), and the post count only went down by 0.19% (3,251,533). Basically a pretty negligible impact on fedi, even if federating with them worked as normal.
     Chinese man ? #nobot likes this.
   • Embed this notice
     Chinese man ? #nobot (iamal_pharius@poa.st)'s status on Thursday, 27-Mar-2025 16:54:16 JST Chinese man ? #nobot

     in reply to
     @p Is poast in the 13%?
     
     Are we instance blocked by 50%
     pistolero likes this.
   • Embed this notice
     RedTechEngineer (redtechengineer@fedi.lowpassfilter.link)'s status on Thursday, 27-Mar-2025 17:08:40 JST RedTechEngineer

     in reply to

     • Chinese man ? #nobot
     @IAMAL_PHARIUS @p the 6 million WordPress instances
     Chinese man ? #nobot and pistolero like this.
   • Embed this notice
     pistolero (p@fsebugoutzone.org)'s status on Thursday, 27-Mar-2025 17:09:34 JST pistolero

     in reply to

     • Chinese man ? #nobot
     @IAMAL_PHARIUS I think more like by 3%.
   • Embed this notice
     pistolero (p@fsebugoutzone.org)'s status on Thursday, 27-Mar-2025 17:10:32 JST pistolero

     in reply to

     • Chinese man ? #nobot
     • RedTechEngineer
     @RedTechEngineer @IAMAL_PHARIUS If we include the instances that are down, it's 13,755.
     Chinese man ? #nobot likes this.
   • Embed this notice
     Chinese man ? #nobot (iamal_pharius@poa.st)'s status on Thursday, 27-Mar-2025 17:11:36 JST Chinese man ? #nobot

     in reply to
     @p I will do better
     pistolero likes this.
   • Embed this notice
     SilverDeth (silverdeth@fsebugoutzone.org)'s status on Friday, 28-Mar-2025 12:48:33 JST SilverDeth

     in reply to
     @p Maybe Wordpress's USAID money dryed up?
     pistolero likes this.
   • Embed this notice
     pistolero (p@fsebugoutzone.org)'s status on Friday, 28-Mar-2025 13:33:09 JST pistolero

     in reply to

     • SilverDeth
     @SilverDeth Ha, maybe they just noticed FediList. The crawler identifies itself pretty clearly but until they started including ActivityPub by default, it was just a handful of WordPress instances. When the Wordpocalypse happened and Wordpress ate fedi, suddenly FediList was sending off, like, one cycle per hour under ideal circumstances and 3 requests per cycle, so 12k reqs/hour on their managed-hosting instances, not counting attempts to fetch WordPress instances that still resolve but that no longer exist (though resurrections are rarely attempted if it's been dead for more than a week; it tops out at once a month). A bot that does a trickle, people don't notice, this would be maybe
     
     Here's the visualization of the Wordpocalypse; it's just the cumulative total of Wordpress instances, including the ones that went down, just a count over time of how many Wordpress instances Fedilist noticed. Huge jump in the fall of 2023, and it basically continues until summer 2024, at which point it calms way down (probably the switch got flipped on the same day for all of them and they just took a few months to federate) and the increase is probably just the rate at which Wordpress blogs are created multiplied by the likelihood that they federate (which we don't know but can probably figure out). (The big gap is :brucecampbell::callmesnake:. I fudged the numbers by skipping two days: the first day in FediList's data, and the day it came back after being down since early December, so the totals are off by about 1k.) The other graph is just the rate of increase, so you can see when there were big rollouts.
     wordpocalypse-rate.png
     wordpocalypse.png
     wordpocalypse.tsv