Hey, all. Very interesting. So, I've mentioned this a few times in the comments, but I've done a survey of about 85,000 servers that have been seen by existing live servers. Of those servers, about 32% reply successfully to a Nodeinfo request. Nodeinfo is a per-server statistics endpoint.
Now, there's some dark matter out there: servers that never connected to our large connected component on the Fediverse. We can't know if they're up or down or what. They're unreachable.
So, by my estimate, of these servers, about 32-40% of Fediverse servers seen by other live servers are alive today. The rest return errors, don't resolve, have bad SSL certs, and so on.
Not all ActivityPub servers support Nodeinfo. Of that same data set, about 8% give a successful 404 or 410 response for Nodeinfo. Some percentage of those are going to be non-nodeinfo Fediverse servers, and with a bit more work I could probably figure out which are which.
That's interesting. I figured there'd be a LONG tail of servers that someone spun up, ran for a few days, and then shut down.
But maybe there are, but live servers wouldn't remember them?
It seems like it would be a good design for them to eventually forget inactive accounts?
I know that some accounts are listed by my server as "inactive", simply because no one on my server follows them anymore (but following back to the origin server, they are still there and active).
- 27K live servers. - 47% have one user. - 55% of all users are on the top 20 servers. - 88% of all users are on the top 100 servers. - 99% of all users are on the top 1000 servers.
@ricci So, my crawler has been running for 5 years, and it also pulls data from other sources like the-federation.info, fediverse.observer, fedidb.com etc. Its database has 44142 alive nodes and 777384 dead ones, so the answer is 5.4%. If .gab.best and .cheeseland.xyz are excluded, then only 98621 nodes are dead and the answer is 30.9%.
Someone in the thread suggested that nodes might be pruning their peers lists from dead nodes. Judging by the negligible difference between Evan's number and my number, no pruning is done.
BTW, my crawler currently tracks 22 newly discovered nodes, i.e. ones that were seen in someone's peers list but whose NodeInfo hasn't been checked yet. I'm happy the Fediverse keeps growing _^
@liaizon@evan well... what I'm doing is using a list of servers that are currently up, and just getting nodeinfo for them, so I do have a list of *some* historical servers that are gone now, but I doubt it's anywhere near as good as Evan's data.
Someone who might have good data is @minoru who has been crawling the lists of federated servers for several years now to make https://nodes.fediverse.party/ (the list I use), and would have probably have a bunch of no-longer-reachable nodes in his database, including ones that were only known by nodes that are themselves no longer up, etc.
@evan i would love a breakdown on the one user servers if you did a followup to this. are these actual servers in actual use, or developer test servers?