@ruud @dumpsterqueer Done!I'm overhauling the crawler to better handle downtime or failed requests (while honoring robots.txt rules)https://fedidb.org/network/instance/friendica.world