Public
- Public
- Network
- Groups
- Featured
- Popular
- People

By blocking these crawlers, bandwidth for our downloaded files has decreased by 75% (~800GB/day to ~200GB/day). If all this traffic hit our origin servers, it would cost around $50/day, or $1,500/month, along with the increased load on our servers.

Download link

By blocking these crawlers, bandwidth for our downloaded files has decreased by 75% (~800GB/day to ~200GB/day). If all this traffic hit our origin servers, it would cost around $50/day, or $1,500/month, along with the increased load on our servers.
https://cdn.masto.host/frontendsocial/media_attachments/files/112/863/640/389/656/380/original/b94b82d83e3b9844.png

Notices where this attachment appears

Embed this notice
Stefan Judis (stefan@front-end.social)'s status on Saturday, 03-Aug-2024 22:38:06 JST Stefan Judis

This is pretty wild — our friends at "Read The Docs" saw file download traffic drop by 75% after blocking AI bots.
https://about.readthedocs.com/blog/2024/07/ai-crawlers-abuse/

In conversation about 6 months ago from front-end.social permalink