Conversation

Notices

1. Embed this notice
   Brodie Robertson (brodieonlinux@mstdn.social)'s status on Sunday, 23-Mar-2025 07:52:06 JST Brodie Robertson

   We've entered the Era of the AI wars, AI scrapers are openly ignoring robots.txt and other measures designed to tell bot website visitors to behave in a certain way.

   If that's how it's going to be let's get into an arms race then. There's open source tools like Anubis, Cloudflare announced AI Labyrinth. This won't be the end of AI scrapers, they will get better and so will these tools and so many resources will be wasted on a problem that was already resolved if the bots worked properly

   • Embed this notice
     Brodie Robertson (brodieonlinux@mstdn.social)'s status on Sunday, 23-Mar-2025 09:00:35 JST Brodie Robertson

     in reply to

     • Ritchie

     @acidiclight I think you missed the 2nd paragraph lol

   • Embed this notice
     Ritchie (acidiclight@mastodon.social)'s status on Sunday, 23-Mar-2025 09:00:36 JST Ritchie

     in reply to

     @BrodieOnLinux Go look into AI Labyrinth by Cloudflare. It's the exact kind of arms race you're thinking of.