Conversation

Notices

1. Embed this notice
   dave (dthompson@toot.cat)'s status on Friday, 03-Jan-2025 08:03:48 JST dave

   AI bots are scraping my git server and slowing things down I hate the future so much leave me alone no one uses scheme anyway you're wasting your time

   • Darnell Clayton :verified: repeated this.
   • Embed this notice
     dave (dthompson@toot.cat)'s status on Friday, 03-Jan-2025 11:52:20 JST dave

     in reply to

     I used https://github.com/ai-robots-txt/ai.robots.txt/blob/main/robots.json to generate some nginx config to block based on user agent and that did the trick for now. fuck off fuckers!

   • Embed this notice
     dave (dthompson@toot.cat)'s status on Friday, 03-Jan-2025 21:37:47 JST dave

     in reply to

     woke up this morning
     checked the server logs

     more bots! blocked another user agent but there's at least one other that is spoofing their user agent. fuckers!

   • Embed this notice
     dave (dthompson@toot.cat)'s status on Friday, 03-Jan-2025 21:55:32 JST dave

     in reply to

     the remaining 200 requests that I suspect are bots are very low volume, though, so I'm mostly happy for now. also added a /robots.txt in case that helps any polite bots not send requests in the first place.

   • Embed this notice
     dave (dthompson@toot.cat)'s status on Friday, 03-Jan-2025 22:38:33 JST dave

     in reply to

     • Andres Moreno

     @monkey1 I use a vps on digital ocean. not sure how they found me, exactly, but links to my software are out there enough that it's easy to find. this could have been happening for a long time and I'm only noticing now. not sure, really.

   • Embed this notice
     Andres Moreno (monkey1@fosstodon.org)'s status on Friday, 03-Jan-2025 22:38:34 JST Andres Moreno

     in reply to

     @dthompson

     What's your setup? Do you have a VPS or do you run your own machine?

     The real question is: are they finding you because your IP is in a known range or are they just systematically trying every possible IP?

   • Embed this notice
     dave (dthompson@toot.cat)'s status on Saturday, 04-Jan-2025 01:55:25 JST dave

     in reply to

     • Kaito

     @kai I had the sopranos theme in mind

   • Embed this notice
     Kaito (kai@ajin.la)'s status on Saturday, 04-Jan-2025 01:55:28 JST Kaito

     in reply to

     @dthompson (in case it's not obvious, "woke up this morning" always gets followed by the bad to the bone riff…)

   • Embed this notice
     Kaito (kai@ajin.la)'s status on Saturday, 04-Jan-2025 01:55:31 JST Kaito

     in reply to

     @dthompson 🎼 🎸