Conversation

Notices

1. Embed this notice
    (mint@ryona.agency)'s status on Sunday, 11-May-2025 13:12:21 JST 
   Dollar store sneedflare appears to only work with browser useragents.
   https://gitlab.com/zipdox/anubis-bypass
   Screenshot_20250511_070955.png
   • lainy, ✙ dcc :pedomustdie: :phear_slackware: and Phantasm and 2 others like this.
   • Johnny Peligro, nyanide :nyancat_rainbow::nyancat_body::nyancat_face: and pwm repeated this.
   • Embed this notice
      (mint@ryona.agency)'s status on Sunday, 11-May-2025 13:13:05 JST 

     in reply to
     Yup, works on the archwiki.
     Screenshot_20250511_071249.png
     nyanide :nyancat_rainbow::nyancat_body::nyancat_face: likes this.
   • Embed this notice
      (mint@ryona.agency)'s status on Sunday, 11-May-2025 13:14:32 JST 

     in reply to

     • Jelqer Skelter
     @sneeden Yes.
   • Embed this notice
     Jelqer Skelter (sneeden@0.5dollah.click)'s status on Sunday, 11-May-2025 13:14:33 JST Jelqer Skelter

     in reply to

     @mint That's the anti-AI scraping tool made by DeVault-adjacent people right?

     Phantasm and Johnny Peligro like this.
   • Embed this notice
     T man :sex: :puffgiga: :puffpowerroll: (theorytoe@ak.kyaruc.moe)'s status on Sunday, 11-May-2025 13:17:11 JST T man :sex: :puffgiga: :puffpowerroll:

     in reply to
     @mint
     color me surprised that mentally comatose devs create retarded software
      and Phantasm like this.
   • Embed this notice
     Matt Hamilton (eriner@noauthority.social)'s status on Sunday, 11-May-2025 13:19:43 JST Matt Hamilton

     in reply to

     @mint "acktchually we only want to inconvenience casual browsers, not bots"

     , ✙ dcc :pedomustdie: :phear_slackware: and Phantasm like this.
   • Embed this notice
     butterbutt (butterbutt@social.thisisjoes.site)'s status on Sunday, 11-May-2025 15:31:39 JST butterbutt

     in reply to

     @mint the dev's blog post announcing anubis makes it pretty clear they are retarded
     >If the client has a User-Agent that does not contain "Mozilla", the client is allowed through.
     so all the scrapers just using curl or aiohttp or axios or whatever keep hammering your site. when anthropic first started scraping they just sent "claudebot" as the user agent

      and Phantasm like this.
   • Embed this notice
     lainy (lain@lain.com)'s status on Sunday, 11-May-2025 15:43:48 JST lainy

     in reply to
     @mint
     > you can circumvent the bot defense by admitting you're a bot
     Phantasm, Johnny Peligro and pwm like this.
   • Embed this notice
     Cocoa Hoto in real life (cocoa@nekosat.work)'s status on Monday, 12-May-2025 03:41:58 JST Cocoa Hoto in real life

     in reply to
     @mint contributor covenant moment
     Phantasm likes this.
   • Embed this notice
     wuhan.bat™ (jae@darkdork.dev)'s status on Tuesday, 13-May-2025 12:46:26 JST wuhan.bat™

     in reply to
     @mint i'd say this is a clever way to hop the fence, but the author really didn't think this through. much like how the author didn't think nix/nixos through, then kubernetes, then golang, instead decided to continue focus on vtubing and shilling their open-source anubis project. i hear some people made their own hashcah implementation with better featureset. this one guy made one davis in it. you may know him :niggawink:
      likes this.
   • Embed this notice
      (mint@ryona.agency)'s status on Tuesday, 20-May-2025 18:00:20 JST 

     in reply to
     BUT WAIT, IT GETS BETTER
     image.png
     1746962069094.png
     nyanide :nyancat_rainbow::nyancat_body::nyancat_face: likes this.
   • Embed this notice
     menherahair (menherahair@eientei.org)'s status on Tuesday, 20-May-2025 18:58:26 JST menherahair

     in reply to
     @mint to be fair the idea was probably, drop (in reality redirect to communist manifesto) all connections from know scraper user agents yourself, and let anubis deal with the other stuff they don't teach about in http 101
      likes this.
   • Embed this notice
     𝅙𝅙𝅙𝅙𝅙𝅙𝅙𝅙 (sally@freesoftwareextremist.com)'s status on Tuesday, 20-May-2025 19:12:47 JST 𝅙𝅙𝅙𝅙𝅙𝅙𝅙𝅙

     in reply to
     @mint
     
     So all this javashit blocker degeneracy was never about stopping AI scrapers but about stopping competition and also annoy the shit out of free software enjoyers that don't run arbitrary code. It's always the same shtick with this corporate scum, next thing that'll show up is that OpenAI funded Anubis as the cherry on top.
      likes this.
   • Embed this notice
      (mint@ryona.agency)'s status on Tuesday, 20-May-2025 19:55:43 JST 

     in reply to

     • 𝅙𝅙𝅙𝅙𝅙𝅙𝅙𝅙
     @sally Wanted to try archwiki, but it appears they've relaxed it as even a regular browser useragent lets me in (it appears to still be running it though as bypass extension still gets triggered).
   • Embed this notice
     𝅙𝅙𝅙𝅙𝅙𝅙𝅙𝅙 (sally@freesoftwareextremist.com)'s status on Tuesday, 20-May-2025 19:55:44 JST 𝅙𝅙𝅙𝅙𝅙𝅙𝅙𝅙

     in reply to
     @mint
     
     This seems to be a GNOME thing specifically. Did you try other repos and sites?
   • Embed this notice
     Kirino Kousaka (kirino@shitpost.cloud)'s status on Tuesday, 20-May-2025 21:57:30 JST Kirino Kousaka

     in reply to
     DEFINITELY DON'T POST A LIST OF USER AGENTS THAT ARE NOT BROWSER ONES!!! THAT WOULD BE SUPER DUPER BAD!
      likes this.
   • Embed this notice
     Dumb Idiot Retard (idiot@shitposter.world)'s status on Tuesday, 20-May-2025 22:01:14 JST Dumb Idiot Retard

     in reply to
     @mint apparently that’s the chat gippity search mode and not the scraper. As if proprietards are going to let on the difference. But they want the slopalisk to be able to search archwiki when it’s vomiting.
      likes this.
   • Embed this notice
     Kirino Kousaka (kirino@shitpost.cloud)'s status on Tuesday, 20-May-2025 22:01:21 JST Kirino Kousaka

     in reply to

     • butterbutt
     Ohhh that changes things!
     
     Definitely do not post a list of user-agents NOT containing variants of the letters "moz"! That would be SUPER DUPER BAD!!!
      likes this.
   • Embed this notice
     Daniel Stevens (eris@p.enes.lv)'s status on Tuesday, 20-May-2025 22:26:56 JST Daniel Stevens

     in reply to

     • lainy
     If your website actually has a bot scraper problem you probably already have a rule that heavily rate limits non-browser UAs. Whether sites that use Anubis actually do that is another question.
     
     CC: @mint@ryona.agency
     
     lainy likes this.
   • Embed this notice
     翠星石 (suiseiseki@freesoftwareextremist.com)'s status on Wednesday, 21-May-2025 15:12:41 JST 翠星石

     in reply to

     • lainy
     • Daniel Stevens
     @eris @lain @mint Aggressive LLM scrapers tend to swap to browser useragents after the scraper useragent gets 403'd.
     
     Such useragent usually has "AppleWebKit" in it and nobody is going to complain if you attack isheep with more proprietary software, but alas they attack GNU icecat users etc instead.