Conversation

Notices

1. Embed this notice
   Perfect Devil Anna (amalthea@pl.noob.quest)'s status on Tuesday, 18-Feb-2025 02:49:23 JST Perfect Devil Anna
   does ryona agency fba respect robots.txt? has anyone looked into this? 🤔
   • Embed this notice
     Johnny Peligro (mischievoustomato@tsundere.love)'s status on Tuesday, 18-Feb-2025 02:49:21 JST Johnny Peligro

     in reply to

     • 
     • Protoss
     @Cyrillic @amalthea @mint disrespecting robots.txt is kino
      likes this.
   • Embed this notice
     Protoss (cyrillic@lab.nyanide.com)'s status on Tuesday, 18-Feb-2025 02:49:22 JST Protoss

     in reply to

     • 
     @amalthea I doubt it, @mint was also notorious for making private posts that federated visible
   • Embed this notice
      (mint@ryona.agency)'s status on Tuesday, 18-Feb-2025 02:51:27 JST 

     in reply to

     • Protoss
     • Johnny Peligro
     @mischievoustomato @Cyrillic @amalthea It's only there to control crawling (recursively following links), FBA simply looks up the predefined API endpoints. Same reason why web.archive.org respects robots.txt but archive.fo (which only saves single pages upon user request) doesn't.
   • Embed this notice
     pomstan (pomstan@xn--p1abe3d.xn--80asehdb)'s status on Tuesday, 18-Feb-2025 13:33:34 JST pomstan

     in reply to

     • 
     • Protoss

     @Cyrillic @amalthea @mint lockfags deserve hanging

      likes this.