Embed Notice

HTML Code

<blockquote style="position: relative; padding-left: 55px;"><section><a href="https://saturation.social/users/clive/statuses/113878996590959757">Clive Thompson (clive@saturation.social)'s status on Friday, 24-Jan-2025 04:35:55 JST</a><a href="https://saturation.social/@clive" title="clive@saturation.social"><img src="https://gnusocial.jp/avatar/21777-48-20230219204149.webp" width="48" height="48" alt="Clive Thompson" style="position: absolute; left: 0; top: 0;">Clive Thompson</a><div><ul><li><li><a href="https://gnusocial.jp/user/174616" title="404mediaco@mastodon.social">404 Media</a></li></ul></div></section><article><p>A hacker developed an "infinite maze" to trap web-crawlers/scrapers from AI companies</p><p>basically, if the server code detects that a web crawler from an AI firm is trying to scrape the site ...</p><p>... the code begins spinning up an infinite, nesting warren of new sham pages, filled with random text</p><p>so the crawler gets stuck crawling and scraping endless and meaningless pages</p><p>fun <a href="https://mastodon.social/@jasonkoebler">@jasonkoebler</a> piece at <a href="https://mastodon.social/@404mediaco">@404mediaco</a> </p><p><a href="https://www.404media.co/email/7a39d947-4a4a-42bc-bbcf-3379f112c999/?ref=daily-stories-newsletter" rel="nofollow noreferrer">https://www.404media.co/email/7a39d947-4a4a-42bc-bbcf-3379f112c999/?ref=daily-stories-newsletter</a></p></article><footer><a rel="bookmark" href="https://gnusocial.jp/conversation/4434605#notice-8676484">In conversation</a><time datetime="2025-01-24T04:35:55+09:00" title="Friday, 24-Jan-2025 04:35:55 JST">about a month ago</time> <span>from <span><a href="https://saturation.social/@clive/113878996590959757" rel="external" title="Sent from saturation.social via ActivityPub">saturation.social</a></span></span><a href="https://saturation.social/@clive/113878996590959757">permalink</a><h4>Attachments</h4><ol><li><article><header><div>Domain not in remote thumbnail source whitelist: www.404media.co</div><h5><a href="https://www.404media.co/email/7a39d947-4a4a-42bc-bbcf-3379f112c999/?ref=daily-stories-newsletter">Developer Creates Infinite Maze That Traps AI Training Bots</a></h5><div> from <span>@jason_koebler</span></div></header><div>"Nepenthes generates random links that always point back to itself - the crawler downloads those new links. Nepenthes happily just returns more and more lists of links pointing back to itself."</div><footer></footer></article></li></ol></footer></blockquote>

Corresponding Notice

Embed this notice
Clive Thompson (clive@saturation.social)'s status on Friday, 24-Jan-2025 04:35:55 JSTClive Thompson
- Jason Koebler
- 404 Media
A hacker developed an "infinite maze" to trap web-crawlers/scrapers from AI companies
basically, if the server code detects that a web crawler from an AI firm is trying to scrape the site ...
... the code begins spinning up an infinite, nesting warren of new sham pages, filled with random text
so the crawler gets stuck crawling and scraping endless and meaningless pages
fun @jasonkoebler piece at @404mediaco
https://www.404media.co/email/7a39d947-4a4a-42bc-bbcf-3379f112c999/?ref=daily-stories-newsletter
In conversationabout a month ago from saturation.socialpermalink
Attachments
1. Domain not in remote thumbnail source whitelist: www.404media.co
  Developer Creates Infinite Maze That Traps AI Training Bots
  from @jason_koebler
  "Nepenthes generates random links that always point back to itself - the crawler downloads those new links. Nepenthes happily just returns more and more lists of links pointing back to itself."

Public

Embed Notice

HTML Code

Corresponding Notice