Embed Notice

HTML Code

<blockquote style="position: relative; padding-left: 55px;"><section><a href="https://tldr.nettime.org/users/w0bb1t/statuses/113833706094624250">mr.w0bb1t (w0bb1t@tldr.nettime.org)'s status on Thursday, 16-Jan-2025 16:20:56 JST</a><a href="https://tldr.nettime.org/@w0bb1t" title="w0bb1t@tldr.nettime.org"><img src="https://gnusocial.jp/avatar/134984-48-20240801021950.webp" width="48" height="48" alt="mr.w0bb1t" style="position: absolute; left: 0; top: 0;">mr.w0bb1t</a></section><article><p><a href="https://tldr.nettime.org/tags/LLM" rel="tag">#LLM</a> scrapers are relentless and brutual [...] This is a tarpit intended to catch web crawlers. Specifically, it's targetting crawlers that scrape data for LLM's .. </p><p>👉🏻 <a href="https://zadzmo.org/code/nepenthes/" rel="nofollow noreferrer">https://zadzmo.org/code/nepenthes/</a></p></article><footer><a rel="bookmark" href="https://gnusocial.jp/conversation/4384069#notice-8574720">In conversation</a><time datetime="2025-01-16T16:20:56+09:00" title="Thursday, 16-Jan-2025 16:20:56 JST">about 3 months ago</time> <span>from <span><a href="https://tldr.nettime.org/@w0bb1t/113833706094624250" rel="external" title="Sent from tldr.nettime.org via ActivityPub">tldr.nettime.org</a></span></span><a href="https://tldr.nettime.org/@w0bb1t/113833706094624250">permalink</a><h4>Attachments</h4><ol><li><article><header><div>No result found on File_thumbnail lookup.</div><h5><a href="https://zadzmo.org/code/nepenthes/">ZADZMO code</a></h5><div> from <span>https://zadzmo.org/humans.txt</span></div></header><div></div><footer></footer></article></li><li><label><a rel="external" href="https://gnusocial.jp/attachment/3927679">NepenthesThis is a tarpit intended to catch web crawlers. Specifically, it's targetting crawlers that scrape data for LLM's - but really, like the plants it is named after, it'll eat just about anything that finds it's way inside.It works by generating an endless sequences of pages, each of which with dozens of links, that simply go back into a the tarpit. Pages are randomly generated, but in a deterministic way, causing them to appear to be flat files that never change. Intentional delay is added to prevent crawlers from bogging down your server, in addition to wasting their time. Lastly, optional Markov-babble can be added to the pages, to give the crawlers something to scrape up and train their LLMs on, hopefully accelerating model collapse.</a></label><br><a href="https://tldr.nettime.org/system_nettime/media_attachments/files/113/833/704/293/845/539/original/3aab29b674763a33.png" rel="external">https://tldr.nettime.org/system_nettime/media_attachments/files/113/833/704/293/845/539/original/3aab29b674763a33.png</a></li></ol></footer></blockquote>

Corresponding Notice

Embed this notice
mr.w0bb1t (w0bb1t@tldr.nettime.org)'s status on Thursday, 16-Jan-2025 16:20:56 JST mr.w0bb1t
#LLM scrapers are relentless and brutual [...] This is a tarpit intended to catch web crawlers. Specifically, it's targetting crawlers that scrape data for LLM's ..
👉🏻 https://zadzmo.org/code/nepenthes/
In conversationabout 3 months ago from tldr.nettime.orgpermalink
Attachments
1. No result found on File_thumbnail lookup.
  ZADZMO code
  from https://zadzmo.org/humans.txt
2. Nepenthes This is a tarpit intended to catch web crawlers. Specifically, it's targetting crawlers that scrape data for LLM's - but really, like the plants it is named after, it'll eat just about anything that finds it's way inside. It works by generating an endless sequences of pages, each of which with dozens of links, that simply go back into a the tarpit. Pages are randomly generated, but in a deterministic way, causing them to appear to be flat files that never change. Intentional delay is added to prevent crawlers from bogging down your server, in addition to wasting their time. Lastly, optional Markov-babble can be added to the pages, to give the crawlers something to scrape up and train their LLMs on, hopefully accelerating model collapse.
  https://tldr.nettime.org/system_nettime/media_attachments/files/113/833/704/293/845/539/original/3aab29b674763a33.png

Public

Embed Notice

HTML Code

Corresponding Notice