I've been helping some friends and colleagues block some of the site scraping bots that are feeding "AI" models. Decided to take some of my notes and make something others could use too. It's a work-in-progress. Happy to add to or correct things.
https://neil-clarke.com/block-the-bots-that-feed-ai-models-by-scraping-your-website/
Conversation
Notices
-
Embed this notice
Clarkesworld (clarkesworld@mastodon.online)'s status on Thursday, 24-Aug-2023 21:19:54 JST Clarkesworld -
Embed this notice
pettter (pettter@mastodon.acc.umu.se)'s status on Thursday, 24-Aug-2023 21:26:02 JST pettter @gregeganSF @clarkesworld Works For Me, fwiw. Maybe just a reload will fix it?
-
Embed this notice
Greg Egan (gregegansf@mathstodon.xyz)'s status on Thursday, 24-Aug-2023 21:26:04 JST Greg Egan @clarkesworld This link gives me: “Error establishing a database connection”.
-
Embed this notice