GenAI bots are pushing websites into a corner that imperils open access, and perhaps worse, the web's historical record. From @gluejar:
https://go-to-hellman.blogspot.com/2025/03/ai-bots-are-destroying-open-access.html
Assuming that the web will continue to evolve instead of getting crushed underfoot, there is some interesting work going on over at the IETF about how to build on the now aged robots.txt protocol to allow rights holders to express how their content can be used online: