@feld @tdp_org There is probably a combination of both. I wasn’t the one making a statement about the current state of this.
It’s easy to verify if the Googlebot user-agent is authentic, so I don’t think spoofing of that is widespread. The worst scraping I have personally seen originated from Amazon IP ranges and as far as I recall the user-agent also indicated that it was Amazon.