Embed Notice
HTML Code
Corresponding Notice
- Embed this notice@monsieuricon Throttling is not what you want - you want to temporarily null route such scrapers until they go away.
Too bad git and also apt is very bursty, meaning you cannot add a ratelimit sufficiently low to stop auto ratelimit adjusting scrapers, while also never blocking people who merely run `git clone` (I was wondering why git clone wasn't working until I realized that a high ratelimit wasn't high enough).
The only thing that seems to reveal those scrapers is the hours of time they spend connected scraping - so maybe a rate+connection time ratelimit could work - too bad you can pull of a lot of scraping in a few hours.