Embed Notice
HTML Code
Corresponding Notice
- Embed this notice@mactonite scraping is where someone downloads all the pages from a website in bulk as fast as line speed allows (without following any kind of rate limit) and then scrapes the HTML (or even the JavaScript data sometimes) to extract the wanted information.
I remember correctly now that you get a 403 if the IP you're connecting from is being used for scraping and not 504.
504 is gateway timeout - I guess the gateway sometimes times out, but a page refresh a while later fixes it.