@lunarised I still think they should just put a macro on /bottrap/ or whatever that just returns a bunch of internal links at random and link that at the start of the homepage, then disallow it in robots.txt. Any compliant bot will avoid the trap, the rest can suffer.