Wow, I didn't even know that existed. Much cleaner way of doing it! -- Alex On Sun, Oct 24, 2010 at 2:56 PM, Jeremy Visser <jeremy at visser.name> wrote: > (Yes, you'll need to hook into do_robotstxt to add that line, but that's > left as an exercise to the reader.) >