[wp-trac] [WordPress Trac] #15647: robots.txt not correctly changing between allow and disallow

[WordPress Trac]

#15647: robots.txt not correctly changing between allow and disallow
---------------------------+------------------------------------------------
 Reporter:  ipstenu        |        Owner:            
     Type:  defect (bug)   |       Status:  closed    
 Priority:  normal         |    Milestone:            
Component:  Accessibility  |      Version:            
 Severity:  major          |   Resolution:  worksforme
 Keywords:                 |  
---------------------------+------------------------------------------------

Comment(by ipstenu):

 It may just be becuase Google's a freakin' dink.  I read through their
 webmaster whoopla, and it LOOKS like they're giving weighted preference to
 allow vs disallow. So while both are, technically, correct, they won't
 always scan a {{{Disallow: (nothing)}}}.

 I'm playing around with their webmaster tools, and seeing different
 results with 'fake' robots.txt files when I set it as disallow nothing or
 allow everything.

-- 
Ticket URL: <http://core.trac.wordpress.org/ticket/15647#comment:2>
WordPress Trac <http://core.trac.wordpress.org/>
WordPress blogging software