ok... it looks like slurp is obeying the robots.txt file now... but it looks like it took about 7 hours to take effect.
Labels: website
|
|
|
|
|
Frankie's Blog
4.11.2008ok... it looks like slurp is obeying the robots.txt file now... but it looks like it took about 7 hours to take effect. Labels: website
Last Modified: 04.11.08
|
0 Comments:
Post a Comment
<< Home