Comments on: Why you should have a robots.txt in your sites http://rauru.com/2008/04/why-you-should-have-a-robotstxt-in-your-sites/ Rauru.com is an Internet-related blog. Here I share anything I find unusual, weird, funny and/or interesting like: sites, new technologies, web development tips, css, design tricks, web advertisement, SEO, etc. Thu, 21 Aug 2008 17:42:54 +0000 http://wordpress.org/?v=2.6.1 By: Ikki http://rauru.com/2008/04/why-you-should-have-a-robotstxt-in-your-sites/#comment-249 Ikki Sun, 06 Jul 2008 13:21:14 +0000 http://rauru.com/?p=30#comment-249 @sunil: Great! Nice to know that you found my post useful ;) Thanks for your comments! @sunil: Great! Nice to know that you found my post useful ;) Thanks for your comments!

]]> By: sunil http://rauru.com/2008/04/why-you-should-have-a-robotstxt-in-your-sites/#comment-247 sunil Sun, 06 Jul 2008 07:22:02 +0000 http://rauru.com/?p=30#comment-247 hi dude, i setup my robots.txt after seeing ur article... i am const my own site and took to change robots.txt reading a well quoted article.. Kudos to you hi dude, i setup my robots.txt after seeing ur article… i am const my own site and took to change robots.txt reading a well quoted article..
Kudos to you

]]> By: Ikki http://rauru.com/2008/04/why-you-should-have-a-robotstxt-in-your-sites/#comment-93 Ikki Fri, 11 Apr 2008 19:31:04 +0000 http://rauru.com/?p=30#comment-93 <p>Hey Wakish ;) Thanks for sharing this! :D I've read your post and it's pretty good, too! Again, Thank you!</p> Hey Wakish ;)

Thanks for sharing this! :D I’ve read your post and it’s pretty good, too!

Again, Thank you!

]]> By: Wakish http://rauru.com/2008/04/why-you-should-have-a-robotstxt-in-your-sites/#comment-92 Wakish Fri, 11 Apr 2008 19:17:20 +0000 http://rauru.com/?p=30#comment-92 Hi ikki! It's a cool article.. I also wrote something about it last year and it could be an addition to what you wrote ;) Here's the link: http://wakish.info/robotstxt-why-it-this-simple-file-still-so-widely-used/ Cheers! - Wakish - Hi ikki! It’s a cool article..
I also wrote something about it last year and it could be an addition to what you wrote ;)
Here’s the link: http://wakish.info/robotstxt-why-it-this-simple-file-still-so-widely-used/

Cheers!
- Wakish -

]]> By: Google launches robot.txt generator - SEO & Internet Marketing Forums http://rauru.com/2008/04/why-you-should-have-a-robotstxt-in-your-sites/#comment-88 Google launches robot.txt generator - SEO & Internet Marketing Forums Fri, 11 Apr 2008 02:12:47 +0000 http://rauru.com/?p=30#comment-88 [...] to block them using .htaccess ;) For those who don't know about this robots.txt thing, read here: Why you should have a robots.txt in your sites | Rauru.com __________________ [...] [...] to block them using .htaccess ;) For those who don’t know about this robots.txt thing, read here: Why you should have a robots.txt in your sites | Rauru.com __________________ [...]

]]> By: Ikki http://rauru.com/2008/04/why-you-should-have-a-robotstxt-in-your-sites/#comment-87 Ikki Fri, 11 Apr 2008 01:59:32 +0000 http://rauru.com/?p=30#comment-87 <p>Hi there, I'm glad you liked my tutorial :) About your request, well no I haven't covered that. Maybe I will sometime! Thanks for the tip!</p> Hi there,

I’m glad you liked my tutorial :) About your request, well no I haven’t covered that. Maybe I will sometime! Thanks for the tip!

]]> By: Budhi http://rauru.com/2008/04/why-you-should-have-a-robotstxt-in-your-sites/#comment-86 Budhi Fri, 11 Apr 2008 00:42:27 +0000 http://rauru.com/?p=30#comment-86 This is exactly what I am looking for. Thanks a lot. I am also wondering about "honeypot" term, do you think you can cover it for your next post? (or is it already somewhere here? This is exactly what I am looking for. Thanks a lot.

I am also wondering about “honeypot” term, do you think you can cover it for your next post? (or is it already somewhere here?

]]>