Courtesy of Last.fm, h/t Nick Halstead.
Quite possibly the best Robots.txt file ever.
27th July 2010 by Zee
Shareables
Part of The Next Web family
Trackbacks
-
[...] SHARETHIS_ary.push(SHARETHIS_post); Asimov would be proud.Source:http://thenextweb.com/shareables/2010/07/27/quite-possibly-the-best-robots-txt-file-ever/ var stLink = jQuery('a:last')[0]; stLink.href = 'javascript:void(0)'; SHARETHIS_post = [...]
-
=== popurls.com === popular today…
yeah! this story has entered the popular today section on popurls.com…
-
[...] Quite possibly the best Robots.txt file ever. (tags: funny humor awesome robots) [...]
-
[...] Quite possibly the best Robots.txt file ever. – deffo up there [...]
Yeah, but shouldn’t those Laws be at the *beginning* of the file? :-)
mnmbnmbn
as seen on reddit.com a few days ago
You are a retard. It originated on last.fm, even said by your beloved reddit.com. You are trying wayyy to hard to be the biggest troll under the bridge. Sit down, fool.
Who the fuck are you?
internet hard asses, obviously
Your split personality. We are all anonymous
I’m Anonymous too! WTF?
ROTFL, no way dude that is just way too cool
http://www.anonymous-surfing.es.tc
Wow, if only something like that could work….
it does
lol a classic! Will be adding that to all future robots.txt files I create.. can’t be too careful. ;-P
Awesome
I want that robot txt .. Is Google smart enough to read that?
Nice!
Robots.txt is quite important.
Fantastic! :D