The robots.txt file on any site is how search engines discover what a site wants indexed and what it doesn’t.
Generally you won’t find many people checking the files unless they’re technically minded, passionate about SEO and curious. The Daily Mail of all sites thought it would be an ingenious idea to place a job advert in the file and SEO consultant Malcolm Coles discovered it.
This is most likely the work of The Mail Online’s MD James Bromley.
The 💜 of EU tech
The latest rumblings from the EU tech scene, a story from our wise ol' founder Boris, and some questionable AI art. It's free, every week, in your inbox. Sign up now!
I’m rather impressed…The Daily Mail thinking well outside of the box.
Get the TNW newsletter
Get the most important tech news in your inbox each week.