Aaaaaaaggghhh! Google!! in Site Updates

  • Oct. 18, 2013, 12:27 p.m.
  • |
  • Public

I've noticed some recent concern about Google indexing public site content.

(I, for one, welcome our not-so-new search overlords)

To address this concern, I've created a robots.txt file. My understanding is that Google and reputable search engines will honor this, and not index URLs that begin with:

  • /box/
  • /book/
  • /entry/
  • /search/

Basically, any URLs I could think of that may provide user-specific data in search results. So hopefully that solves it and puts people at ease. Obviously the effects won't be instantaneous; Google will have to crawl the site again and cycle out their current content. So we shall see.


Loading comments...

You must be logged in to comment. Please sign in or join Prosebox to leave a comment.