Post & Page Securty & Search

Mar 8, 2010 at 5:25 PM

I am trying to implement page & post level security. I have reviewed these posts

and understand the logic behind these but I have one nagging question.

How can you get search engines to not find this content?  Is there a way to do this by using a robot.txt file?



Mar 9, 2010 at 1:21 AM

Adding post.aspx and page.aspx to the robots.txt might do it.

User-agent: *
Disallow: /post.aspx
Disallow: /page.aspx
Mar 10, 2010 at 4:58 PM

So after a bit more research I figured this,

1)  If I make the modifications to only show a blog category if the user is logged on and in a certain role,

2) Without being logged on a search bot is never going to see a href to my secure posts.

Does that sound right ?





Mar 10, 2010 at 5:20 PM

If you hide the category link from users who are not logged in, the crawler will not see that link.

You need to keep in mind that there are several ways that a bot can get to your post, such as the "archive", "sitemap", "tags", etc.

There really are two things that you should do to hide your page from unauthenticated users:

  1. On the actual page, verify that the user has permission to view it, and if they do not, redirect them somewhere else such as a login page, error page, or home page.
  2. Hide all of the links to that page when the user is not logged in. 
Mar 11, 2010 at 2:43 PM

Thanks for all the input so far; I am using the PostSecurity extension as modified by JP. This allows me to set an internal secure category that you must be logged in to view. It works great..

The next issues are these

1) I am using a category list widget. I only want this to show up if the user has see the posts.

2) If you scroll through the posts , and the next post is "Secure Post", the post serving function Redirects to the error404 page

3) If you use the internal blog search it does find the post and gives a preview of the post.

I am not an accomplished c# & .net developer so before I start to hack the relevant code to solve these, does anyone have any elegant solutions for this.

Thanks in advance



Mar 21, 2010 at 10:43 PM
I use the user control for securing dynamic pages. Like fstanek points out, it redirects the unauthorized user to the login page. For 2. - I have created a list control that filters the list for the given user. Have you found anyway to filter the search?
Mar 22, 2010 at 11:12 AM

I have not. I am only assuming that as you need to logon to view these pages, search engines will not index these