On post... The resource cannot be found.

Topics: ASP.NET 2.0
Mar 16, 2009 at 6:36 PM
Hi all,

I have an application that when I post a blog it is almost always available and then about 10-20 minutes later if you click on the post name it goes to a resource cannot be found server application error.  It seems like it always works for a few minutes and then it starts working again usually 6-7 hours later.  I have tried changing the web.config to many of the examples I have seen on this forum, but my issue seems to be a little different as it works for a little while and then starts failing for several hours and then starts working again. 

I am running under IIS 7 on a MOSSO cloud server.

Thanks for any help that can lead me in the right direction!

Pete
Coordinator
Mar 16, 2009 at 6:54 PM
I'm not familiar with a MOSSO cloud server, but it sounds like a type of web farm?

There's a strong chance the problem could be that BE is running on multiple machines within that environment.  The new posts are being saved in the data store (App_Data folder or Database) on all the machines, but the other instances of BE on these other machines aren't aware of the new blog post.  This is a common problem reported here by people in web farms.  Normally, BE doesn't check the data store to see if there are new posts since BE was designed to run as a single instance where people add posts through that single instance, and BE would know when a new post is created.

Probably when the post comes up for you, the machine in that environment serving your request is the same machine where you added the post.  But then when you later get the errors (404 errors?), it is a different machine in the environment serving your request.  And that other machine isn't aware the new post exists.

At least, this is my initial gut feeling ...
Mar 16, 2009 at 7:23 PM
Thanks for getting back to me Ben...  All of my data is saved in SQL Server and I was actually having this issue on my dedicated server a couple weeks ago before I moved to MOSSO (Rackspace cloud servers).  When I go directly to the be_Posts table through management studio they are all there and set to published.  These posts actually disappear from the blog list page for the time that the url stops working, leading me to believe there is an issue with dates and timezones of the posters or some other reason BlogEngine would not allow posts to be viewed.  Is there another propogation that BE does to allow for publishing besides just IsPublished in DB and the date time check?  I did, however, remove the check for times in my Post.cs file:

from:
//if (IsAuthenticated || IsPublished && DateCreated <= DateTime.Now.AddHours(BlogSettings.Instance.Timezone))
to:
if (IsPublished)

They still don't show up in the blogEngine lists even when I comment out the date created check even though they are set to published in the db.

I have several custom list views of my blog posts (most recent, top rated) that I use throughout my site outside of the blogging area that just check for isPublished and for the amount of time the posts aren't up in the blog area, that is the only way to get to them... and get the app error.

I feel like it could have something to do with a UrlRewrite issue, but can't figure it out. 

Thanks,
Pete
Coordinator
Mar 16, 2009 at 8:08 PM
The web farm problem I described would still be a problem if you are storing posts in SQL Server or the App_Data folder.  With this problem, even though the new posts are in the data store (SQL Server), the BE instance doesn't check the data store for new posts.  The BE instance will query the DB the first time, and then store those posts it retrieved in the server's memory (their cached).  While the posts are cached, the BE instance doesn't check the data store for any new posts.

You're right that the date/time check could cause posts to not show up, but assuming all these machines have the same time (and time zone), then that probably wouldn't be the problem in this case.

Depending on how busy your blog is, and also depending on how the server is setup and how much traffic it is getting, often times, the cached data will get cleared out after so many hours.  If other apps need more memory, or your blog has been inactive for a while, or there's a app pool restart, etc., then cached data is cleared out.  After the cached data is cleared out, the next time the data is needed, BE would re-query the database.

I'm guessing you might also have the same problem with any other cached data.  For example, comments.  If you comment on one of your blog posts, probably after that first 10 minutes or so, I'm guessing you won't see the new comment anymore.

Another reason the problem is what I'm describing and not with the UrlRewriter is because the new posts aren't showing up in the blog list page.  No url rewriting is involved for showing posts on the front page of your blog.

If this is the problem you're experiencing (I'm guessing it is), there currently is no elegant solution for this.  For the posts not showing up, one option is explicity calling Post.Reload() every so many minutes.  I have a code sample of this in this discussion (see March 4th message).  This should help the problem with posts.  But you will still probably have problems with other cached data (e.g. comments).

Another option is whenever you save a post, you update the date/time of the web.config file.  This should cause an app restart which will clear out any cached data on the servers.  I'm guessing this would help in the environment you're in, but this is definitely not an elegant solution and would only clear cached data out when you save a post.  It wouldn't clear out cached data when someone leaves a comment or in other situations ... although the same code below could be copied over to these other spots too.  In Add_Entry.aspx.cs, in the btnSave_Click event handler, right after post.Save(), you could add the new code below:

post.Save();

string appPath = HttpContext.Current.Request.PhysicalApplicationPath;

ThreadPool.QueueUserWorkItem(delegate {
    Thread.Sleep(8000);  // wait 8 seconds for other extensions to finish processing.
    string ConfigPath = appPath + "web.config";
    System.IO.File.SetLastWriteTimeUtc(ConfigPath, DateTime.UtcNow);
});
Mar 16, 2009 at 10:19 PM
Ben,

I will try the post from March 4 and see if the cache is the issue.  Is there any way to not use cache at all for BE?  I don't have tons of blogs on the site, but they are visible all over and I can't have my custom blog lists go to error pages.   I'm a little weary to clear cache for my entire site every 3,5, or 10 minutes, when the only issue that I'm seeing this on is the blog links from my custom lists. 

Thanks,
Pete
Coordinator
Mar 16, 2009 at 10:29 PM
Unfortunately, there's not a simple ON/OFF switch for the caching of posts, etc.

Another option would be to modify UrlRewrite and call Post.Reload() if it can't find the post when someone is requesting it.  This would be a more "on-demand" approach and more efficient than the timer method.  In UrlRewrite.cs, the RewritePost() method I have looks like:

private static void RewritePost(HttpContext context, string url)
{
    DateTime date = ExtractDate(context);
    string slug = ExtractTitle(context, url);
    Post post = Post.Posts.Find(delegate(Post p)
    {
        if (date != DateTime.MinValue && (p.DateCreated.Year != date.Year || p.DateCreated.Month != date.Month))
        {
            if (p.DateCreated.Day != 1 && p.DateCreated.Day != date.Day)
                return false;
        }

        return slug.Equals(Utils.RemoveIllegalCharacters(p.Slug), StringComparison.OrdinalIgnoreCase);
    });

    if (post != null)
    {
        context.RewritePath(Utils.RelativeWebRoot + "post.aspx?id=" + post.Id.ToString() + GetQueryString(context), false);
    }
}


Replace that code above with the following code, and then recompile the BlogEngire core.

private static void RewritePost(HttpContext context, string url)
{
    DateTime date = ExtractDate(context);
    string slug = ExtractTitle(context, url);
    Post post = FindPost(slug, date);

    if (post == null)
    {
        Post.Reload();
        post = FindPost(slug, date);
    }

    if (post != null)
    {
        context.RewritePath(Utils.RelativeWebRoot + "post.aspx?id=" + post.Id.ToString() + GetQueryString(context), false);
    }
}

private static Post FindPost(string slug, DateTime date)
{
    return Post.Posts.Find(delegate(Post p)
    {
        if (date != DateTime.MinValue && (p.DateCreated.Year != date.Year || p.DateCreated.Month != date.Month))
        {
            if (p.DateCreated.Day != 1 && p.DateCreated.Day != date.Day)
                return false;
        }

        return slug.Equals(Utils.RemoveIllegalCharacters(p.Slug), StringComparison.OrdinalIgnoreCase);
    });
}
Mar 16, 2009 at 10:40 PM
Ben,

I think that looks like exactly what I need.  I'll try that and thanks again for all the help!!!...  will check back tomorrow to let you know (and anybody else that has this issue) how this solution works.

Cheers,
Pete
May 10, 2011 at 3:48 PM
Edited May 10, 2011 at 3:48 PM

Hello everyone-specially BenAmanda! I know it has been a while since this post, but webfarming blogengine is still an issue :-) Ben Amanda, thank you for all your helpful posts which I believe I covered fully and benefited a lot from. I am using your webfarm extension 1 to detect new posts and comments and it looks like it is working..yay! Now, I am trying to do that for 'settings' by using your code from Mar 16 2009 above (writing time stamp to web.config). It seemed to work when I set up multiple subdirectories on IIS for the blog to work as different webclusters (for testing purposes only). when i tried to create a new parent directory (on the same box) it did not restart the blog as I modified settings from another parent directory and thus did not really detect the changes. any ideas of why is this happening?

My second question which maybe could explain the first one too is: how can this method work? Can you explain why writing timestamp in one place to web.config could force applications running in different places to restart their app as well since they have thier own separate web.config(s)? Your response is much appreciated and thank you already for all your other posts.

Coordinator
May 11, 2011 at 7:01 AM
Edited May 11, 2011 at 7:02 AM

Hi.  For your 2nd question, you're right that updating the timestamp on a web.config file in one directory won't force applications running in different places to restart their apps.  I don't remember saying that doing that would force applications in other places to restart their apps (my memory could be failing me as well).

I'm really not familiar with this topic, but in some setups, I think when you update a file in one server (the primary web server), that changed file gets automatically mirrored to the other, slave locations?  If this is the case, then updating a web.config file in the primary location would probably lead to the updated web.config file being propagated to the other servers, and that seems like that would lead to an app restart on those other servers.

I'm not exactly sure what you're asking in your first question ... is the new parent directory another mirrored copy of your blog?  Sorry, I know I'm not even getting the terminology right.  But creating a new directory sounds like a non-recurring situation where you are doing this manually.  If that's the case, I personally wouldn't be concerned with this type of non-daily update not being detected.  It's the daily updates of posting blog entries, adding comments, etc that you of course want to be propagated to the other application instances to prevent stale caches.

The idea behind the Webfarm extension is that when data gets updated in one instance, it notifies the other instances that data has changed.  The other instances can use that information/notification by clearing their caches.  I think the Webfarm extension uses Post.Reload() or something similar to clear out the post data.  For settings changes, etc., the Webfarm extension could be used to notify the other instances the same way it does for updated posts, and those other instances can then each update their own web.config file's timestamp to clear the cache.

May 11, 2011 at 2:39 PM

Hello and thank you for your reply. In your post on Mar 16 2009 quote"Another option is whenever you save a post, you update the date/time of the web.config file.  This should cause an app restart which will clear out any cached data on the servers"  and that is where I got that writing the timestamps to web.config would restart the apps. That did work for the different blog applications running under the same app pool but not with blogs running under different parent app pool which makes sense. 

I am happy to announce now that we have a fully functional blogengine.net application running on different servers but working correctly. To accomplish that we added more event handlers for every single function one would do with the blog whether you are admin or user and currently it is working with no problems (using your web farm extension). thank you very much for the clarifications. If anyone needs help with applying the extension to cover more aspects of the blog I'd be more than happy to help! Good day all!