Application Pool Exceeds 1 GB on v1.6 after 20 minutes

Topics: ASP.NET 2.0
Mar 7, 2010 at 10:48 PM

I finally upgraded my web site from BlogEngine 1.3 to 1.6 (I know, a bit behind the times)
With v1.3 my app pool would typically sit around 600 mb with around 20-30 % cpu usage (blogengine compression turned on)

When I upgraded to 1.6 the memory shoots up to 1.1 GB before it recycles (I have it set to recycle at 1.1 GB, any higher machine will crawl) after about 20 minutes
CPU usage is around 70-80%
If I turn off compression in admin.  CPU usage drops to around 50% and memory rises slower but still recycles around 45 minutes later.

So... Is v1.6. caching more pages and I need a server with more memory?  Will the memory usage balance out at some point?  If so at what level do you think?

Site is a dedicated server with approx:
35K visitors a day
180k Page views a day

Thanks to all!

Steven

Coordinator
Mar 8, 2010 at 4:41 AM

Do you have the Referrer Tracking turned on?  This could be using memory, so if it's on, you can try turning it off.  If you turn it off, I believe you need to restart the blog (recycling the app pool works) for the change to be detected.

Although probably not very likely, the Twitter widget or Blogroll could possibly eat up some memory.

If you have any other custom widgets or Extensions, I would try turning them off to see if it makes a difference.

Mar 8, 2010 at 6:57 PM

Thanks for the suggestions.

Referrer tracking is turned off
Not using the Twitter and the blogroll widget has only 1 item and images/links
My widgets used are:

Recent Posts
Recent Comments
Most Comments
Visiter Info
Newsletter
Tag Cloud (limited)
Author
Some blank widgets with no heading containing JavaScript ad code

The problem is I cannot easily try this as I've put the site back to v1.3 until I can troubleshoot this.  I cannot have the site going down every 20-30 minutes.

Coordinator
Mar 8, 2010 at 8:18 PM

How many posts do you have?

Not sure if it will make a difference, but you could try removing the Visitor Info widget.  I haven't looked into it, but a person reported this issued with it, and I can see that it makes a callback to the server every time the page loads to try and get information, and this results in potentially multiple web requests being made on the server to load pages/URLs for every link found in the page being served to visitors on your site.  At least this is what I gather by glancing thru the code ... I haven't personally done much with the Visitor Info widget.  But this looks like it could be potentially resource intensive.

Mar 8, 2010 at 9:36 PM

Stats of blog are (Using the XML provider):

943 posts
21266 comments
37769 raters

I'm going to play a little with the CLR profiler.  Try to narrow down the area that has the problem.  Personally, I'm guessing it's a leak from somewhere.  The previous version had a memory leak in the related posts that caused me a similar problem at the time.

I think this is going to be a long one...

 

Aug 27, 2010 at 8:51 AM
Edited Aug 27, 2010 at 9:06 AM

@scottishguyinca: Did you manage to find a solution? I have the same "problem".

My memory usage increases to 200MB using the XML Provider and Blogengine.Net 1.6.1 although I have only
-120 posts (but with a lot of images inside) and about
-700 users a day.

I tried all the things described above and also changing back to the standard theme or removing all widgets but nothing worked for me. I also minified JavaScript, CSS and the images (in total they have 30MB).
My application pool recycles every 2 minutes since I'm using shared hosting and the provider has set a limit.

thx andi

Aug 27, 2010 at 5:45 PM
Edited Aug 27, 2010 at 5:45 PM

@moddie

Hey.  I wish I could give you a solution, but never found one.  I went back to v1.3.  It works well, without any leaks.  I think the newer versions are just not tested in a high load environment.

I have a heavily customized version with video features too, so it's a real pain to upgrade.

Good luck!  If you find a solution, let me know!

 

Aug 27, 2010 at 9:05 PM

It appears to be leaking memory. I'm running 1.6.0 and as I navigate from post to post the memry use on the server seems to creep up.

Aug 31, 2010 at 12:13 PM

Hi there. I've had the same issues with 1.6.1: BlogEngine.NET was eating ~600kb in my test installation, per post view!. I did some research and found some places where it's leaking memory and how to fix it. See the details here:

http://www.pitorque.de/MisterGoodcat/post/Fixing-the-memory-leaks-in-BlogEngineNET-1-6-1.aspx

I hope this helps some of the folks here and hopefully it can be integrated into the next release.

 

Coordinator
Sep 1, 2010 at 9:57 AM

Good findings in your blog post.  I left a comment.  The first two changes can be made, and some different changes to WidgetZone can be made which would effectively solve any issues with the events being over-wired (or over-subscribed).

Sep 1, 2010 at 11:13 AM

Hallo Ben. Thank you for the feedback! I've updated the post with a solution that also works with multiple zones.

Sep 2, 2010 at 6:53 PM

@Mister_Goodcat, @BenAmada: Thx for your work... but it still doesn't work for me. I'm trying out a fresh install of 1.6.1 with the improvements described by Mister_Goodcat... then I will disable Extensions, Widgetzone, Features and see what happens.

Anyway BlogEngine.NET is still a cool platform and I'm looking forward for the next version. ;) If my provider wouldn't limit my memory...

Sep 4, 2010 at 7:31 AM

Hi moddie. Can you elaborate on how it "doesn't work" for you? I think you may run in a different issue. For example, even after my changes, BE will still use caching.

Sep 5, 2010 at 3:12 PM

@all Thanks for your help so far. Here are the things I tried:

-Disabled Referrer Tracking, removed all widgets (and the widget zone itself...)
-Disabled all extensions
-Used the standard theme without customizations
-Used a clean install of 1.6.1 and 1.5
-I used the latest customizations you provided in your blog post... http://www.pitorque.de/MisterGoodcat/post/Fixing-the-memory-leaks-in-BlogEngineNET-1-6-1.aspx
-I renamed the files folder under App_Data just to make sure it's not related to my 1000 images which are about 30MB in total. The total amount of blog post data is about 33MB.
-I disabled the features provided in the settings of the admin page.

After disabling the Related posts I 'felt' that the memory usage was increasing more slowly but it increased over the limit of 130MB and the application pool recycled. If I look at the memory I can see that 30MB are allocated within 1 second... I don't really know but it feels a little bit wrong if my data (33MB) uses 200MB memory... but maybe that's OK.

Unfortunately I can't be more precise because I don't have more control over the shared hosting environment. Anyway I continue to use 1.6.1 since there are no other issues than the recycle...

Coordinator
Sep 6, 2010 at 5:04 AM

Mister_Goodcat, thanks these changes have been made in BE 1.6.2.5.

Coordinator
Sep 6, 2010 at 5:33 AM

@moddie, I think the initial 30 MB memory within the first few seconds is normal.  On my local computer, if I stop IIS (w3wp.exe is gone), and restart it and pull up a test blog with only 1 post in it, w3wp.exe goes up to 27 MB.  I think most of this is just common overhead.

The number of images you have shouldn't have any impact -- which is good for you!

The number of posts does matter since all of the posts are cached in memory.  But 120 posts shouldn't be a problem, and shouldn't cause it to go anywhere near 130 MB.

It sounds like you really disabled about everything.  In the past, I've seen a memory problem with the Twitter widget which I think is fixed now, and it looks like you removed widgets all together.

What if you temporarily remove ALL 120 posts, and then see what happens with the memory.  You can remove them by clearing out (or moving) the XML files in the App_Data\Posts folder and restarting the blog application so the cache is cleared and data is re-read.  If it's okay after removing all 120 posts, then maybe try adding 1 post (and restart) and see what happens.  Then maybe try adding about half the posts (60) and see what happens, etc.

Just curious if the number of posts, or content in the posts has any impact on the memory issue you're seeing.

Sep 7, 2010 at 11:58 AM

I have tested it with the following results (always with an app pool recylce before I did another test):

  1. 0 posts / 0 pages:
    The allocated memory goes from 40MB (initial .NET I think) to 60-70MB
  2. 0 posts / 18 pages:
    The allocated memory goes from 40MB to 75MB
  3. 1 post / 0 pages... here I had some different results:
    - The allocated memory goes from 40MB to 80-90 MB and a little bit later suddenly the pool recycles
    - The allocated memory goes from 40MB to 60 MB for 30 seconds, then to 80 MB and suddenly the pool recycles
  4. All posts / 0 Pages:
    The allocated memory goes from 40MB to 100MB and then slowly allocates more memory and after 1 minute it recycles / or it suddenly recycles
  5. With 15, 30 and 60 Posts memory fills more slowly than with 120 posts.

At the end of my test it happened that the allocated memory was about 55MB (!) and after I opened my site it jumped to 100MB.

So maybe it depends on the amount of users trying to access the site... it feels like the allocated memory depends on the amount of users and some object is recreated for every user. Maybe the variable containing all posts...

Sep 23, 2010 at 2:18 PM

I copied a backup to 2 different DEV environments (IIS6 / IIS7) and I could not reproduce the memory allocation. It never reached more than 65MB ... even with 4 different sessions it stayed at the same memory.

Thx for your help... I think I will stop at this point.