This project is read-only. W3wp.exe process Memory Consumption very High

Topics: ASP.NET 2.0
Nov 9, 2010 at 2:08 AM


I'm running BlogEngine.Net and I'm getting really high Memory consumption from the sites. I isolate each site to run in it's own App Pool ID and with Task Manager open I can see each process consumes between 20,000 K to 100,000 K of memory, under the Memory (Private Working Set) column. I have the IIS App Pool configuration set to allow up to 4 processes per App Pool, the recommended is 8 - 2 per CPU core. So with this modest set up, I have 400 MB of Ram consumed per BlogEngine.Net Site. With my other sites running I'm at 30% Memory consumption with just twelve sites running on a high end server with 8 GB of RAM. With my Wordpress sites running under PHP none of the processes go beyond 8,000 K of Memory consumption.

There seems to be a memory leak in the code because the Memory consumption climbs by 2 or 3 K every few minutes if I watch it under Task Manager. I had to set each App Pool process to recycle every two hours to not allow the memory consumptions to climb, otherwise it has gotten to about 200, 000 K or more of Memory per process at the default IIS App Pool settings, and again it is set to 4 processes per site.

So could you help me figure out how to debug this and where to look to find the memory leak? I have Visual Studio 2010 Professional and could download the BlogEngine.Net source code, let me know how to proceed to use the Debugging Tools to find the leak.

Also, I figure that Output Caching should help this not happen. What Parameters should I set to have the Output Caching vary by to have this work correctly. any other insights that you have would be appreciated. Here are my server specs to give you an idea of the environment:

Sever specs -

Dedicated Server with Windows Server 2008 R2 64-bit Standard Edition
Intel Xeon Quad Core 2.67 Ghz CPU
8 GB DDR2 Ram

Running IIS 7.5 latest updates as of 11/8/2010

Nov 9, 2010 at 4:25 AM

Hi, the information you provided is helpful, but what version of BE are you using?

Nov 9, 2010 at 4:43 AM

I ask because there were some memory leaks that appear to be in BE 1.6 that were fixed afterwards.  This post here identified 3 areas (BlogRoll, CodeFormatter, WidgetZone) that were causing high memory on his system.  With the fixes, it appears memory became normal.  We put these fixes into a developer build of BE that will be included in the next BE release (version 2.0 later this year).

These same 3 areas could also be leading to the high memory you are seeing.

Nov 9, 2010 at 4:43 AM

The latest one, 1.6.1 I believe it is.

Nov 9, 2010 at 4:45 AM

Okay, looks like we cross posted here.  Please see my last message from a couple of minutes ago.

Nov 9, 2010 at 4:49 AM

OK, I'm reading your post about the Memory leaks and fixes and I'll let you know what happens after I apply the fixes. Besides that, my other Sites are still high in Memory consumption by comparison - they go from 20,000 K to 60,000 K is there a good way to reduce that? My php wordpress sites never go beyond 8,000 K I'm trying to get my Asp.Net sites in that range, any ideas on how to go about that, what sort of debugging to do in Visual Studio? Would Output Caching help that?

Nov 9, 2010 at 5:03 AM

I think ASP.NET sites use a little more memory than PHP sites -- there's more overhead in general.  But they should still not keep growing in memory like you're seeing.

I personally don't have any experience debugging the cause or source of where memory usage is coming from with ASP.NET.  But in that post, towards the end of the comments, Mister Goodcat says:

For analyses like that I use WinDbg and the built-in profiler of Visual Studio. Please note that you need Visual Studio Premium or better for that, it's not included in the Professional or Express versions.

... so maybe WinDbg if you don't have a higher end of VS.  There could be other tools too.

How the application is designed can make a difference in the memory you see.  For example, when BE first starts, it loads ALL the posts and other data into memory/cache and keeps them there.  By doing this, it doesn't need to keep going back to the data store (XML or DB) to retrieve posts.  This is a little extreme in my opinion since once you get up to having thousands of posts, you'll start to see high memory.  Under "normal" blogging conditions, it would take an individual BE user several years of regular blog posting to reach the level of posts that would cause consistent high memory.  So it still works well.

Output caching I think would help most with CPU.  With that, the page doesn't need to be regenerated on each request and can be served from memory.  So if anything, output caching might lead to higher memory (but not a lot) since the pages are cached in memory.

If things go well, and these memory leak fixes work for you, then your BE installation might use around 50 MB to 75 MB of memory at all times.  I don't think output caching is necessary.  Plus with output caching, it needs to be designed so authenticated user information that might appear on the page doesn't get cached and served to unauthenticated users (or users who are logged in under their own account).  As an example, if you are logged into the site and looking at some unpublished posts (logged in users can see unpublished posts), you wouldn't want that page rendering to be cached and displayed to subsequent users who are not logged in -- since they shouldn't be able to see unpublished posts.

Nov 9, 2010 at 4:50 PM

OK, thanks for all your help. I applied the fixes and now the site I applied them to is steady at 50 MB of Ram use per process. I'll apply the fixes to the other BE sites I have running which should reduce the Memory foot print overall. I've read that running all of the sites under one App Pool reduces memory usage because then the sites can share the .Net framework and CLR and rely on App Domains for isolation, thus not duplicating instances of the .Net framework for each site. It didn't work before because the code was leaking anyway so it kept climbing and there was no notable benefit of doing this but now that the leaking is fixed I'll try it and see if it helps.