Concerns over scalability

May 23, 2008 at 6:09 AM
Edited May 23, 2008 at 6:37 AM
I've been investigating moving my personal blog over to BlogEngine.net and have been reviewing the source prior to doing this. During my review I became concerned over the pattern you're using to retrieve posts. On the first page load you seem to load a complete set of posts for the blog into a static property (using the provider's FillPosts()) method. For a small blog with a limited number of posts this is fine, but it doesn't scale. As an example my own blog has 895 posts with around 500 comments, each are pretty large. The pattern used in BlogEngine.net results in this entire set being stored in a static property for the lifetime of the application and of course a significant hit whenever the app reloads. I had begun writing a LINQ enabled provider which adds paging capability and lazy loading to the post loading but have now abandoned this effort; it would require changing most of the post pipeline.
Can I suggest that you examine this issue? One relatively simple solution could be to use ASP.NET caching to store each post / reload when necessary. Although not a perfect solution it works pretty well and would be far more responsive to server memory pressure than the static property method used now.
May 25, 2008 at 4:27 AM
This has been noted by many people and Mads has acknowledged that it is a design choice that needs looking at as the project expands.

See http://blog.madskristensen.dk/post/Cache-expiration-puzzle.aspx for some related commentary.
May 25, 2008 at 6:24 AM
Thanks Leigh, I wasn't aware of that post.
May 27, 2008 at 6:54 AM
I also noticed this, but I view it more as design choice than design flaw. Caching data into in-memory objects can make access faster, especially when the website got many hits and 'live'.

But I also realized that when data grows bigger and web application often started and stopped, this design choice becomes design flaw.

Feb 15, 2009 at 4:47 PM
i am also working on creating a LINQ to Sql Provider for BlogEngine.NET i got the Idea from Wrox Linq-ifying TheBeerHouse. if Caching is needed then i'll see what i can do and get back to you