I've been working a little with the feeds from BE.
As far as I can tell - the Atom feeds for a category are double encoding ampersands.
So instead of & in the content of the feed item, a &amp will appear.
Should I log this as an issue?
Also I'm not entirely sure that the Summary element should be used for Atom feeds unless the content is actually a summary of the post. I thought in this case that the Content element should be used.
Hope this helps,