Thanks for the canonical code you provided.
In regards to SEO there are 100's of many different factors that effect website ranking.
In terms of Google I had worked for Google as a Search Engine Evaluator and I know for fact they use humans to review some of the websites
that they index.
The major #1 thing Google is looking for now is "Usefulness" and "Utility" that websites provides to users.
With that said you can have your website 100% correctly with on page "SEO Optimized" with the correct and proper use of meta tags and the
right amount of keyword density per page and etc.
Even with having the correct "on" page "SEO Optimization" your website can still have lower ranking than websites that does not.
This can be the case with websites that have higher server response rates, have unique authoritative content, provide more usefulness and utility than the "proper"
SEO Optimized websites.
There are many different "Signals" that Google gets from websites and the sources comes from many different sources.
They are all then added and weighted in and then factored in to come up with the end result of Website rank.
Also you need "Off" page "SEO Optimization" too.
This is where I bet 95% chance the "crappy" website you are referring to has more "Off" page SEO.
Off Page SEO includes other websites that links to your website, not only how many but specifically which type of websites link to your website.
If you have a website about purely asp.net c# BlogEngine.net
Websites that has content only about "PHP" and nothing about BlogEngine in Google eyes would be valued less than
other websites that links to your website that has content about asp.net c# BlogEngine.net
The more websites content goes away from the content of your website the lower their link value gets to be added to your over all ranking score.
If that same website has links from shopping websites or other websites that has nothing to do with "Programming" "Technology" or the Internet
it is possible for those links to have an "negative" effect on your over all ranking score.
This is why it is very important to check exactly who is linking to you, what link they are using, and what anchor text they are using on their
website to link to yours for all of those factors also has an effect.
The take away here for SEO is the following:
- Think like your User / Reader of your website
If your website is unique, provides usefulness and utility to your users Google and others are going to like your website :)
- Content Freshness
Must have unique, original, content that are fresh and added periodically.
You simply can not just post 20 or 30 posts at a time and then never post new ones again or update the ones you already have.
- Social Media
Got to share it with social websites and etc
- "On" page SEO
- "Off" page SEO
If you follow these steps and thinking in order you will do good.
With recent changes and current trends "On" and "Off" page SEO are starting to be valued less in the over all ranking score calculations.
Even through they are now "valued" less than before they are still important through.
Steps #4 and #5 can be viewed as squeezing out all of the SEO value points you have left in your website.
Now days you can not just do "parts" of SEO Optimization but must include the entire picture now.