Search Engines / Website Traffic.

[HeliCraig]

Evening all,

I have been working on an osCommerce implementation for a friend for a little while now (see www.spinneyfarm.co.uk - meat by mail!!).

He asked me today how we can improve its performance in the search engines and generally drive some traffic to it.

I am going to go and add some Meta tags tomorrow and do some stuff specific to osCommerce and SEO; but I haven't been involved with any search type stuff for a while - so just looking for any recommendations anyone may have??

Ta.


HC.

[NickFitz]

Google's Matt Cutts recently recommended an article by Jefferson Graham; I'm linking to Matt's post rather than directly to the article, as the "Related posts" section links to another post with a short video interview with Matt on the subject.

The Google Webmaster Central blog is another excellent resource.

Also, use Google's Webmaster Tools to keep an eye on how Google views the site.

BTW, be aware that neither Google nor Yahoo! bother with meta keywords any more, as they're used so much for spamming. I think Microsoft Live Search and a few others might still take them into account. The important thing is to keep them to a small number, and add them on a page-by-page basis to increase specificity rather than just using a generic set on every page. Limit them to no more than ten or twenty per page, as Google and Y! will still penalise sites that they think are using excessive numbers of keywords, on the basis that they're probably SEO spammers.

Meta descriptions are worth having, but again, make them specific to each individual page, or at least section/category.

[Liability]

As well as the above there are Backlinks - VERY important - Google, amongst other things, uses the site popularity as a key indicator on where it places your site so ensure you have good backlinks from sites that have a higher PR [Page Rank] then yours.

Keywords - make sure your tags and descriptions and aligned for users and not to spam or get around search engines. Google looks at content and knows if you are trying to insult its intelligence.

Make sure the keywords you use are research on the google keyword tool- in fact make all your keywords aligned with their popularity - that way you know you are being searched and will, if done right, appear high on the listings.

Remember - Now 85% of online sales comes from ORGANIC listings in Google - NOT PAID ADWORDS - so work this well and business will be good! Screw it up and itll be tulipe.

[NickFitz]

I'll add one more point: meaningful URLS are popular with search engines. For example:

http://www.spinneyfarm.co.uk/index.php?cPath=40

tells me (and Google) nothing about what that page speaks of - it also tells me more than I need to know about the underlying implementation of the site (PHP, using a Front Controller model to dispatch requests for specific categories of content).

http://www.spinneyfarm.co.uk/product...products_id=99

is no better - "Mmm, I think I'll have some 99 for dinner tonight"

Even worse, there's no way of telling that the second URL points to a sub-resource of the resource denoted by the first URL - thus the structure of the site's information cannot be intuited by either a user or a search engine

Now, if those URLs were

http://www.spinneyfarm.co.uk/lamb/

and

http://www.spinneyfarm.co.uk/lamb/rolled-breast/

then not only could humans make more sense of them, but search engines would also make more sense of them, and (probably) rank them more highly for relevant keywords. Furthermore, it would be obvious what the hierarchy of the information is ("rolled breast of lamb" is a specific instance of the general category "lamb").

This isn't to say that "dynamic URLs" are, per se, bad - Google say not - but for that site, meaningful URLs make a lot more sense.

This should be easy enough to achieve with a suitable .htaccess file if the site doesn't change too much. If you need more complex mappings, maybe because product ranges change quite frequently, you should only have to make minimal changes to the PHP backend and database to support something like this. If you look at a typical WordPress install and see how it handles URLs for categories via .htaccess, PHP, and mySQL, it should convey the general idea. (Hint: autoincrement fields are great for databases, not so much for the web.)

To give an example, I have a site which allows URLs of the form

http://example.com/users/Fred/

which .htaccess turns into

http://example.com/user.php?id=Fred

The second example would still work, but the first one is the canonical form, and is more friendly for both users and search engines.

(Actually, these days I put the scripts in a non-public-accessible directory, so the second wouldn't work, but that's another matter.)


Oh, and if that's more work than makes sense at the moment, then creating an XML sitemap of the form described at the Google Webmaster Tools site is simple and easy to do and will help the search engines (they all recognise such files now) to understand the structure of the site - although it still won't help the humans

Incidentally, although I've talked solely about Google, both Yahoo! and MS Live Search offer similar webmaster tools. Dunno about the other search engines that nobody ever uses...

[HeliCraig]

Nick - Thanks for your excellent response (as always). Some of the tools around to make osCommerce more search friendly do indeed make the URLs much more human readable and that is very much on the list of things to do.

Thanks for all the other information too. Luckily it is a quiet day at ClientCo today so I should have time to read and digest them all.

Liability - thanks for that also! Not too sure about how we are going to get BackLinks; but I shall have a think.

Thanks guys.

Any more for any more?

[DimPrawn]

As the others have said but also:

Title - most important tag on each page. Make each page have a unique title where possible. Include your keywords (meat, farm, fresh, direct, organic, etc) in the titles. Make the title short and descriptive.

Inbound links should contain your keywords and come from relevant sites, especially homepages. Links coming in should link deep into your site rather than all into the homepage.

Have plenty of links between your pages (crosslink them) and include your keywords in the links.

Have a sitemap linked for the homepage that allows the search engine spider to reach every page of your website, again, keywords in anchor text.

Check that the keyword density for your main search keywords matches the top sites in Google. Use tools to measure keyword density on each page and adjust the text to match the same density as the top 2-3 sites in google.

Good luck. It's hard work to get to the top.

[voodooflux]

Any more for any more?

Think most of the bases have already been covered by the chimp.

Sitemaps, URL rewriting, page titles and headers are all worth special attention.

[Sysman]

Don't forget robots.txt

This is a timely thread, as I've just been asked to find out why Google seems to be ignoring a site which has recently undergone a complete rework.

The site in question was running Wordpress but isn't any more, and you can see loads of 404 errors in the logs where Google and other search engines are trying to access the old structure (and largely ignoring the new structure).

What I did yesterday afternoon was to modify robots.txt to disallow anything which looked like Wordpress, with the result that by this morning I could see that Google had walked most (all?) of the new site.

Have a Google for "seo robots.txt", and you will find out more on the subject.

[NickFitz]

This is a timely thread, as I've just been asked to find out why Google seems to be ignoring a site which has recently undergone a complete rework.

The site in question was running Wordpress but isn't any more, and you can see loads of 404 errors in the logs where Google and other search engines are trying to access the old structure (and largely ignoring the new structure).

What I did yesterday afternoon was to modify robots.txt to disallow anything which looked like Wordpress, with the result that by this morning I could see that Google had walked most (all?) of the new site.

Have a Google for "seo robots.txt", and you will find out more on the subject.

Handing them a sitemap file will also help.

Also, if you set things up for a month or so such that all those things they're looking for and not finding return a 410 Gone, rather than just 404 Not Found, they'll clear them out of their indexes straight away. 404 means "Not Found, but it might be back, so feel free to check again" whereas 410 means "Gone for good, don't bother asking in future".

Obviously only do this for the actual things that used to exist but don't anymore - other non-existent stuff should still return 404. The easiest thing, if you're on Apache, is to set up a bunch of mod_rewrite rules (in httpd.conf if you have access to it, otherwise in .htaccess) to return the 410 - then, after a month or two, you can easily get rid of them as surplus to requirements (unless there are external links pointing to them, in which case continue to return 410 for them).

[Sysman]

Thanks for the 410 tip Nick. With that knowledge I did a bit of experimentation with Google on my own website. I discovered that Google was giving a higher rating to what was a temporary parking spot for my stuff, which I dropped over 6 months ago. Unfortunately I don't have httpd.conf access there, but can try the .htaccess route.