Googlebot is getting on my nerves

[TheRefactornator]

It still lists the keywords from the old site, even though the new site has been live for two days and been crawled twice.

Two days is not very long and the Google indexing algorithm takes a while for changes to be applied and actually appear in the SERPs. Rest assured that Googlebot has seen your new content and posted it to the indexer for update..it just doesn't quite "believe" the changes are permanent yet so they are effectively sand boxed until it believes your content is stable again. Even when this time has passed you might still need additional effort to get to page 1 of the natural SERPs.

Important sites are serviced much more quickly because of their "importance" within the Google PR system which is why new items on news sites and blogs with high PR are indexed very quickly into the SERPs.

[AtW]

Appearing in searches for particular keywords is ranking and it has not much to do with googlebot (unless your site fails or can't be parsed etc).

You are probably in a competitive category and in this case in order to go on top you need quality relevant backlinks with good anchor text.

[suityou01]

Virtually all these questions are addressed in Google's own guide on the subject, which I linked to the last time this question was asked in January.

The Google Webmaster Central blog also posted a pretty detailed examination of the matter.

As an aside, I always find it quite amusing that people ask questions about these matters on various forums where they have no real chance of judging the quality of the advice they receive (there's a lot of mumbo-jumbo, voodoo-chicken-waving and cargo-cult-nonsense surrounding SEO) when the answer to their question has usually been published by Google itself.

Google may be understandably secretive about their core algorithms, but they publish an enormous amount of information about SEO best practices. After all, the easier people make it for Google to find stuff, and the more they are taught to avoid gaming the system, the better Google's index becomes, and the more money it makes. Google actually needs people to be good at this stuff, so it tries its best to help them.

In addition to the Webmaster Central blog and the SEO Guide, I highly recommend the blog of Matt Cutts from their anti-spam team. (He's currently carrying out an experiment by moving his blog wholesale to a new domain to see how it affects his position in search results, so don't be surprised if that link redirects you to dullest.com instead.)

Well, firstly the site it hosted in Drupal, which has it's own SEO module check off list, which I have followed to the letter. I have also read ALL of googles advisory. I have created a sitemap, and notified google to come and crawl the site which it has done, twice.

It still lists the keywords from the old site, even though the new site has been live for two days and been crawled twice. There are no crawler errors. Hence the thread, googlebot is p*ssing me off, I have done everything by the book and yet I am not any further forward.

[NickFitz]

I have read the google SEO doc, what I was asking was the magnitude of the difference, as I previously pointed out I understood there was a difference. I am refactoring in REST URL conventions into Plan B, I just wanted to know the priority.

It's just another thing to add to the Plan B list, i just wanted to know where to put it on the list.

The Webmaster Central piece should give you a better idea of how to prioritise it. However, it's probably worthwhile going for meaningful URLs right from the start, as if you switch later you'll have to put a whole bunch of redirects in place.

I'd suggest going for the ".html" extension (e.g. http://example.com/foo.html), whatever the backend technology might be; the URL should reflect the content type of the resource's representation, rather than revealing details of the backend implementation. Alternatively, don't use an extension at all (http://example.com/foo).

Also consider whether you'll want to offer different representations of the same resource; if you want to present a resource like a list of products as, say HTML, Atom, CSV and JSON (so you can provide an API) then using the appropriate extension can make content negotiation much easier:

http://example.com/foo
http://example.com/foo.html
http://example.com/foo.xml
http://example.com/foo.csv
http://example.com/foo.json

would all be references to the same resource, just served in different formats. (The first without an extension would allow for content negotiation using the "Accept" header, and probably default to HTML.)

If you go the road of serving different representations of the same resource at different URLs, you should look into using <link rel="canonical"> as well.

[AtW]

I am refactoring in REST URL conventions into Plan B, I just wanted to know the priority.

Make it high - optimising your pages titles, urls etc is one of the easiest long term wins that you can get, the sooner you do (especially the urls) it the better.

[minestrone]

I have read the google SEO doc, what I was asking was the magnitude of the difference, as I previously pointed out I understood there was a difference. I am refactoring in REST URL conventions into Plan B, I just wanted to know the priority.

It's just another thing to add to the Plan B list, i just wanted to know where to put it on the list.

[NickFitz]

So I take it that ending with .html is preferable to having the "googlebot-getting-my-nerves" somewhere in the URL but with the "thread=41821" queryString remaining like..

http://forums.contractoruk.com/technical?thread=41821&pageTitle=googlebot-getting-my-nerves

So how much does putting the title into the page URL impact on the page placement and how much does removing the query string and hiding it in the URL with the page title make? OK, that is 2 questions.

Virtually all these questions are addressed in Google's own guide on the subject, which I linked to the last time this question was asked in January.

The Google Webmaster Central blog also posted a pretty detailed examination of the matter.

As an aside, I always find it quite amusing that people ask questions about these matters on various forums where they have no real chance of judging the quality of the advice they receive (there's a lot of mumbo-jumbo, voodoo-chicken-waving and cargo-cult-nonsense surrounding SEO) when the answer to their question has usually been published by Google itself.

Google may be understandably secretive about their core algorithms, but they publish an enormous amount of information about SEO best practices. After all, the easier people make it for Google to find stuff, and the more they are taught to avoid gaming the system, the better Google's index becomes, and the more money it makes. Google actually needs people to be good at this stuff, so it tries its best to help them.

In addition to the Webmaster Central blog and the SEO Guide, I highly recommend the blog of Matt Cutts from their anti-spam team. (He's currently carrying out an experiment by moving his blog wholesale to a new domain to see how it affects his position in search results, so don't be surprised if that link redirects you to dullest.com instead.)

[AtW]

Keyword matches in path and filename are very good for ranking, avoid having same words in query params - they might not be matched at all or have lower score.

Actual ranking will depend on other factors too, but it makes sense to deal with things that are under your control.

[minestrone]

I have a quick SEO question about URLs, I will use this page URL as an example

Once upon a time this URL would probably have been..

http://forums.contractoruk.com?thread=41821

But to get the words in to the URL they moved it to

http://forums.contractoruk.com/technical/41821-googlebot-getting-my-nerves.html

So I take it that ending with .html is preferable to having the "googlebot-getting-my-nerves" somewhere in the URL but with the "thread=41821" queryString remaining like..

http://forums.contractoruk.com/technical?thread=41821&pageTitle=googlebot-getting-my-nerves

So how much does putting the title into the page URL impact on the page placement and how much does removing the query string and hiding it in the URL with the page title make? OK, that is 2 questions.

[AtW]

Does everyone not use SKA these days?

Those who want to rank well do...

[minestrone]

Does everyone not use SKA these days?

[TheRefactornator]

DMOZ although it undoubtedly helps google page rank if you can actually get listed, is a submit and forget arrangement fraught with editors protecting their own interests in many of the categories. It's no longer an authority of the internet and it should die.

Inbound links (at present) power the core of the google page rank algorithm. Volume, relevance and page rank of the referring site is important but getting them is a catch22 - free internet directories (other than DMOZ) can help if you're willing to put the effort in.

Press releases from good newswire feed companies can help long term page rank if the site content is interesting enough to get bloggers and journalists writing about it. See PRWeb for an example.

Content is very important if you intend to get the entire internet's blog community writing about your site to drive traffic.

You can also check the competitor websites that appear in the natural SERPs for your search term to see what's causing them to appear high up.

[dinker]

Post a link on here, I`m sure we`ll all help out and click it.

[Grinder]

Register site in DMOZ
Google Ads
Get some links in from well used sites

[suityou01]

I have put a new site live for my friend's firm of electricians. He wants to appear highly in the ranks for "electrician Dunstable" when people search, for instance.

I have done all of the SEO stuff, verified the site with google, created a sitemap, included the keywords in the content, meta tags and title.

The stats indicate that googlebot has visisted twice now, and yet I have every other bloody keyword from the site, but not those two.

Those two keywords are on the front page, in meta tags on all pages and in the flaming title.

What more can I do?