Does Google crawl the redirect? - Contractor UK Bulletin Board

Visitors can check out the Forum FAQ by clicking this link. You have to register before you can post: click the REGISTER link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below. View our Forum Privacy Policy.
Want to receive the latest contracting news and advice straight to your inbox? Sign up to the ContractorUK newsletter here. Every sign up will also be entered into a draw to WIN £100 Amazon vouchers!

You are not logged in or you do not have permission to access this page. This could be due to one of several reasons:

You are not logged in. If you are already registered, fill in the form below to log in, or follow the "Sign Up" link to register a new account.
You may not have sufficient privileges to access this page. Are you trying to edit someone else's post, access administrative features or some other privileged system?
If you are trying to post, the administrator may have disabled your account, or it may be awaiting activation.

Guest replied

15 August 2004, 10:55
re

"GET /robots.txt 0" 404 1213 "-" "Mozilla/5.0 (compatible; Yahoo! Slurp; help.yahoo.com/help/us/ysearch/slurp)"
"GET / 0" 200 189 "-" "Mozilla/5.0 (compatible; Yahoo! Slurp; help.yahoo.com/help/us/ysearch/slurp)"
"GET /dyn/index?var=home&p=1&id=1 0" 200 73206 "-" "Mozilla/5.0 (compatible; Yahoo! Slurp; help.yahoo.com/help/us/ysearch/slurp)"

Damn it, they're all doing the above. Anyone know why? arghh
Leave a comment:
Guest replied

14 August 2004, 00:59
re

if you don't have a robots.txt file does it mean that the crawler will crawl everything?
Leave a comment:
Guest replied

14 August 2004, 00:55
re

Looks like MSN bot didn't like it:

207.46.98.103 - - [13/Aug/2004:14:37:24 +0000] "GET /robots.txt 0" 404 1213 "-" "msnbot/0.11 (+search.msn.com/msnbot.htm)"
207.46.98.103 - - [13/Aug/2004:14:37:28 +0000] "GET / 0" 200 189 "-" "msnbot/0.11 (+search.msn.com/msnbot.htm)"
Leave a comment:
Guest replied

11 August 2004, 15:57
Re: re

Use site:www.mysite.co.uk to see if it has

Personally I don't use meta refresh because I heard that search engines don't like them. I do use javascript redirects with no apparent problem though.
Leave a comment:
Guest replied

10 August 2004, 18:26
re

If my index.html page is a redirect, will Google follow it and parse my site or stop?
Leave a comment:
Guest started a topic Does Google crawl the redirect?

10 August 2004, 18:24
Does Google crawl the redirect?

Meta tags are not allowed
Tags: None

Working...

X