Does Google crawl the redirect?

**reynolds** · 10 August 2004, 18:26

re

If my index.html page is a redirect, will Google follow it and parse my site or stop?

**fiddleabout** · 11 August 2004, 15:57

Re: re

Use site:www.mysite.co.uk to see if it has

Personally I don't use meta refresh because I heard that search engines don't like them. I do use javascript redirects with no apparent problem though.

**reynolds** · 14 August 2004, 00:55

re

Looks like MSN bot didn't like it:

207.46.98.103 - - [13/Aug/2004:14:37:24 +0000] "GET /robots.txt 0" 404 1213 "-" "msnbot/0.11 (+search.msn.com/msnbot.htm)"
207.46.98.103 - - [13/Aug/2004:14:37:28 +0000] "GET / 0" 200 189 "-" "msnbot/0.11 (+search.msn.com/msnbot.htm)"

**reynolds** · 14 August 2004, 00:59

re

if you don't have a robots.txt file does it mean that the crawler will crawl everything?

**reynolds** · 15 August 2004, 10:55

re

"GET /robots.txt 0" 404 1213 "-" "Mozilla/5.0 (compatible; Yahoo! Slurp; help.yahoo.com/help/us/ysearch/slurp)"
"GET / 0" 200 189 "-" "Mozilla/5.0 (compatible; Yahoo! Slurp; help.yahoo.com/help/us/ysearch/slurp)"
"GET /dyn/index?var=home&p=1&id=1 0" 200 73206 "-" "Mozilla/5.0 (compatible; Yahoo! Slurp; help.yahoo.com/help/us/ysearch/slurp)"

Damn it, they're all doing the above. Anyone know why? arghh

Does Google crawl the redirect?