How to setup a search engine?

[cojak]

<puts on tin helmet, dives into foxhole>

[Bright Spark]

Why don't you look it up on Google?

I did it brought me here as loads of people on here know about
search engines.

[Xenophon]

I do search engines.

I don't do mingers.

[fullyautomatix]

I use Sphider for my products. Its a PHP based search engine. It does a great job and is very fast.

[doodab]

hmmm, i've got all the above except the 'Huge wodges of cash', so
what sorts of computers and bandwidth will I need?

also what kind of technology do you recommend i use, c++/ Java/ C# /Perl?

Well you need enough bandwidth & computing power that you can crawl the entire internet, analyze each page and put the results in some sort of database, plus sufficient bandwidth and power to support a front end that allows people to query the database and obtain .

I would let the smart people choose the programming language they use, but if they say perl or c++, sack them and get some smarter people.

[Bright Spark]

I use Sphider for my products. Its a PHP based search engine. It does a great job and is very fast.

i've used that only thing is it's just loading data into mysql and not really
creating a true search engine algorithm, like BM25 and page rank etc...
have you heard of SOLR, Sphinx ?

[Bright Spark]

I would let the smart people choose the programming language they use, but if they say perl or c++, sack them and get some smarter people.

itsn't c++ good on memory and speed which is vital for a good search engine?
what does google use?

[cojak]

I'm sure that you could get a good monthly deal from Majestic-12.

[Bright Spark]

I'm sure that you could get a good monthly deal from Majestic-12.

hmm 12 bottles of wine for a tenner sounds good to me

[fullyautomatix]

i've used that only thing is it's just loading data into mysql and not really
creating a true search engine algorithm, like BM25 and page rank etc...
have you heard of SOLR, Sphinx ?

I assumed ( you did not provide the context ) that you wanted a crawler for a particular website. If you want page rank etc ( its redundant anyway, google do not use page rank) you might have to write your own algorithm.

So how many websites are you planning to crawl and what is the end point of what you are trying to achieve ?