Reply to: SKA news

searches

Not So Wise: point taken -- I am going to use your searches to work with ranking formulae to see what thresholds needs to be set to get reasonable results. Its still work in progress but I appreciate you trying it now, I hope you will give it another go in next releases to see if any positive changes took place.

METAs: this is true that I stopped indexing META tags, however I think you confuse ability for search engine to display short description of a site taken from META, from actually searching through METAs -- these are notorious for spam and most search engines give them near nil score value. Google is said to ignore META keywords completely.

I agree about showing descriptions from META -- will modify indexer to take these into account.

Re: easy

I run mine on an old PII laptop with 200MB of RAM that was gathering dust.

Install is a simple click and run.

AtW - Post a link to the software and .NET framework plus service pack for those that wish to join in.

easy

> I meant, is it reasonably simple to run the crawler to help you out as Dim requested we do?

Oh that -- its easy, you can even configure it to work only when you walked away from your computer, but generally it uses little CPU and can run in background

Re: fairly hard

Alexei,

I meant, is it reasonably simple to run the crawler to help you out as Dim requested we do?

Well it's definatly fast, but honestly i would not use it.
First off it's relevance matching sucks, just to try I put in ultima online (as know there are a lot of sites about it but not too many), while it did find the offical site, it was listed about 5 down

But when i put in UO (common abv for Ultima online) or even uo.com (offical url) the official site was not listed on first page never mind first entry

2nd reason would not use it is you are not baseing it on meta's, descriptions or anything except raw text as far as i can see, while i can understand where you are coming from in basing an engine on the text and not the meta's/descriptions here for me is the problem

General News Game Updates Forum Sign In! Forum Registration **DAOC **Final Fantasy **Lineage 2 **Star Wars **World of Warcraft™ Arirang Asuka Atl

This is of zero use to me, if the site name or url does not tell me what the the site is about,it is a useless record to be displaying
Where as google has the following to say

Community news, game updates, shard information, and active player forum boards for in-game trading of rares, houses and other items and services.

Now when i want to find something usefull where do you think i will go?

I

fairly hard

> Is it reasonably simple to do, bearing in mind I’m no expert in web thingies?

Its fairly hard -- checked codebase last night, its at ~80k lines, the only library I use is gz/bz2 compression. Looking back won't say its fooking impossible, but there were many hard moment and there will be -- doing these things that require high performance and good memory management is certainly a challenged in .NET.

A lot of it is a learning curve -- it took a while to create decent crawler even though first version of the crawler was running 7 months ago. Its just real world has unique requirements that can't be predicted. There are many more lines of code in crawler than search engine itself, at the moment that is -- I am now working primarily on searching components so they are growing nicely.

riches

> When Alexei is as rich as Bill Gates he'll buy us all a Ferrari,

I tell you what guys -- should I become seriously wealthy as the result of this project then I would be obliged and pleased to share big chunk of cash with those who helped me -- pro-rata of contribution and taking time of joining into account as well as extra stuff.

This will be as much to reward those who helped, but also to seriously annoy those who did not

Re: PR

Some of us already have...
(not at the moment as machine busy doing something else)

Re: PR

Is it reasonably simple to do, bearing in mind I’m no expert in web thingies?

Re: PR

Anyone here going to join the early adopters and run AtW crawler on their PC and help the guy out?

When Alexei is as rich as Bill Gates he'll buy us all a Ferrari, except Threaded of course, he's get another Lambo to add to his collection.

Make or break time...

/. it!!!!!!

cheeky post

> How about a cheeky post in their forum to lure people across?

I can't do it Dim :rolleyes

Once search engine is of decent size relevant people will start talking and these guys there will hear -- grub lacks search engine, people crawl but it appears all for nothing. If I was them I'd be pretty pissed off, and they are -- lots of people left them.

PR

Thanks Milan.

> When are you going to do some PR and where will you start ?

Can't and don't want to start PR before search engine has decent number of pages indexed and results are relevant -- normally I start version from 0.5.0 on my projects, but on search engine I started from 0.1.0 (ALPHA!) to show that its very very first version so people have to be reasonable and not expect it to compete with Google, not yet at least.

A decent ready-to-use product is really a pre-requisite. I expect that 500 mln indexed pages should provide reasonably good answers in majority of cases, going to create plug-in for Firefox to search directly in my search engine shortly -- dog food so to speak.

Good one SB.

:rollin

Re: originality

If only people who crawl in their program knew about my site -- we could have been getting 20 mlns URLs a day

How about a cheeky post in their forum to lure people across?

This is worth a read news.bbc.co.uk/1/hi/business/4118770.stm