SKA news

[DimPrawn]

Re: PR

Anyone here going to join the early adopters and run AtW crawler on their PC and help the guy out?

When Alexei is as rich as Bill Gates he'll buy us all a Ferrari, except Threaded of course, he's get another Lambo to add to his collection.

[planetit]

Re: PR

Is it reasonably simple to do, bearing in mind I’m no expert in web thingies?

[SupremeSpod]

Re: PR

Some of us already have...
(not at the moment as machine busy doing something else)

[AtW]

riches

> When Alexei is as rich as Bill Gates he'll buy us all a Ferrari,

I tell you what guys -- should I become seriously wealthy as the result of this project then I would be obliged and pleased to share big chunk of cash with those who helped me -- pro-rata of contribution and taking time of joining into account as well as extra stuff.

This will be as much to reward those who helped, but also to seriously annoy those who did not

[AtW]

fairly hard

> Is it reasonably simple to do, bearing in mind I’m no expert in web thingies?

Its fairly hard -- checked codebase last night, its at ~80k lines, the only library I use is gz/bz2 compression. Looking back won't say its fooking impossible, but there were many hard moment and there will be -- doing these things that require high performance and good memory management is certainly a challenged in .NET.

A lot of it is a learning curve -- it took a while to create decent crawler even though first version of the crawler was running 7 months ago. Its just real world has unique requirements that can't be predicted. There are many more lines of code in crawler than search engine itself, at the moment that is -- I am now working primarily on searching components so they are growing nicely.

[Not So Wise]

Well it's definatly fast, but honestly i would not use it.
First off it's relevance matching sucks, just to try I put in ultima online (as know there are a lot of sites about it but not too many), while it did find the offical site, it was listed about 5 down

But when i put in UO (common abv for Ultima online) or even uo.com (offical url) the official site was not listed on first page never mind first entry

2nd reason would not use it is you are not baseing it on meta's, descriptions or anything except raw text as far as i can see, while i can understand where you are coming from in basing an engine on the text and not the meta's/descriptions here for me is the problem

General News Game Updates Forum Sign In! Forum Registration **DAOC **Final Fantasy **Lineage 2 **Star Wars **World of Warcraft™ Arirang Asuka Atl

This is of zero use to me, if the site name or url does not tell me what the the site is about,it is a useless record to be displaying
Where as google has the following to say

Community news, game updates, shard information, and active player forum boards for in-game trading of rares, houses and other items and services.

Now when i want to find something usefull where do you think i will go?

I

[planetit]

Re: fairly hard

Alexei,

I meant, is it reasonably simple to run the crawler to help you out as Dim requested we do?

[AtW]

easy

> I meant, is it reasonably simple to run the crawler to help you out as Dim requested we do?

Oh that -- its easy, you can even configure it to work only when you walked away from your computer, but generally it uses little CPU and can run in background

[DimPrawn]

Re: easy

I run mine on an old PII laptop with 200MB of RAM that was gathering dust.

Install is a simple click and run.

AtW - Post a link to the software and .NET framework plus service pack for those that wish to join in.

[AtW]

searches

Not So Wise: point taken -- I am going to use your searches to work with ranking formulae to see what thresholds needs to be set to get reasonable results. Its still work in progress but I appreciate you trying it now, I hope you will give it another go in next releases to see if any positive changes took place.

METAs: this is true that I stopped indexing META tags, however I think you confuse ability for search engine to display short description of a site taken from META, from actually searching through METAs -- these are notorious for spam and most search engines give them near nil score value. Google is said to ignore META keywords completely.

I agree about showing descriptions from META -- will modify indexer to take these into account.