SKA news

[wobbegong]

AtW, I'm sorry, I must be missing some of the history of this. While the coding is good, the appearance familiar, and the results satisfactory, I have to ask; what's the point, when something better already exists, why not think of something original?

You really do need to think about Bogeymans questions, pitching something that "looks like, but isn't quite as good as it's main rival" to investors could prove interesting.

[DaveB]

[QUOTE=wobbegong]AtW, I'm sorry, I must be missing some of the history of this. While the coding is good, the appearance familiar, and the results satisfactory, I have to ask; what's the point, when something better already exists, why not think of something original?[QUOTE]

I'd have to disagree with the satisfactory results.

As an example I looked for "danish cartoons" in ska, yahoo and google. Specifically I was after links to the story about the cartoons published in a Danish paper recently that have caused a degree of controversy amongst the arab world.

Both Yahoo and Google return 10 relevant matches on the first page, google slightly faster but I'm not that fussy. Ska produced none on the first page untill I changed the relavancy setting to Best Relevancy. Then it got 4.

Doesnt matter how fast it is if it doesnt find what I'm looking for and what it does find is only relavent after I have to mess with non-intuitive settings.

I'm not a coder, I don't care what the code looks like or how fast it is, within reason, quicker than 2-3 seconds is all I'm worried about. I only care about whether the results returned are relavent to the search terms.

I'm not trying to knock your achivement. Technically it's an impressive piece of work, but rather than banging on about how fast it is or how many words you have indexed you should be concentrating on getting your relevancy improved so that people will actually have a reason to use it.

Even with this resolved I am struggling to see why I should switch to this rather than Google.

[Lucifer Box]

Here's the last bit of code I wrote (on a demo ZX81 in WHSmiths):

10 PRINT "MARTIN WASSALL IS A W ANKER"
20 GOTO 10

Can you guess what it does yet?

[AtW]

You can admire a man who uses GOTO...

I've spend a couple of weeks optimising it with profiler - here is quote from the page where I released it (my quote): "homepage snapshot (20 KB) is parsed as fast as under 2 ms on Athlon 2.2 Ghz PC." The version that I have now is even faster than that - go on, try writing HTML parser with similar performance characteristics.

wobbegong: the point is being better, but it is impossible to become better before first becoming decent, almost as good, definately as good etc. The path is long and I am making progress - if you were able to compare searches in previous versions of SKA you would have seen that a lot definately improved, particularly scale.

DaveB: I accept your comment about relevance fully - this is something I am working on, one part of it is actually scaling the search engine because it is impossible to find most relevant document if its not actually present in the database, hence the importance of speed - if design was inefficient then speed would have dropped dramatically when scaling, luckily this is not happening. Therefore speed is a necessary pre-requisite of relevance - it does not replace it, but its required.

Why would anyone stop using Google? Well, consider for a moment that Google is focussed on mainly English pages, this lead to the fact that Google is not popular in many countries, for example in Russia Google is nobody in terms of traffic driven to sites - no wonder because they have a lot less pages.

With SKA people from all over the world can participate and they can focus on crawling their own countries. This means SKA can grow much bigger than Google in those niches that they ignore - a fantastic reason for people to search beacause in these niched SKA will be better than Google.

[AtW]

I'd have to disagree with the satisfactory results.

As an example I looked for "danish cartoons" in ska, yahoo and google. Specifically I was after links to the story about the cartoons published in a Danish paper recently that have caused a degree of controversy amongst the arab world.

SKA is currently not good on up-to-dateness - this will change very shortly, but for now if you want to make a fair accessment then run a selection of queries that you normally run.

[AtW]

Here is some of mine - what does it do AtW ?

SKA requires high-performance code - do you know effects of virtual functions on performance?

In one place I had to make choice to use interfaces rather than virtual functions in C# because performance analysis shows that interface calls are much faster, it sure is a bit more painful to do them and I had to resort to some copy/paste, but at least I've got high performance code - and when you have to process terabytes of data, it is hard not to notice difference.

[DaveB]

SKA is currently not good on up-to-dateness - this will change very shortly, but for now if you want to make a fair accessment then run a selection of queries that you normally run.

Actually that is the kind of search I run normally. I see something in the news and I want to kow more about it.

Sorry ATW but it's the same accross the board. After switching to Best Relevancy ( Why do I need to do that? Why isnt it the default? )

jaffa cakes - to pick up on anothe thread here.

Google, Yahoo, MSN 10/10 on the first page
SKA 3/10

cold fusion - The Macromedia web development product
Google, Yahoo, MSN 10/10
SKA 3/10

cold fusion physics - The science stuff.
Google, Yahoo, MSN 10/10
SKA 9/10 Best result for SKA so far.

[AtW]

Best relevancy is just the name given by someone - in SKA you can edit ranking formulae, you are not forced to accept single one designed by search engine owner, now the name for that formulae was a bit too ambitious.

Its good that you did this test, in a few weeks you will see significant improvements - remember we only have 200 mln pages indexed (a lot more crawled though), and its not possible to find something that is not present in index, that's why I am pushing to scale it higher and that's why speed is important

[AlfredJPruffock]

Best relevancy is just the name given by someone - in SKA you can edit ranking formulae, you are not forced to accept single one designed by search engine owner, now the name for that formulae was a bit too ambitious.

Its good that you did this test, in a few weeks you will see significant improvements - remember we only have 200 mln pages indexed (a lot more crawled though), and its not possible to find something that is not present in index, that's why I am pushing to scale it higher and that's why speed is important

Did you read the news today about Googles share price demsie on Wall Street ?

When do you think you will make any money out of this, or is it just a bit of a hobby ?

[Francko]

AtW, have you ever thought that your graphics skills seriously suck?

The site is so dull. Russian style?

How about adding some graphical decency to your logo and website in general?