SKA news

[Lucifer Box]

Look AtW, if you're not looking for comments, then stop posting about your project on a public bulletin board. If you think that warning you that it is dangerous to concentrate on one aspect of a project to the exclusion of all others is "trying to put you down", you need to stop, push your chair back, go for a short break away from it all and get your tulip wired together.

Okay, I am an etailer with a million product lines. You can offer me a 0.1s search time. Why is that better to me than a 0.5s search time? You need to have the answer ready, maybe not in detail right now, but you need to have at least considered that there is an answer and you need to be thinking about that every day. If you consider you project as only a series of loosely related technical challenges to be overcome, there will be tears before bedtime.

Napoleon [paraphased]: "a good general must ask himself 20 times a day, 'what will I do if the enemy appears here, or there?'. If you are ever stuck for an answer, you are a poor general."

[AtW]

But obviously Alex, you think you know it all and nobody else on this forum should dare to contradict your superhuman knowledge of the subject.

You are just using features designed for you by others - having actually implemented much bigger database that you will ever have I think I am within reasonably degree of probability to consider myself far more knowledgeable on subject of low-level database workings because unlike you I actually implemented rather than used somebody else's work.

so you can count and tag billions of records (matching a given set of criteria) in seconds (depending on hw of course, IBM z/x best).

Oh feck off you retard! FFS, if you can't see difference between running a search over billions of words with real-time ranking and delivery of most relevant matches in matter of sub-seconds with mere counting of matches in seconds then you are even more clueless than clueless squared.

FFS, if I just wanted to count matches it would not even take 0.1 second on current database of, let me remind you, 80,000,000,000 indexed words. Now feck off and try to load billion words into your database and come back when its done loading at least 1% of what I have here.

[AtW]

Okay, I am an etailer with a million product lines. You can offer me a 0.1s search time. Why is that better to me than a 0.5s search time?

Easy - it will take you 5 times less hardware to achieve the same. Note that for 2 mln products my stuff can keep it all in memory and actually deliver sub-0.1 sec searches, with better relevance - most home brew searches don't have positional info and different scores for different parts of document (ie title should get better rank).

This means that my search engine can achieve faster and more relevant searches using a lot less hardware - searching often accounts for well over 60% of database usage for big online etailers, this means serious savings in terms of needing less boxes.

Note - my quoted time of 0.1 sec for 2 mln products is for one consumer grade PC - many etailers have to resort to using more than one box for database, or use huge 4-8 processor boxes with unholy amount of RAM.

[bogeyman]

You are just using features designed for you by others - having actually implemented much bigger database that you will ever have I think I am within reasonably degree of probability to consider myself far more knowledgeable on subject of low-level database workings because unlike you I actually implemented rather than used somebody else's work.


Oh feck off you retard! FFS, if you can't see difference between running a search over billions of words with real-time ranking and delivery of most relevant matches in matter of sub-seconds with mere counting of matches in seconds then you are even more clueless than clueless squared.

FFS, if I just wanted to count matches it would not even take 0.1 second on current database of, let me remind you, 80,000,000,000 indexed words. Now feck off and try to load billion words into your database and come back when its done loading at least 1% of what I have here.

I think you are losing the plot old boy.

>>having actually implemented much bigger database that you will ever have

How do you know how big my db implementations have been (or the size of my dick - which I refuse to wave about - unlike you)?

>>Oh feck off you retard! FFS

Your logic is impecable, as ever.

>>rather than used somebody else's work

I'm 48 years old and have created a hell of a lot of original and acclaimed work in the DB arena. You are who?

You, Sir, are a rude and egocentric jerk who thinks he has all the answers but who is ultimatley a patheric loser living in a daydream.

Sorry, but that's my answer.

[AtW]

I'm 48 years old and have created a hell of a lot of original and acclaimed work in the DB arena. You are who?

How many rows your biggest database kept and what hardware was used for it? If its below 100 bln rows then don't bother answering

[bogeyman]

How many rows your biggest database kept and what hardware was used for it? If its below 100 bln rows then don't bother answering

Waving your dick again AtW?

For God's sakes! Does it matter how many rows?

It's not the rows that count, it's the information in the tables, how many tables, how they relate by keys, how effective is the DBMS at optimising queries with joins, etc. etc.

If I told you I designed the BT/CSS database for itemised telephone billing in 1983 would that impress you?

Anyway, if you think that a SEARCH ENGINE should use a RELATIONAL DATABASE as its index, then you're a bigger ***** than I thought you were?

You're full of tulipe AtW!

[AtW]

For God's sakes! Does it matter how many rows?

Yes it does matter - it shows level of complexity of the problem at hand.

You're full of tulipe AtW!

Unlike you I have something to show - full text search engine that handles 80 bln words from 200 mln web pages that had size of 4.5 TB. And this will be at least tripled in February and increased further in March.

You acclaimed work in DB arena is not known to me - you certainly not the person who is known to have build a very big database using feck all hardware.

I expect to have 1 trillion words indexed very shortly - when you design database that was used to held so much data then please let me know the hardware and software requirements, perhaps you would have achieved a much more cost effective solution than mine?

So move along old man - let the young and creative lead the way.

[bogeyman]

So move along old man - let the young and creative lead the way.

Lovely!

[AtW]

Lovely!

I take it no answer to my specific question about your biggest database? Nothing to brag about? I thought so!

[Fungus]

Isn't technology wonderful? Thanks to modern innovations such as the internet, we can make new friends, and learn so much, all without leaving our homes.

ATW: It pays to listen to others. Who cares how fast your search is? If a search takes 5 seconds or 0.005 seconds, I don't give a XXXX.

I too am old enough to have seen enough ventures go tits up as a result of people thinking that the world will beat a path to something that is technically superior. Psion were the perfect example of that. They used to boast about the superiority of their product. Then along came Palm, followed by Microsoft, with a better approach, that was less geeky and more suited to the real world. All people care about is the bottom line: what does it do for me?

I do seem to have a history of starting with a company, finding out that it was all complete ego-wank, and then leaving just before it all went tits up, or being laid off when it went TU.

I do admire Steve Jobs. There is a nice story about him and Woz. He came back from the first sale, and gave Woz $500, which was half the profit. (I forget the actual figures.) Decades later Woz found out that Jobs had in fact made $5,000 and that he had ripped off Woz. Woz cried. That's the difference between a visionary and a talented geek.

Fungus