Technical challenge

[DaveB]

Pointless excercise. With out basic information to make an initial estimate from there is no point in doing it. You are as likely to get it right by dumb luck as you are by trying to work anything out. Might as well ask how long a piece of string is.

[MarillionFan]

First things first AtW. Whats the rate? Is this IR35 friendly? What are you payment terms? Can you put the contract up first for us to look at?

[AtW]

ok, here is what you have: 100 mln unique strings - 20 bytes each

Your job is to create a system that would allow to either confirm that a given string does not exist in the list of those unique strings you have, or, if it does exist, return unique numeric ID that you can associate with each of those unique strings - database equivalent of RowID.

In terms of performance the system should allow for at least 50 searches per second, or in other words one search should not take more than 0.020 sec.

How fast my system performs (writing benchmark code now) is irrelevant since I want to know what kind system would YOU need to build in order to get around that performance. It does not have to be exact, say 10 servers with X GB ram and Y CPUs each using Oracle etc: estimated cost £Z.

MarillionFan: if you can't give me estimate of work then you ain't getting contract in the first place because clearly you would not have a clue how to do execute it in a way that puts my interests first, ie: very cost effective solution.

[MarillionFan]

£60k. Same as a deposit.

[sasguru]

ok, here is what you have: 100 mln unique strings - 20 bytes each

Your job is to create a system that would allow to either confirm that a given string does not exist in the list of those unique strings you have, or, if it does exist, return unique numeric ID that you can associate with each of those unique strings - database equivalent of RowID.

In terms of performance the system should allow for at least 50 searches per second, or in other words one search should not take more than 0.020 sec.

How fast my system performs (writing benchmark code now) is irrelevant since I want to know what kind system would YOU need to build in order to get around that performance. It does not have to be exact, say 10 servers with X GB ram and Y CPUs each using Oracle etc: estimated cost £Z.

MarillionFan: if you can't give me estimate of work then you ain't getting contract in the first place because clearly you would not have a clue how to do execute it in a way that puts my interests first, ie: very cost effective solution.

Remember what I said earlier about Aspergers Syndrome? This is a classic symptom, not having the basic empathy to understand that your post is quite probably the most boring ever to be posted on CUK ...

If Carlsberg did boring threads, this would be it ...

[DaveB]

ok, here is what you have: 100 mln unique strings - 20 bytes each

Your job is to create a system that would allow to either confirm that a given string does not exist in the list of those unique strings you have, or, if it does exist, return unique numeric ID that you can associate with each of those unique strings - database equivalent of RowID.

In terms of performance the system should allow for at least 50 searches per second, or in other words one search should not take more than 0.020 sec.

How fast my system performs (writing benchmark code now) is irrelevant since I want to know what kind system would YOU need to build in order to get around that performance. It does not have to be exact, say 10 servers with X GB ram and Y CPUs each using Oracle etc: estimated cost £Z.

MarillionFan: if you can't give me estimate of work then you ain't getting contract in the first place because clearly you would not have a clue how to do execute it in a way that puts my interests first, ie: very cost effective solution.

Current performance is very relavent since we have no idea of what the software is capable of on it's current platform so we have no indication of the loads it is likely to put on the hardware.

If you are getting X% of your desired performance on a particular hardware platform then we can start to predict performance on other configurations taking into account OS overheads, disk speeds etc.

Is the idea of this excersise to show how much cheaper your solution is to an oracle based one?

Anyway, I'm of to do something more interesting with my free time than take on unpaid systems design work

[AtW]

There is no current system - new system developed by me is totally brand new, which is in any case irrelevant - what is relevant is that you know what the system should do and I want your own estimate on the basis of your own experience.

How would you even approach this problem - use database?

Anything goes - don't worry about getting non-optimal answer as it is pretty much impossible to get it done better than my approach.

[threaded]

All very well AtW, wondering about the system, but you ought to know: most of us here could've got the client to pay us more to do the same thing.

threaded in "and here endeth the first lesson" mode

[PRC1964]

don't worry about getting non-optimal answer as it is pretty much impossible to get it done better than my approach.

At this point I gave up. There is no point in me living any more. Some fat moustachiod Russian immigrant in a bedsit in Birmingham has the answer to all the worlds problems and it is impossible for anyone to be better than him.

If only I could have such a great life.

[AtW]

I will have my benchmarks in just a few minutes, hurry up with your own estimates because I will actually get it all working