SKA news

[AtW]

So you just told me that it was a read only DB which does 1 billion inserts a week.

It's a read only DB after insertion, so no updates necessary, no need for transactions, no need to return data older than 7-10 days, no need to even be up all the time since users don't pay any money.

And there are hundreds of millions of dollars available to buy best technology out there.

[AtW]

So you just told me that it was a read only DB which does 1 billion inserts a week. This is fantastic.

We have about as much every day and unlike Twitter our data can actually be updated rather than just new insert.

[minestrone]

It's a read only DB after insertion.

So you are saying it is read only database after the 1 billion inserts per week.

I pause for 10 seconds...

...how many inserts have went into the twitter databse in that time?

[d000hg]

If there was a 1% chance that you were executed then I can assure you that you'd feel that you are in mortal danger and about to die.

No I'd feel worried but not that I expected it to happen

Alexi has always mouthed off about his 1 billion links claim and how that is the highest publicised amount when other have publicised it for years. Greater amounts in fact.

Maybe those figures are not to be trusted anyway based on this thread. Might have misplaced a couple of zeros.

[doodab]

A billion links isn't very much. I managed to follow around half that many yesterday, and that was just on youporn.

[sasguru]

If there was a 1% chance that you were executed then I can assure you that you'd feel that you are in mortal danger and about to die.

Seeing as the risk of your dying from some diseases is higher than this in your thirties, you should be in a permanent state of fear.

You really are a complete moron.

[minestrone]

Maybe those figures are not to be trusted anyway based on this thread. Might have misplaced a couple of zeros.

I have a 1 year old excel page of about 800 links to a site I own and SKA still thinks it only has one

[PAH]

When was Google last down for you? I can't remember.

When was Twitter failing to run search due to overload? Yesterday.

Failure of the internet's top players is extremely rare:

BBC News - Microsoft online services hit by major failure

[PAH]

We have about as much every day and unlike Twitter our data can actually be updated rather than just new insert.

If you're scraping so much every day, how do sites know you're not DoSing them?

Is there a level where you can scrape the info in a timely fashion without them crying foul?

[minestrone]

Is there a level where you can scrape the info in a timely fashion without them crying foul?

The majestic 12 bot hits my sites with such infequency that is never going to be a problem for him.