Big data and Hadoop

[NickFitz]

"Big Data" is in the same category as "RESTful" and "Agile" for me. Which is to say that different people seem to use these phrases to mean radically different things, and when they're used without context it's usually a sign that the speaker isn't really sure what they're doing or why.

There's a good rant by Roy Fielding (whose doctoral dissertation defined REST) on the subject of misuse of the term "REST API": REST APIs must be hypertext-driven: "There is so much coupling on display that it should be given an X rating."

[NotAllThere]

"Big data" has value only when mathematically trained statisticians or data mining or AI specialists (and there is a convergence of all 3) apply their skills to make sense of it.
All these other buzzwords involve low-level manipulaton of data ready for the experts to do their stuff.

Darn. So my writing of stuff to load and process hundreds of millions of accounting records doesn't count. Oh well. I'll stick at being an 'umble programmer.

[Platypus]

1.2 TB is "big data" ????

FWIW, MyClient thinks that Big Data is going to be a massive market and have invested $16BN in this area (so they claim).

[ChimpMaster]

Millions of bobs are already all over Big Data, so as a skill it will have little value even if when the application for it takes off.

[Platypus]

Millions of bobs are already all over Big Data, so as a skill it will have little value even if when the application for it takes off.

Millions of Bobs != Skill

[NickFitz]

Millions of bobs are already all over Big Data, so as a skill it will have little value even if when the application for it takes off.

I think Big Data took off quite some time ago: Yahoo!, Google, Bing are all examples.

[minestrone]

I think Big Data took off quite some time ago: Yahoo!, Google, Bing are all examples.

Dear God.

NF's first post ascribing some merit to microsoft.

Fell off my seat there.

[Incognito]

An example of 'big data'. A client I was working with handles over 100 million financial transactions a day. Transactions that indicate who the consumer is, their fiscal health, what they're purchasing and when they're purchasing it. It's a gold mine of data which they are working out how to monetise.

[AtW]

Processing a trillion edge web graph daily, no hadoop required

[minestrone]

My client is bigger than yours, so there.