Big Data

[Mich the Tester]

"Big data" has some meaning in that the tools and techniques you use for both low level stuff like ETL, data warehousing and high-level stuff like analysis, data mining and modelling need to be modified as opposed to when you have "small data"

I know, and I think it's an important development; getting value out of the huge data collections in companies and outside of them will be a big thing. However, forgive me for being a little bit jaded by hearing umpteen thousand buzzwords in the last 15 years.

[Ketchup]

Big Data is one of terms that gets picked up as a buzzword by recruiters (much like "agile" a few years ago).

If you look below the bulltulip, it does actually have a realy meaning (which is probably not required by 90% of the adverts).

[d000hg]

Getting seriously peed off by the term Big Data appearing in role requirements.

For those of you in HR / Agency - Big Data is a marketing term. There is no difference to what you can do with data now.

For those responsible for data security / safety / accuracy - Big Data also equals 'uncontrolled'. The sort of thing most companies have been battling against for years.

FFS

Apart from all the specific BD technologies people want to (mostly needlessly) used, which you either do or do not have experience with.

[DodgyAgent]

Getting seriously peed off by the term Big Data appearing in role requirements.

For those of you in HR / Agency - Big Data is a marketing term. There is no difference to what you can do with data now.

For those responsible for data security / safety / accuracy - Big Data also equals 'uncontrolled'. The sort of thing most companies have been battling against for years.

FFS

Does that mean then that someone who has built a chicken hut could easily turn his hand to building a sky scraper?

[Mich the Tester]

Does that mean then that someone who has built a chicken hut could easily turn his hand to building a sky scraper?

Question irrelevant. Question should be;

'Can I convince HR numpties that someone who has built a chicken hut could easily turn his hand to building a sky scraper?'

Answer is probably;


'Yes'.

[GB9]

Apart from all the specific BD technologies people want to (mostly needlessly) used, which you either do or do not have experience with.

Typically I am seeing recruiters use it to mean 'high volume'.

Now there's high volume, and there's high volume. If you wander off and do some stochastic forecasting e.g. drug development or liability forecasting over a 30 year time-frame, you will generate terabytes of data. Most databases can handle that. Its only when you get into really really big data capture (imagine Hadron Collider sensor recordings, Walmart's transactional data or Google searchy type stuff) that things get tricky.

A lot of companies define Big Data as meaning 'wide ranging'. Oracle is one, from memory. Includes all data not traditionally analysed e.g. social network / media / photos etc.

[GB9]

Does that mean then that someone who has built a chicken hut could easily turn his hand to building a sky scraper?

Not necessarily, but if they are good they could build a bloody great big chicken hut.

[sasguru]

Typically I am seeing recruiters use it to mean 'high volume'.

Now there's high volume, and there's high volume. If you wander off and do some stochastic forecasting e.g. drug development or liability forecasting over a 30 year time-frame, you will generate terabytes of data. Most databases can handle that. Its only when you get into really really big data capture (imagine Hadron Collider sensor recordings, Walmart's transactional data or Google searchy type stuff) that things get tricky.

A lot of companies define Big Data as meaning 'wide ranging'. Oracle is one, from memory. Includes all data not traditionally analysed e.g. social network / media / photos etc.

It's not storing the stuf that's the problem, it's doing something useful with it, like modelling.

[DodgyAgent]

Not necessarily, but if they are good they could build a bloody great big chicken hut.

[Ketchup]

It's not storing the stuf that's the problem, it's doing something useful with it, like modelling.

I disagree, i think part of the requirement of Big Data is to store the data in such a way it can be extracted for any purpose not just for 1 application