Any Data analysts/scientists out there

[bobspud]

Ok will do.

2. Key attributes of a data analyst in a nutshell:

- experience and knowledge of programming (scripting/hacking skills not software dev) in at least a couple of languages (mainly R, SAS, Python)
- Statistical qualification (with a strong element of mathematical stats, not just cookbook stats) and experience with modelling
- hadoop/mapreduce ( I admit I don't know much about this yet).

3. Where:
- internet sites that collect masses of data (not my area)
- as eek says mainly marketing departments (banks, supermarkets), specialised consultancies (e.g. dunnhumby, TNS)
- increasingly (and this is my area) Health data/HEOR. This area is booming and its biggest problem is staffing.
Sorry I am not going to say who my clients are as I am direct with them.

Interesting that the ability to link trends or anthropical practices is not on there...

With big data being the key to all this I would have thought that "if this / then that" type of interest would be the face of the gold mine...

[sasguru]

Interesting that the ability to link trends or anthropical practices is not on there...

With big data being the key to all this I would have thought that "if this / then that" type of interest would be the face of the gold mine...

That's essentially what I mean by modelling: e.g. linear/logistic regression, survival analysis will give you those associations.
And stuff I'm not really interested in in my sector like market basket analysis, PCA etc.
There are a range of statistical techniques and increasingly CS algorithms that do what you expect.
One thing that really interests me and that I have not got my head round yet is the interface between CS and Statistics. The latter is all mathematical and proof-based and the former is all hacking and its hard to jusge whether the results are meaningful.

[aussielong]

One thing that really interests me and that I have not got my head round yet is the interface between CS and Statistics. The latter is all mathematical and proof-based and the former is all hacking and its hard to jusge whether the results are meaningful.

CS=comp sci?

The foundations of CS is maths. I encourage you to read PI calculus π-calculus - Wikipedia, the free encyclopedia


Stats is maths for those that could not do pure maths. We all know that!

Your unknown unknown is the size of your ignorance.

[BigRed]

I thought the software did all the data mining these days instead of having to hire sasguru to do all the brain numbing number crunching?

[sasguru]

CS=comp sci?

The foundations of CS is maths. I encourage you to read PI calculus π-calculus - Wikipedia, the free encyclopedia


Stats is maths for those that could not do pure maths. We all know that!

Your unknown unknown is the size of your ignorance.

You're a bit mouthy for an unemployed, middle-aged programmer with no resources.
Yes I know CS is based on maths - that was my first degree.
In the data science/statistical learning arena, for the sake of argument there are 2 approaches: the statistical and what statisticians refer to (usually wrongly) as the CS approach: the latter consisting of algorithms that produce results like clustering but are very ad-hoc: a good example is k-means.*
Statos are suspicious of the latter, CS people say "if it works it works".

*On second thoughts k-means isn't a good example, as it does have some statistical basis.
Here's an interesting discussion on what I mean:

http://stats.stackexchange.com/quest...chine-learning

[sasguru]

I thought the software did all the data mining these days instead of having to hire sasguru to do all the brain numbing number crunching?

If that were the case you could expect constant chaos - contrary to popular impression and sensational articles real AI does not exist yet.

[Mich the Tester]

If that were the case you could expect constant chaos - contrary to popular impression and sensational articles real AI does not exist yet.

I've seen a lot of AS in my time though.

[bobspud]

I thought the software did all the data mining these days instead of having to hire sasguru to do all the brain numbing number crunching?

Mining data is not a problem. But reading it and making the connections is. If you want a good analogy try using Splunk log aggregation indexing. Even with just a 100 or so log devices you can generate several thousand traps in an hour.

I can look at a report and give you a pointer where to look for the problem in a matter of a few minutes but it still needs that human eye to spot the notional "this does not look like it should" I have no doubt that one day I will be replaceable for an AI unit that is less petulant.

[Spacecadet]

I am looking to set up a speciality business placing contract data analysts.
Are any of you knowledgeable enough to help me:
1. Find data analysts to place
2. Understand the key attributes of a good data analyst
3. Understand where in organisations these people sit
4. Who to talk to in organisations about requirements (please god not HR/internal recruitment)
Or is this a bit off topic for CUK?

I posted this on here a couple of weeks ago:
Becoming a Data Scientist - Curriculum via Metromap ← Pragmatic Perspectives!

Problem with the whole data science/big data thing is that it is about to hit the top of the hype cycle:
Hype cycle - Wikipedia, the free encyclopedia

If that model is correct then it will take 2 years to get back to normal

[sasguru]

I posted this on here a couple of weeks ago:
Becoming a Data Scientist - Curriculum via Metromap ← Pragmatic Perspectives!

Problem with the whole data science/big data thing is that it is about to hit the top of the hype cycle:
Hype cycle - Wikipedia, the free encyclopedia

If that model is correct then it will take 2 years to get back to normal

True - although I think the its well and truly in the hype cycle already . I know people who have "big data" and "data scientist" on their CV. To me that's BS. Either you have fantastic data-munging skills or stats skills or both.