Data science and predictability of outcomes.

[BrilloPad]

So, I work with people who spend more time telling you about their ability and qualifications on machine learning than making accurate predictions.

I'm kind of going to say much of this is pseudo science.

Or religion as it is otherwise known.

[original PM]

Brexit was not predicted for two reasons

1) There was no previous data which could be relied on to be used in predicting an outcome - the fact we joined the EEC in 1973 could not be used as it is to old.
2) In clearly marginal cases such as brexit a sample poll of 2000 or even 10000 clearly did not scale up to the 17 million who actually voted.

I have always been very wary of people who can 'predict' the outcome of anything - even if they have a lot of data to work with mainly because it is impossible to know the future.

Of course we could all put data sets together saying that x is a more likely outcome than z assuming a,b,c etc do not happen

In fact I think in todays fast moving market place 'pretending' you can do some science to predict the future is more harmful than accepting you only know what you know and gearing up your business to be able to react quickly.

[NotAllThere]

Or religion as it is otherwise known.

Cretinism Creationism is a pseudoscience, but religion isn't.

[northernladyuk]

Cretinism Creationism is a pseudoscience, but religion isn't.

Religion is pseudo-history.

[NotAllThere]

Religion is pseudo-history.

Partly. It's also a mindset and a philosophy.

[northernladyuk]

Partly. It's also a pseudo-mindset and a pseudo-philosophy.

FTFY

[scooterscot]

Did you do a postgrad stats course or something? All sorts of deterministic models make use of historical data to estimate their parameters without introducing any stochastic terms. As I said, it's a modelling choice. No idea what you're on about with regard to "similar event" or the time invariance of a stochastic process or "likelihood" w/r to a deterministic prediction. I think you're getting confused by stationarity or, perhaps, just in general. An event is simply a subset of a sample space. You don't need a "similar" event to make a prediction, you need a model, which may or may not have parameters to estimate from data.

It's true some deterministic predictions may use of historical data. Typically not the case with myself, I would not depend on the results nor would my clients. In my industry accuracy is a prerequisite or I'll be sent packing back to Blighty. Probabilistic predictions offer far more utility using historical data than deterministic ever will.