Data Modellng tools?

[scooterscot]

As I have a degree in Mathematics and develop business models for people using Business Intelligence, where i have to model data and processes as well as undertaking the odd underwear photo shoot for catalogues the term model means lots of things to me.

Well done. OK, so now, how do you model human behaviour? If BI is a fancy name for statistical modelling/SPSS, how do you make predictions of future events with a system that has so much human involvement?

[MarillionFan]

Well done. OK, so now, how do you model human behaviour? If BI is a fancy name for statistical modelling/SPSS, how do you make predictions of future events with a system that has so much human involvement?

You're now talking about predictive modelling. Which is based on analyzing the past. BI is Business Intelligence which encompasses all available tools. PM me if you want my catalogue shots.

[scooterscot]

You're now talking about predictive modelling. Which is based on analyzing the past.

So your assuming the conditions of the past are the same as those in the future? Ekk! We engineers would be shot down in flames for making such an assumption.

PM me if you want my catalogue shots.

Are they available with handcuffs?

[MarillionFan]

So your assuming the conditions of the past are the same as those in the future? Ekk! We engineers would be shot down in flames for making such an assumption.



Are they available with handcuffs?

Predictive modelling is about using the conditions of the past to make assumptions about the future. Not the same.

[scooterscot]

Predictive modelling is about using the conditions of the past to make assumptions about the future.

For the most part I would agree, however, the exact conditions of past and present are rarely identical making for poor predictions. Sometimes with the smallest assumptions having the largest impacts.

Would you use predictive modelling to look at past share prices to make assumptions about future share prices without considering the conditions of the here and now?

Same question for climate models...

Now how about for an aero engine?

[cojak]

Thanks guys.

Just to let you know, I'm in the foot hills with this.

Scooterscot, I mean this sort of thing.

Data modeling

[scooterscot]

Thanks guys.

Just to let you know, I'm in the foot hills with this.

Scooterscot, I mean this sort of thing.

Data modeling

Ach zo, reminds me of a mind map.

[beercohol]

There are all kinds of models - even in software engineering. For instance the whole UML suite of models that look at a system from interactions, classes, use cases etc.

What Cojak is asking about, specifically is Data Modelling.

There are two main approaches to this. Choose betweent them based on the size and complexity of the project. But basically it involves designing a database schema.

If the system is not too complicated, you can get away with stamping out a few database tables as you go along. Most developers work this way. The model is refined through stages of 'Normalisation' through first (1NF), second (2NF) and third Normal Form (3NF). Most people stop at 3NF, but it is possible, and painful, to go as far as Domain Key Normal Form (7NF), but I've never done it and neither can I even remember how it works. These techniques allow you massage a bad or roughly designed data model into good quality, highly flexible database with a good balance of efficiency between quierying and use of storage.

As you start working on large distributed apps that cover vast and complex data requirements, you need to engage in some proper data modelling instead - the second approach that Cojak has been tasked with. This involves analysis and interviews with the client to gather their business data requirements. You then build a series of 'External Views' (a silly name really, because they are more like internal views) the data types. These diagrams are represented as Entities, Attributes and Relationships. A popular diagram technique is the ER diagram, but this confuses the heck out of non-techies, whose input is vital. I've already recommended the Kalido BIM tool as the replacement for ER diagram app like ERWin).

The collection of 'External Views' is concatenated into the Conceptual Model of the entire system. From this, a process of transformation is used to turn these Entities, Attributes and Relationships into relational database Tables, Columns and keys. This gives you the Logical, and finally the Physical model (the final SQL DDL with the CREATE Table statements needed to build the database schema in SQL Server/Oracle/MySQL).

[xchaotic]

There are all kinds of models - even in software engineering. For instance the whole UML suite of models that look at a system from interactions, classes, use cases etc.

What Cojak is asking about, specifically is Data Modelling.

There are two main approaches to this. Choose betweent them based on the size and complexity of the project. But basically it involves designing a database schema.

If the system is not too complicated, you can get away with stamping out a few database tables as you go along. Most developers work this way. The model is refined through stages of 'Normalisation' through first (1NF), second (2NF) and third Normal Form (3NF). Most people stop at 3NF, but it is possible, and painful, to go as far as Domain Key Normal Form (7NF), but I've never done it and neither can I even remember how it works. These techniques allow you massage a bad or roughly designed data model into good quality, highly flexible database with a good balance of efficiency between quierying and use of storage.

As you start working on large distributed apps that cover vast and complex data requirements, you need to engage in some proper data modelling instead - the second approach that Cojak has been tasked with. This involves analysis and interviews with the client to gather their business data requirements. You then build a series of 'External Views' (a silly name really, because they are more like internal views) the data types. These diagrams are represented as Entities, Attributes and Relationships. A popular diagram technique is the ER diagram, but this confuses the heck out of non-techies, whose input is vital. I've already recommended the Kalido BIM tool as the replacement for ER diagram app like ERWin).

The collection of 'External Views' is concatenated into the Conceptual Model of the entire system. From this, a process of transformation is used to turn these Entities, Attributes and Relationships into relational database Tables, Columns and keys. This gives you the Logical, and finally the Physical model (the final SQL DDL with the CREATE Table statements needed to build the database schema in SQL Server/Oracle/MySQL).

Or you drop the whole pointless exercise and dump the whole data as XML, noSQL or RDF triples.
The problem: Normalisation only applies if you store your stuff as traditional relational tables.
Squeezing your data as tables will result in more efficient queries as long as users don't try to put unexpected, structured data in your flat row (hint: that will happen always)

Consider a simple (real life) scheme: registering cars parked in a garage.
Initially it was simple enough - registration numbers were used
Eventually a car from abroad came - the validation was loosened to allow any text.
As now it was loose some gurds started registering mopeds, bicycles, all kinds of tulip using the now free form text field - as there was no other place to put it.
I guess one could have done a very long list of fields to classify an object stored in a garage - some people stored shopping carts, how does your 7th form normalisation deal with "Real Life(™)"?

[Spacecadet]

Consider a simple (real life) scheme: registering cars parked in a garage.
Initially it was simple enough - registration numbers were used
Eventually a car from abroad came - the validation was loosened to allow any text.
As now it was loose some gurds started registering mopeds, bicycles, all kinds of tulip using the now free form text field - as there was no other place to put it.
I guess one could have done a very long list of fields to classify an object stored in a garage - some people stored shopping carts, how does your 7th form normalisation deal with "Real Life(™)"?

Bit of a crap example really, the vehicle registration would either have to be free text or have a DVLA lookup to cope with personalised number plates.
10-12 characters limited to a-Z, 0-9 would be enough for any foreign cars (foreign car registrations may contain a dash - but this could be ignored)
The guards should have been told to stop ******* about and trying to register bicycles on a system made for recording cars which needed parking in the garage.