Suity boomed CUK style

[MarillionFan]

Its not rocket science but it requires attention to detail and documented data schemas on both sides of the system. Granted Bob may be able to do it but unless its all documented its a world of pain mainly because people don't realise how complex it may be and the consequences of what happens if it goes wrong.

Data migration is a piece of piss. Of course every system I've ever seen that has moved it's data has made it useless for reporting or business intelligence, but then again what the **** does anyone in IT know about adding value to data.

[aussielong]

Its not rocket science but it requires attention to detail and documented data schemas on both sides of the system. Granted Bob may be able to do it but unless its all documented its a world of pain mainly because people don't realise how complex it may be and the consequences of what happens if it goes wrong.

That's why you usually have an SME involved for nontrivial schemas, that knows the data inside out. Not worth getting a techie who is new to the systems to grasp a number of large data models quickly. You'd only have a techie doing it all for a trivial migration.

[mudskipper]

Its not rocket science but it requires attention to detail and documented data schemas on both sides of the system. Granted Bob may be able to do it but unless its all documented its a world of pain mainly because people don't realise how complex it may be and the consequences of what happens if it goes wrong.

And if you're migrating secure data such as passwords or credit card numbers, you've got encryption etc to worry about too.

Address data can be its own special brand of fun.

[Freamon]

That's why you usually have an SME involved for nontrivial schemas, that knows the data inside out. Not worth getting a techie who is new to the systems to grasp a number of large data models quickly. You'd only have a techie doing it all for a trivial migration.

Which is fine, unless all the permies that understood the source schema have left.

Which they usually have.

[aussielong]

Which is fine, unless all the permies that understood the source schema have left.

Which they usually have.

You get the business to guide you through the data model, not techies. Business people tend to stick around longer in organisations , perhaps because their knowledge is less transferrable.

[aussielong]

And if you're migrating secure data such as passwords or credit card numbers, you've got encryption etc to worry about too.

Good point.

[MarillionFan]

You get the business to guide you through the data model, not techies. Business people tend to stick around longer in organisations , perhaps because their knowledge is less transferrable.

WHS

And that's the Bob proof skill. Don't sit in IT. Sit in the business with IT skills, delivering to the business and explaining to the dullard IT Techies or Bobs how to do it.

Works for me. Kerchiiiiiing.

[aussielong]

Rama Chemudupati, explains his specific experiences on a recent SAP data migration project.

SAP Data Migration

This is one of his "Key Concepts" that must be grasped to be successful:

Identifying and involving the right resources and clearly defining their roles and responsibilities can help you achieve significant success in a data migration project. Over- defining roles can mean blurring the responsibilities. Under-defining the roles can affect the migration timelines and deliverables. Involving business representatives in your migration process and ensuring their availability at various stages along the process also factor into this role definition.

[eek]

And if you're migrating secure data such as passwords or credit card numbers, you've got encryption etc to worry about too.

Address data can be its own special brand of fun.

My favourite one was the promise to the head of marketing - Don't worry about the 5 years of crap data people have been entering the data migration will fix that. Someone promised as the justification of the migration that the migration would fix the missing first names, surnames and email addresses that sales people hadn't been bothered to find out over 10 years.

I provided two options:-

1) your current dataset with all the crap
2) a probably valid complete dataset (with only 8% of the original data). Needless to say neither option was acceptable outside of IT.

[MarillionFan]

My favourite one was the promise to the head of marketing - Don't worry about the 5 years of crap data people have been entering the data migration will fix that. Someone promised as the justification of the migration that the migration would fix the missing first names, surnames and email addresses that sales people hadn't been bothered to find out over 10 years.

I provided two options:-

1) your current dataset with all the crap
2) a probably valid complete dataset (with only 8% of the original data). Needless to say neither option was acceptable outside of IT.

I've just had clientco purchase me a DNB license with 250,000 downloads and 15000 credits for data cleanup.

Interestingly, I provided a sample of around 300 records to do some data matching to DNB. I checked the data, but didn't manipulate hugely. The results came back at a disapointing 60% match rate with a 5% error.

I then knocked up a simple Excel function using the Google Maps Web service. I then ran the address data through that, which returned a full address string, where I sent that off to DNB for matching with company info.

It came back with a 97% match rate, with a 3% error. The next phase is to merge contact data, and the first pass I have highlighted 30% (approximately 15000) records to remove. That Mickey Mouse fella must get an awful amount of spam.