I’d like to introduce Richard O’Neill from the Financial Times who will be speaking at DMM7 on agile data migration being held in excellent setting of the BCS HQ near Covent Garden.

But first let us have a quick look at agile and the particular challenges it brings to data migration.

It seems like agile methods and Salesforce implementations go hand in hand. Something about the promise of ‘no software’ software and the lure of process-lite delivery methods are made for each other. The software is available almost immediately so why not the delivery?

An agile method, with its emphasis on providing useable deliverables quickly, somehow seems the ideal fit for software which equally needs zero development time. But can we make data migration fit with these aggressive time scales?

Well in some senses yes we can. We can get our hands on data relatively easily but transforming it and enhancing its data quality to something you would likely use can be the sticking point. Sometimes the business just doesn’t seem to want to play ball with your aggressive timescales.

Maybe it’s period end in the finance department but surely they could find time just to check those two thousand major customers for the correct account details? And the marketing team? Weren’t they the ones most in favour of Salesforce? Yet here they are quibbling about being given a whole development sprint (two solid weeks!) to de-duplicate a mere 10,000 customer records?

An issue - as I’ve noted before in this blog - with agile, for all its business focus, is that it tends to march to a rhythm that can get out of phase with business cycles. The two-week drumbeat of our sprints don’t really map onto the monthly/quarterly/annual cycles of business life. And those pesky business folks seem to have other concerns (sales conferences, major bid work) that drag them away from delivering the data you require even when they have promised it.

So how do you square this circle? Well, in this admirable presentation, Richard O’Neill shows how. Working on a significant seven million-customer migration to Salesforce, he overcame these challenges, using an agile variant of the Practical Data Migration (PDMv2) method.

Richard is a ten-year veteran in the data quality and data management fields and also a PDMv2 certified practitioner. I think his presentation is a must see for anyone who is looking at or engaged in an agile development. But even if agile is not your thing right now, this is still a great introduction to the practical implementation of PDMv2 in a real project setting.


About the author

John Morris has over 20 years experience in IT as a programmer, business analyst, project manager and data architect. He has spent the last 10 years working exclusively on data migration and system integration projects. John is the author of Practical Data Migration.