Using techniques from extreme programming and agile development in data migration environments.

For some time now I've been working with clients who have adopted Agile development techniques and, as usual, there is always something we can pick up to add to our kit bag of tools.

I have been banging the drum for data migration having its own methods as a domain in its own right. However I also recognise that tools with an industry proven heritage can be incorporated into our toolbox but they need to be thoughtfully re-positioned.

And so it is with user stories. For the uninitiated, user stories are "A software system requirement formulated as one or two sentences in the everyday language of the user" (according to Wikipedia). Now I'm finding this a really useful technique when getting the first cut statements for the business owners' system retirement policies.

A quick reminder about system retirement policies:

For a data migration project to be successful business engagement is vital. And I prefer active engagement, not viewing our business colleagues as the passive recipients of controlled messages from the project team. To get that engagement one of the first order activities, before any mapping or even data discovery activity, is the commencement of the System Retirement Policies (SRP).

SRP are a statement from the business perspective of the conditions that must apply prior to the old system being allowed to be switched off. It has a number of effects. It makes the, possibly, distant migration a reality to the business and starts to highlight any show stopping problems before they become major issues thereby heading off the responsibility gap.

But how to represent them? In the past I have used all sorts of mechanisms including the dreaded 300 page (at least) specification document. However extreme programming has this notion of user stories which, when twisted to our use, can prove really effective.

The form I am using has two parts - a statement of need (the business scenario) which I try to keep to a single sentence and an acceptance criteria (i.e. how we will know that we have satisfied the need expressed in the scenario). An example for an accounting team might be "We will need access to a history of transactions from the old system to be maintained". The acceptance criteria would be "We need seven years offline history of (generic line item data) and 12 months online or near online history of (detailed line item data).

The results are currently held in a spreadsheet (we are planning to add it to our standard project hub) but this is all so much lighter, and easier to use, than a clumsy document with all the change control issues those pose.

Now I am not using this to scope work packages as such (which is one of their uses within Agile) but to define the acceptance go/no-go criteria of the users. I am sure the acceptance criteria will need to be elaborated upon as the new application is developed, but the main areas of concern are mapped out and we have a common language to exchange views on it. I have yet to take the users down to the detail of prioritising their stories. Although there is a column for prioritisation, I think that can wait until the second iteration around SRP (when the new system design is better understood) and the third iteration (when the migration design is better understood).

About the author

John Morris has over 20 years experience in IT as a programmer, business analyst, project manager and data architect. He has spent the last 10 years working exclusively on data migration and system integration projects. John is the author of Practical Data Migration.