Lean Integration, as described in the book of the same name by John Schmidt and David Lyle, is setting out to do is to apply the factory principals of the Toyota Production System (TPS) to the issue of system integration. The question, for this blog, is will that succeed in a Data Migration setting?
Using a factory metaphor we are invited to envisage a centre for integration within our organisations that will own the integration process not just in terms of delivery but for the whole life of the integration and for all integrations. The aim is to replace the mass of spaghetti links that characterise most organisations system topography or at least to start to unravel them. We will replace “work of art”, project by project one off designs with an intelligent process. By intelligent I mean a self reflective practice that will be constantly learning and making itself Leaner by stamping down on waste and upping quality.
The book re-interprets the seven principals of lean manufacturing for system integration contexts as opposed to manufacturing environments. This recasting is almost completely convincing. I have an issue with confusing work colleagues with customers - but more of that later.
The book re-interprets the seven types of waste identified in the TPS, which were, of course, based on physical, shop floor issues, in the light of our system engineering context. This means for instance that Transportation - the unnecessary movement of materials becomes “unnecessary people or data hand offs”. This is where the limitations of metaphor start to become apparent.
I’m always a little concerned when engineers (even a systems engineer) start to use metaphors. A metaphor is a dangerous thing in the wrong hands. A metaphor works by linking together two disparate objects to give greater illumination to one or both of them. But when Hamlet complains of the “Slings and arrows of outrageous fortune” we are not to understand that he has been literally shot at by a bloke called Outrageous. The same is true here. A Toyota shop floor, where machines and people perform physical activities on metal and plastic to produce automobiles is not the same as an IT shop where people and machines perform logical activities on fields and columns, entities and their attributes. We are working with semantic understandings and their changes over different locations and different times. Toyota production line workers are not.
One of our clients is a luxury brand motor vehicle manufacturer but as I look across their production line I do not see the operatives having issues of meaning. From the days of Whitworth and the pioneering work he did in standardising screw sizes in the mid nineteenth century, a 40mm countersunk rivet has meant exactly the same thing, across the years and across the world. (OK so Whitworth was working in imperial measures but I hope you get the point).
We, on the other hand, find ourselves constantly battling with conflicting meanings for apparently identical values. (How many definitions of Customer are there in your organisation?).
To be fair to the authors, they acknowledge this in their sections on Canonical Modelling, making it clear that there is not one model but different models in different domains. However it worries me that without providing the tools to explore and resolve these conflicts this might easily become one of those underestimated and misunderstood issues in implementation. As we saw with the blog from the week before last on the troubled Health Service implementation, a technically perfect implementation but with a data set that is mistrusted, will fail just as surely as a technically flawed one. In most cases it will fail in a more drawn out, painful and expensive manner because it looks as if it has worked and the technical side will defend the execution to the death. If we have a factory and it produces goods according to specification and quality then the end result must be fit for purpose mustn’t it? Well no, not in our area. Information, knowledge and meaning add whole new dimension where things can go wrong. You just do not have this problem on a production line.
How then does PDM fit into the picture? Firstly PDM is very strong on business engagement and resolving semantic issues. It has a richer and more clearly defined set of Data Stakeholders and methods of engaging with them.
Secondly, as I have often said in the past, Data Migration is a project not a process. It has a start, a middle and an end. If you don’t reach that end and turn off the legacy to plan then your migration joins the 80% that fail. A process can have feed back loops, a project can’t. An integration can be tuned post implementation, a Data Migration can’t. Once the data is in the target and the legacy is decommissioned your business has to live with what you’ve got. Now this does not mean that you can’t reflect on the processes you use in your Data Migrations. That is how we have progressed to PDMv2, but each migration is a self contained one way journey.
One of the case studies in the book looks at a Migration Factory approach. The company involved is a frequent purchaser of other companies. It made sense for them to set up a Migration Factory to hone their internal practice and maximise re-use of metadata etc. However for each company they purchase, the exercise is a one off, unique experience. So, if you like, the right had end of the equation is repeated, the left hand end is not.
This, plus the customer metaphor, shows us where the De-Militarised Zone (DMZ) is within Lean.
PDM does not view the business as Customers but as fellow workers to be co-opted onto the virtual team we need to make a Data Migration work. Our metaphor is one of a quest rather than a commercial transaction. In Lean there is a fairly traditional DMZ between the Factory and the Customers they service. Within the DMZ the lean machine increases value by increasing quality and baring down on waste, thereby producing a better product for each successive Customer. Outside of the DM, the rest of the business have a contractual relationship with the factory, providing semantic understanding, prioritising Data Quality issues etc and receiving in return a new system properly populated with data. Intelligence, understanding and meaning remain with the business and are provided as inputs to the Factory - part of their raw materials.
This is, of course, exactly the kind of relationship most of us, on large Data Migration exercises, experience where a Systems Integrator (SI) is involved.
Therefore wouldn’t it be perfect if our partner SI’s or internal IT Functions were to universally adopt Lean Integration and we provided them with a set of standard interfaces into and out of the DMZ tuned to Lean and PDM? That way we wouldn’t have to re-invent them each time (which of course aligns with the waste of Over Processing)?
Lean offers a perfect prescription for Systems Integrators who will be repeating similar migrations over and over again. PDM offers the same for the one off activities that are on the Legacy side of the equation.
It comes down to point of view or, if you prefer, a semantic conflict. For the business Customer, a large migration is a once in a business lifetime experience. A project not a process. From the SI perspective it is a repeatable set of events. A process not a project. Using PDM and its concept of the DMZ bridges these contradictory views.
Finally, however, if you are an internal IT department, grappling unaided with a large migration that will be a genuine once in a business life time activity, the authors of Lean Integration accept that Lean has nothing to offer. Then only PDM will help.
Johny Morris
