While originally developed for the software industry, the standard has been widely used throughout the world for all sorts of engineering organisations. Now the SEI has produced a separate 'constellation' called 'CMMI for Acquisition' (CMMI-ACQ).
The stated goal is to do for the outsourcing world what the 'CMMI for Development' (CMMI-DEV) did for engineering. Peter Leeson, Q:Pit Ltd, asks is this a good idea and will CMMI-ACQ fill a gap in the outsourcing world, or is this just an academic exercise?
Outsourcing has become a major trend throughout the world, companies have understood that the 'not invented here' attitude is not allowing them to be truly competitive. Global communication has allowed for increased sharing in the work load and the benefiting in various ways of international facilities.
Temptation
The interest in using the significantly cheaper labour that can be found throughout Asia and in the new Europe - meaning the central and eastern European countries that were formally within the Soviet Union's group of satellite countries - is overwhelming.
It is difficult to resist the temptation of being able to use the skilled workforce that has not been spoilt by years of social actions, trade unions, comfortable lifestyles, a culture of leisure and modern amenities, just plain hardworking, badly paid people.
Additionally, it makes sense that the companies take benefit of more specialised developers. Why invest in expensive software development teams, when that is not your business? It makes sense to ask suppliers to develop products that are not part of your key business. If you are a bank, you should not be required to be a software development company or phone support centre.
Finally, outsourcing developments allows sharing responsibility for the quality of the resulting products. If the product is bad quality, you can require your supplier to correct the issue, to improve the product. If the product is good quality, your customers (and shareholders) will reap the benefits. Apparently, you can outsource the blame while keeping the credit. Surely this must be the best of all worlds?
Failures
And yet, most companies are showing increased disappointment with their outsourcing experience. They are finding that the costs are significantly higher than estimated, and the quality of the products delivered does not satisfy their customers.
When they complain, they find that the sub-contractor refuses to accept the blame and efficiently points to minor points in the contract or the requirements documents that appear to support their position. The reasons for failure are multiple and every case has a different excuse.
Many companies do not understand the increased cost related to working with teams across borders, time-zones and language barriers and the need to put in the additional management and technical effort in order to ensure that there is a clear understanding of needs and expectations on both sides and ensure that the cheap workforce is actually able to meet those needs.
Working with outsourced staff means that you are not in control of what they are doing on a daily basis, and you need to build up a certain amount of trust before you can truly collaborate. There is, therefore, a need for direct communication (travel) staff that are permanently on monitoring duty, ensuring that progress and quality is being achieved on a daily basis.
Differences in culture between organisations and nations form a minefield that is too often under-estimated. Differences related to national pride, prejudices, racism are completely ignored. Levels of respect of authority or customer satisfaction are not considered.
Some organisations are proud to deliver high-quality products, others are more interested in selling whatever they can to the highest bidder - these differences can have a huge impact on customer-supplier relations or the quality of the products delivered.
Frequently today, we assume that modern technology has replaced the need for face-to-face, in-person communication. As a result, we trust in emails and conference calls to set up and negotiate contracts.
These media do not truly allow the participants to understand those with whom they are working and the importance of being able to physically shake hands with work-partners is too frequently discarded. And yet, a team that has a meal together, that goes out, has a team-building exercise, is seen as more productive than others, hence the amount of money spent on these activities locally.
When working internally, should a requirement or statement be unclear or confusing, it is normal for colleagues to walk over to each others' office (or cubicle) to ask for clarifications. When working with people in a different location, it gets a lot more difficult when working with people in different time-zones.
When office-workers arrive at their work at 9.00 am in the United Kingdom, the Chinese are finishing their day as it is 5.00 pm. The weekend in Saudi Arabia is on Thursday-Friday, leaving only three days overlapping during the work-week. Communication needs to be managed carefully for these reasons.
The communication issue impacts enormously on the understanding and the consequences of requirements that might be misunderstood or misinterpreted. Sod's law states that if your sub-contractors can misinterpret anything, they will. The result will be a product that corresponds to the words on the requirements documentation, but not in a product that can be used by your customers.
Verification and validation criteria are often left out of requirements statements and contractual agreements. As a consequence of this, products are delivered with unacceptable levels of response time, typing errors in screens, and other issues that would be identified locally but are not really required contractually, so are not implemented.
Supplier selection using CMMI
Now, you will be expecting me to say at this point that all these issues can be solved by ensuring that your suppliers have been appropriately appraised using the CMMI (or ISO or some other standard). But we know that does not work. The question to be considered is why it does not work as expected.
Basic CMMI concepts
A quick reminder for those who are not familiar with the CMMI - if you know the model, please skip this section.
The CMMI (Capability Maturity Model Integration) is a model that seeks to identify key processes or practices that should be in place and demonstrate that an organisation knows what it is doing and what it can or needs to do in order to deliver something that corresponds to its promises.
The original idea is to be able to analyse an organisation's products and ways of working in order to determine if they have made a proposal or bid based on an understanding of what the customer wants and what they are in state to deliver, or whether they have made a rash promise to deliver something they cannot.
The CMMI is a registered trademark of the Carnegie Mellon University, developed by their software engineering institute division. It has become, over the years, an international standard of good practice and quality.
The basic concept is that industry best-practices are placed on a scale requiring that base practices be implemented systematically throughout an organisation allowing for the establishment of good foundations for the more complex practices. The scale ensures that the processes are implemented in order to ensure
- at maturity level 2, basic project management is in place, as well as a number of key practices seeking to identify what needs to be done, what is being done and what is producing results that facilitate the organisational goals;
- at maturity level 3, lessons are learnt from level 2 and local best practices are shared across the organisation - this includes an integration of management and engineering practices as well as key support and improvement practices;
- at maturity level 4, key processes (or sub-processes) for the organisational strategy and goals are placed under statistical control, ensuring that the results are constant and predictable;
- at maturity level 5, those statistics are being used in order to continuously improve the practices by systematically performing defect avoidance activities.
Logically, one would assume that a maturity level 5 organisation can provide systematic, predictable and measurable added-value to all their products, ensuring systematic customer satisfaction.
Issues
Suppliers with high maturity levels still appear to regularly deliver products that do not satisfy their customers. There are a number of reasons for this and it is important to understand that a blind belief that CMMI appraisal results in guaranteed quality is not to be recommended. The reasons that apparently mature organisations deliver products that are not as good as one might be led to believe include:
- the basic premise that maturity levels do not guarantee the performance of the next project, but only indicate the potential;
- many organisations are not seeking the quality and improvement that one would expect from a CMMI organisation, but are only chasing the rating for advertising purposes, sadly many CMMI service providers are willing to provide the rating requested for a reasonable fee;
- organisations are not simple: not every part, not every site, not every department, not every team should be considered at the same level; frequently the communication about an appraisal result may be confused or deliberately misleading on the coverage;
- organisations, seeking to demonstrate their maturity, choose to only let their best projects be available for an appraisal - and appraisers lack the commercial courage to challenge their paying customers' choices or restrictions;
- mature organisations are mature based on an understanding of their business objectives (high-maturity organisations are even required to demonstrate that their processes focus on those business objectives), however, those business objectives may be very different from their customer's objectives (e.g. you want quality, your supplier wants profitability);
- and many more...
Using CMMI on your suppliers
When using products, such as CMMI to select potential suppliers, a few elementary practices need to be put in place.
Understand the process capability
What are the needs for process capability of your programme or project? What are the key elements that you need to ensure the supplier can perform effectively and efficiently?
Understanding your dependence on the supplier's key processes should help you to identify what you are looking for. You might find that you are not really interested in a supplier that is rated at maturity level 4 or 5, provided they can... But, first, you need to understand what the things are that they need to be able to do for you.
This means you need a clear understanding of what you are requesting your supplier to perform: what is the complexity or the size of the project you are outsourcing? What history does the supplier have in this area? How stable are your requirements? How much do you understand your own marketplace and customers' needs? What are the restrictions you need to place on schedule or funding issues?
Understanding these issues should allow you to identify the areas in which the supplier needs to be able to demonstrate capabilities; these can then be deliberately checked, either by requesting to review specific areas, practices or processes, or perform a focused assessment of the supplier's abilities.
Leverage the supplier's capabilities
Understanding the needs for processes and capabilities will then allow the supplier to truly understand the solutions and possibilities being offered by potential suppliers. This should be implemented during the solicitation phase.
Rather than trusting the supplier to have had an appraisal and believing that they have been appraised as satisfying some standard or model at some point in time, research should be performed. The customer can request for detailed findings from the appraisal, including a list of the weaknesses.
Other options include requirements as to the management and monitoring of activities. It is possible to require that a supplier provide details as to how the processes or practices will be developed, improved or managed throughout the life of the relationship.
This would start with requesting historical data, improvement programmes, etc; then, requesting an integrated master plan demonstrating the integration of engineering, service, management, quality and improvement programmes throughout the project's life.
Formal appraisals can be conducted during the solicitation phase, or at the time of the awarding the contract. Finally, it is possible to include financial rewards and penalties based on the results of these appraisals.
Monitor the supplier's process performance
Once the contract has been established and the work has started, too many organisations sit back and wait to hear. You need to understand that you are on the same team as your supplier; you are both seeking to deliver the best possible results.
As a consequence, it is highly recommended that a formal collaboration agreement be established between the customer and the supplier, during which the customer has clear and complete visibility into what is being done and how it is being managed.
The acquiring organisation can request to review a number of artefacts and metrics throughout the life of the collaboration, but should also consider how it can help the supplier improve their quality and efficiency.
Again, you are both seeking to deliver the best possible result, so if the quality of your supplier's work is improving, you will directly benefit from that improvement. Likewise, if your customers are seeking to help you improve your work practices, why would you refuse that level of assistance?
Naturally, ongoing reviews of progress, risks, quality activities, etc should be an integral part of the contract.
CMMI for acquisition
CMMI for Acquisition is a recent add-on (constellation) in the CMMI family. It aims at assisting organisations to understand how best to implement the practices, such as those that have been explained here-above.
Based on the understanding that 20 to 25 per cent of all outsourced projects fail within the first two years and nearly 50 per cent fail in five years, this model seeks to bring together, under the familiar and proven structure of the CMMI models, the practices that need to be implemented to improve the success rate.
All CMMI practices focus clearly on the acquirer's practices, and not on the supplier's practices. These practices are directly and clearly aimed at identifying the practices required to ensure appropriate levels of management of the outsourcing projects and the suppliers.
A quick overview of some of the key areas follows.
Project planning
The planning of the project, from the acquirer's side, must necessarily start with clearly identifying the acquisition strategy to be implemented. This strategy determines the key features that will determine what and how outsourcing and acquisition will be performed.
It will be used in the ensuing practices and activities to determine how the requirements will be allocated, how the solicitation will be handled, what are the key criteria to be used in the selection and management of suppliers, etc.
The development of the project plan should start early in order to determine the manner in which the outsourcing solicitation and management will be organised, but also to start, from the beginning, the planning of the transition of the acquired products and services into operations and support. Appropriate levels of risks related to the outsourcing and management activities need to be identified, documented and managed from the beginning.
It is indispensable in this phase to ensure that the appropriate stakeholders have been involved in generating the plan and that commitment is reached as to its implementation.
Integrated project management
The idea of integrated project management is to ensure that best practices and lessons learnt are shared throughout the organisation. As such, it is expected that the project's planning and management activities are based on recognised best practices and the results are used to improve the organisational practices, each and every time they are used. Lessons are learnt and are actively fed back into the system so that they can be used next time around.
The integrated project management approach seeks also to ensure that the engineering and management practices are not separated, but that all activities and stakeholders are appropriately managed, and critical dependencies are clearly identified, documented and managed.
Acquisition requirements development
Within a normal development environment, a number of phases are established in which the customer's (and other stakeholders) needs, expectations and constraints are translated into clear customer requirements, then into product specifications documenting how practices will be implemented in order to satisfy the customer. These are then decomposed into component or sub-product requirements.
Within the acquisition context, the same basic practices are performed. Additionally, the requirements are broken down into contractual requirements; in other words, what do you expect your suppliers to be able to deliver.
These contractual requirements need to be detailed in such a manner that, not only can they be used for the design and development of the product, but they can also be used to establish the solicitation material and the acceptance criteria. This is performed in accordance with the acquisition strategy.
Solicitation and supplier agreement development
Preparing the solicitation package, distributing it and reviewing the results based on clearly documented criteria are, naturally, a key part of any outsourcing procedure. In this procedure, a clear understanding of the purchasing organisation's needs and expectations is defined based on the acquisition strategy and the contractual requirements.
The resulting activity of sending solicitation material, receiving responses and evaluating proposed solutions are key in preparing for the negotiation phase with the potential suppliers.
The whole process is completed with the commitment to an established agreement.
Agreement management
Once the agreement has been established, it has to be clearly managed. This means that it is not acceptable for a purchasing organisation to sit back and wait for a product or service to be delivered, but there is a clear need to ensure, on an ongoing basis that the activities are being performed and are progressing as stated in the agreement.
Key processes are to be monitored, depending on their impact on the potential quality of the product or service being delivered. The management of the agreement is an activity that starts as soon as the agreement is established and through the life of the collaboration, up to and including the acceptance of the product and services and the finalisation of the supplier's invoices.
Requirements management
Naturally, as in all projects, requirements will change as soon as the agreement is formalised. It is, therefore, necessary for the purchasing organisation to ensure that the requirements are being managed appropriately and that all parties involved have a clear and common understanding of what will be delivered.
Traceability from the original expression of need to the requirement and onto the final product and service needs to be maintained at all times in order to ensure that the product is progressing as expected. All changes are correctly identified, analysed and documented.
Risk management
The risks related to the outsourcing effort need to be actively managed and appropriate risk-reduction activities need to be performed in accordance with the supplier.
Project monitoring and control
Project monitoring needs to include, in addition to the usual project management activities, a clear and continued monitoring of the supplier's progress and performance with appropriate reviews and corrective actions. Additionally, the control of the facilities needed and provided for the transition to operations is required by the model.
Acquisition technical management
The proposed and developed designs and products need to be regularly managed in order to ensure that the final product will actually correspond to expectations. It is expected that the technical solution and interfaces will be reviewed and managed, as well as having formal qualitative reviews of selected key products and services, at every level of the development life cycle. These reviews will be performed with the supplier throughout the lifecycle, ensuring that an appropriate solution is proposed, selected and developed.
Acquisition verification and validation
V&V activities focus on ensuring that the product corresponds to the requirements and will satisfy the customer or user when placed in the intended live environment. The V&V activities include reviews and testing of the products.
A particular focus is placed within this model in ensuring that the acquirer's products (documents, solicitation material, requirements, plans, etc) are appropriately verified. The verification and validation of the supplier's products, as performed in the acceptance phase, are also naturally included in this approach.
Conclusion
Will this mode make you succeed in all things? No! Of course not, it is only a model. The real work needs to be done by the people implementing the model's recommendations. That means that we expect clear policies produced requiring that all outsourcing work be managed according to appropriate rules and practices, and produce the documentation required to ensure the success of the outsourced projects.
In addition to this, the people doing the work need to have the appropriate levels of responsibility and resources (including tools, time, budget, people and others), activities are to be managed and monitored; measurements of success and lessons are collected and shared.
The model, however, does recommend (or require if you wish to be appraised officially at some level) a number of key practices. These include:
- a clear understanding of the needs and requirements;
- continued management of the supplier;
- active collaboration between supplier and customer;
- clearly defined criteria and practices for the formal acceptance of the resulting product or service.
