Theory is great – but what is piloting like in the real world?

Hey, CMMI Appraiser – a few months ago, we participated in your Introduction to CMMI class, and have been enjoying the theory behind piloting processes here at Ask the CMMI Appraiser. But we’re wondering if you would address what piloting is like in the real world? ~ Manley A.

Hey, Manley – great question! Let me pass it along to Laura Adkins, a Senior CMMI Consultant with Broadsword, who does amazing hands-on work with companies who are piloting processes based on the CMMI. Take it away, Laura! ~ The CMMI Appraiser 

Thank you, CMMI Appraiser!

Manley, you are correct that the theory of pilot testing only takes you so far. I’d be happy to discuss what pilot testing is like in the real world.

We here at Broadsword pilot all of the processes we develop using our AgileCMMI methodology, before deploying them. We use a four-step sequence that looks like this:

It’s important to note that most companies are focused on designing and deploying their processes. That’s only half the battle. What we are asking them to do is add two steps between Design and Deploy. Those two steps are: Pilot and Update.

Piloting and updating before you deploy your new processes is a real world scenario. As you probably know from the CMMI training you took, Broadsword uses the methodology we developed, AgileCMMI approach, which calls for us to work incrementally and iteratively. When designing, piloting, updating and deploying processes, we use a 12-week cycle.

For your CMMI-based processes, if you follow the same time cycle, you would test them for six weeks, then make updates for six weeks. This gives you one Quarter for designing and one Quarter for piloting and updating before you deploy. By adding extra time into the way that your new processes are rolled out, you increase their likelihood of success.

As a practical matter, to develop processes during the design phase, you should pull from a group of users. We think it’s important that the people who are actually going to be using the processes help develop them.

Now, once the piloting completes, and you’ve had projects using these processes for a period of time (6 weeks or so), you need to update process assets using your experience.

To do that, we recommend holding a retrospective on each area that was pilot tested.

Keep in mind, when you develop processes, you might have multiple process areas – for example, requirements and project planning. Therefore you would need to ensure that a retrospective covers each process area that was in the scope of the pilot.

Use the retrospective to talk about strengths, weaknesses, identify any lessons learned, and capture feedback for improvement.

Once you have that meeting with the participating projects, you will reconvene with the user group that developed your processes. Evaluate with them all of the feedback that came back from the pilot testing. As a group, you would then prioritize the process improvements, and select which ones you would like to go ahead and make changes to.

The final step before deploying is updating.

In the Update stage, you will make your changes to the process assets and update any training material. Communicate back to the pilot participants about changes that you made. This allows them to understand how much of their feedback was used, and how their feedback impacted the processes.

Finally you are ready to deploy. Just release your processes into your environment as you normally would.

Hope this helps, Manley! Let me know how it goes.

Like this blog? Forward to your nearest engineering or software exec!

Laura Adkins is a Senior CMMI Consultant with Broadsword Solutions Corporation. She has years of real world experience using pilot testing strategies and tactics to help her clients achieve their goals. Laura also uses the CMMI, in partnership with her clients, to set-up, monitor, and sustain process improvement programs.

Visit www.broadswordsolutions.com for more information about running a successful CMMI program.