Does process pilot testing create too much overhead?

Hey, CMMI Appraiser – we are a small Web and Mobile solutions company in Virginia that is pursuing CMMI Level 3 for the marketing benefits. Here’s the problem: our CMMI consultant wants us to include pilot testing in our Program Plan for every single process we create. We only have 20 people! Doesn’t pilot testing create too much overhead? ~ Larry M.

Hey, Larry – great question! Let me pass it along to Laura Adkins, a Senior CMMI Consultant with Broadsword, who does amazing work with companies who are trying to evaluate the usefulness of pilot testing. ~ The CMMI Appraiser 

Thanks, CMMI Appraiser!

Larry, Broadsword works with all kinds of organizations, ranging from very small to very large. I can tell you, for all of them, regardless of size, pilot testing is an important factor in their success. They may not all like taking the time at first – but if they don’t do it, their results might be … disturbing.

So I agree with your CMMI consultant. Pilot testing the new processes you are developing within the CMMI is not too much overhead. On the contrary, it can save you money.

How? For one thing, pilot testing helps with ownership and acceptance of newly developed processes, no matter what they are. When you test a process with an audience before you deploy it, they are more likely to understand and embrace it. As a result, your CMMI adoption is more likely to succeed.

I know. It’s tempting to avoid pilot testing altogether. For the sake of expediency, you’d rather not even consult with process users. You just want your quality group to get to work designing processes. Then, as soon as they have developed large volumes of process assets, you want them to deploy the processes to all of your process users.

That’s great – but guess what? You may find that none of the new processes are being adopted, and the effort was a failure. What would that cost?

A better way is to think of the CMMI as a model for helping you go from being a good company to a great company. With this mindset, Broadsword recommends the following approach:

Commission small groups of employees to define processes that will improve their performance. Have these small groups design processes iteratively, so they will be “Just Enough, Not Too Much”. Before deploying the new processes, try them out on projects of varying sizes and types. Run a pilot, collect feedback and make changes before you deploy.

The cost savings of this approach far outweighs the expense, even for a small company like yours.

And yes, this goes for all of your processes. We recommend that you create a Program Plan that tells how often and how frequently you’re going to deploy processes. The Program Plan should definitely include some time for pilot testing. Plan to pilot all processes after you’ve developed them and before you’ve deployed them.

I’m glad you are adopting the CMMI, Larry, but keep this in mind. The CMMI is much more than a marketing tool. Used properly, it can serve as a framework for making your team more powerful and productive, and save you a ton of money. That’s why the CMMI Appraiser likes to call pilot testing “process underhead,” not process overhead.

Check back soon and I’ll go into more detail about pilot testing.

Like this blog? Forward to your nearest engineering or software exec!

Laura Adkins is a Senior CMMI Consultant with Broadsword Solutions Corporation. She has years of real world experience using pilot testing strategies and tactics to help her clients achieve their goals. Laura also uses the CMMI, in partnership with her clients, to set-up, monitor, and sustain process improvement programs.

Visit www.broadswordsolutions.com for more information about running a successful CMMI program.