Tuesday, March 12, 2013

Pilot Testing: Real World Problems (and their Solutions)

Dear CMMI Appraiser, I find your posts about pilot testing very useful. What do we do when we don’t have enough projects using the process we want to test? ~ Kevin D.

Dear Kevin, thanks for making Ask the CMMI Appraiser a part of your day! All posts on pilot testing come from Laura Adkins, a Senior CMMI Consultant with Broadsword. Laura does amazing work with companies that are piloting processes that are strengthened by the CMMI, throughout North America. Take it away, Laura! ~ The CMMI Appraiser

 Thank you, CMMI Appraiser!

Kevin, great question! We always love to hear from companies like yours that are applying our piloting tips and techniques in the real world. Since we all live in the real world, I will take this opportunity to answer your question by listing the top three problems (and their solutions) that companies are telling us about.


Top 3 Problems with Pilot Testing:

1) Not enough projects – As you pointed out, Kevin, sometimes organizations do not have enough projects of the right type, size, or complexity to pilot the process they want to test.

2) Not enough time – Organizations are in a hurry. They may feel they don’t have the time to pilot processes. Management may not see the benefits of performing pilot testing.

3) Misaligned projects – Schedules don’t align, where projects are already past the point where you need them to be, or the Lifecycle Phases aren’t measuring up the way you’d want them to be for piloting specific processes.

Top 3 Solutions for Pilot Testing:

1) Not enough projects? It’s important to analyze the project pipeline ahead of time and anticipate what is coming up. That way, if you know a viable project is available within a couple weeks of when you want to pilot test, you can adjust your piloting timeline so that it captures that project at the right time. You might also consider widening the scope of the pilot to other parts of the organization that are not going to be a part of your CMMI appraisal. For example, you can take some other business unit in the company, and have them test the processes. It’s better to have some feedback than none at all.

2) Not enough time? We find the best approach is to build a process release plan, up front, that includes time for pilot testing processes. It should be clear to management that, when you take the extra time to pilot processes and update the processes with feedback from the pilot, you are preventing rework down the road, and eliminating the need to deploy a process a second time. This saves time and effort, and avoids confusion.

3) Misalignment? We recommend that you take what you can get, even if it means piloting a project that’s already in progress. For example, a project may already be past a certain point, but there will still be other phases that you can learn from. It also may make sense to review a project that is recently finished. In that case, you can go through and analyze what they did, and how your new processes would have affected them, and identify some lessons learned that way. It’s kind of like an on-paper pilot testing. If it’s recent, you can find value in that.

I hope this information is useful, Kevin. Please let us know if you encounter any other challenges with piloting your processes. We’re here to help!

Like this blog? Forward to your nearest engineering or software exec!

Laura Adkins is a Senior CMMI Consultant with Broadsword Solutions Corporation. She has years of real world experience using pilot testing strategies and tactics to help her clients achieve their goals. Laura also uses the CMMI, in partnership with her clients, to set-up, monitor, and sustain process improvement programs.

Visit www.broadswordsolutions.com for more information about running a successful CMMI program.

No comments: