Do we need measures for every Process Area if we want to be Maturity Level Three?

Dear Appraiser,

When being appraised for CMMI Level 3 do we need to have one or more measures of EVERY PA or can we just measure the things we actually need to? ~Nabeel.

Nabeel,

Thank you for reading!  Interesting question.

Every-time some asks me this I brace myself for the many people who will write in to disagree with me!  This is a subject that people feel very strongly about, and I'm always afraid of starting a religious war….

So, let me go put on my armor, pick pick out my favorite sword, and I'll be right back….

….. OK, I'm back (albeit a little bit heavier)….

I think you're on to something when you say "things we actually need."  It's important that a company establish measures that reflect the information needs that are required to be a great company.  In fact, the CMMI explicitly states that you should "Develop and Sustain a Measurement Capability Used to Support Management Information Needs."

What does this mean?

It means that only you and your management can determine what your company can benefit from.  Not a consultant.  Not a model.  Not an instructor.  So, I'll agree *somewhat* with the statement implied by your question.

However…..

In order to fully grasp the intent of the CMMI, you need to understand that it was developed as a research project.  And that project examined the successful implementations of many projects at many companies, and came to some conclusions about patterns and practices.

In my experience, many companies (and their management), don't really have any idea WHAT their management information needs are.  As a CIO myself, I often learned about some new technique, measure, or process that I had not been using, only to say "Wow, that is something I REALLY need to do!"

So goes the CMMI.  It catalogues many of the attributes of a measurement "program" that were seen at other great companies, and makes suggestions for how you might use them.

Note that the CMMI doesn't tell you that you need to measure EVERYTHING.  It starts by asking the question, "what are your objectives?" (MA SP1.1).  This should drive everything.  Typical objectives are "projects need to be on time, on budget," "risks should be kept to a minimum," "growth should be at n%," recalls should be below n%," and so on.  The more you can quantify it, the better.

But the Process Areas also give you a hint into what you should measure.  Look at VER SP2.3 - which discusses analyzing the results of peer reviews.  That implies a measure.  Look at GP2.8, "Monitor and Control….", that implies one also (and appears in every process area).

Let's take that one on for a second.  You didn't say it, but I suspect GP2.8 is the one you are referring to when you asked your question. There are many Lead Appraisers who insist that this practice means you must have a measure for each process area.  This isn't exactly true.  The CMMI isn't a requirements document - it's supposed to be interpreted so that it provides the most value for the user. There was a time when this measurement requirement was the conventional approach (back in the SW-CMM days), and even early in the CMMI life-cycle it was carried over, but this was clarified in version 1.3 of the model to read:

"GP2.8 Monitor and Control the Process"

It goes on to say "…. can involve measuring appropriate attributes…."  Neither the  Goal (required) or the practice (expected) say "must."

The sub-practices provide additional data with phrases like "evaluate actual progress," "review accomplishments," "review activities," "identify problems," etc…

I would start by turning each instance of GP2.8 into a question: "How do I know our requirements change process is working well?"  "How do I know that our estimating process is effective?" And so on.

If the answer requires a metric, then so be it.  If you have another creative solution, then by all means use that too!

In other words, measurement should be PART of the toolset you use to monitor and control your process, and there is a very high likelihood that you will have a healthy collection of process performance measures, but it's not the only tool you can use.

In the end it's not important exactly what the mix of tools is - but I WILL ask you to answer that question when I appraise your organization, so be ready!

Good luck!

Like this blog? Forward to your nearest engineering or software exec!

Jeff Dalton is a Certified SCAMPI Lead Appraiser, Certified CMMI Instructor, ScrumMaster, author, and consultant with years of real-world experience with the CMMI in all types of organizations. Jeff pioneered agileCMMI, the leading methodology for incremental and iterative process improvement. He has taught thousands of students in CMMI trainings and has received an aggregate satisfaction score of 4.97 out of 5 from his students.

Visit www.broadswordsolutions.com for more information about engineering strategy, performance innovation , software process improvement and running a successful CMMI program.

To download eBooks about CMMI, visit Jeff’s Author Page on Amazon.