Can we "self-assess" and declare ourselves ML2?

We are currently working through the ISO 9000 registration process and are confronted with the additional need for CMMI Level 2.  I’m told by some of my peers that all I have to do is “self assess” and declare myself CMMI Level 2.  That seems sort of absurd.  I’ve been told by another friend that we need to self assess and have a SCAMPI from a certified assessor who in turn can issue a certificate.  What is the correct path?

Sure!  I declared myself a millionaire last week . . . .  as long as it's my own private fantasy that's no problem, but if others knew about it they'd start to question my sanity!

A CMMI SCAMPI Appraisal is a rigorous event that can only be conducted by an external licensed SCAMPI Lead Appraiser who is authorized by the SEI.  The appraisal is conducted with a team of people made up of both internal and external resources who must complete both the "Introduction to CMMI" class and the SCAMPI Appraisal Team Training class.

Typically organizations conduct a SCAMPI "C" appraisal to identify the gaps.  This is a "non-binding" event that does not become part of the public record.  Then a SCAMPI "A" is conducted.  The results of the SCAMPI A become public record regardless of the result.

Your ISO9000 prep will help a little - especially in the area of documented processes and policies.  But a CMMI SCAMPI appraisal is much more detailed than an ISO9000 audit, as we seek to verify actual process performance.

Your best bet is to start by contacting a licensed Lead Appraiser (www.broadswordsolutions.com) and engaging him/her for a consultation to help develop a plan for succeeding.

Good luck!

www.broadswordsolutions.com

We think GP2.8 is about reviewing and discussing the process. Do you agree?

We've had a disagreement with our Lead Appraiser on GP2.8.  She says every instance of GP2.8 needs to have a metric.  We think "monitor and control the x process" means day-to-day review and discussion of the process in general.  What say you?

I don’t think I agree with you.  That said, I don't think I agree with your LA either.  I started my CMMI journey about the way you’ve described . . . that GP2.8 is about discussing and reviewing the process.  My thinking on GP2.8 has evolved over the last several years to believe that GP2.8 represents potential “process levers” that can be tuned to improve performance.  I have inquired about this several times with folks at the SEI and the clear and consistent message I have received is YES, measures are appropriate here (but not the only thing that might be used). 

As a proponent and advocate of agile methods I’m not  implying that I favor anything “heavy” but I am saying the a healthy dose of process measures is very useful for any organization. That said, asking for a measure for every one is unreasonable, and it's not required by the CMMI.

I’m not clear on how I would “monitor AND control” something as complex as an engineering process by only  having “day to day” discussions about it.  This can be appropriate for some things, but across the 18 process areas in ML3, there are many opportunities to measure process performance related to productivity and quality (among other things) in these PAs. 

Furthermore, if you delve into OPP (or any of the HM PAs) you realize quickly that it is these very measures that give you the most valuable information about process performance – enabling you to make course-corrections to your process to improve performance.  Lacking that, there is way to understand what needs to be improved!

M&A is the PA that you would use to manage and execute those GP2.8 measures.  M&A is simply an infrastructure PA, which is why it is described in the model as a “support” PA.  It does not speak at all to “what” to measure, only how to build and execute a measurement infrastructure.  M&A must be “fed” by measures that are the result of process execution, so the other process areas are M&As “customer.”

 So, IMHO, I stand by my original assertion – GP2.8 exists to help us understand process performance, and measurement is an effective way to do this.  The trick, and I fully advocate this, is to keep it light and useful.

www.broadswordsolutions.com

We are a ML3 company. Can we go directly for Maturity Level Five?

Our company achieved ML3 two years ago.  Can we go directly to Maturity Level Five, and skip ML4?

Let's differentiate between SCAMPI "A" Appraisals and the process improvement work that needs to take place to be "performing at a level of maturity."

You can always skip an appraisal, so the short answer is "you could" go directly to a maturity level five SCAMPI A appraisal.

The longer answer lies in the realization that all of the work in the ML4 PA's (OPP and QPM) will STILL have to happen because CAR and OID (ML5 PAs) depend upon a statistical understanding of process performance that includes Process Performance Baselines and Process Performance Models.

There are examples of companies that have developed robust satistical process control processes, including PPBs and PPMs, and have used them to quantitatively manage projects, but have chosen to not conduct a ML4 SCAMPI Appraisal against those processes.  Then they develop and implement processes for OID and CAR, and eventually conduct a ML5 SCAMPI "A" appraisal.

But I'm not sure why you would want to do this.  It seems like a risky strategy to me - what if you conduct your ML5 appraisal and all of work you've done for OPP and QPM is inadequate?  What if you misundestood the very specialized content of the HM process areas?  What if your data set is statistically invalid?

If you're trying to do this to save money on the appraisal, I would argue that the cost of the SCAMPI A is insignificant compared to the effort you are putting towards achieving high maturity performance, and in the grand scheme of things it's not worth the extra risk.

An additional consideration lies in the concept of organizational learning.  Your OU doesn't learn all at once, but learning takes place incrementaly.  Why not achieve ML4, learn from the data, become experts at SPC for software, and then dive in ML5?  Your chances of getting real value from the PA's is far greater if you take this incremental approach.

www.broadswordsolutions.com

If you only attend ONE CONFERENCE next year, make it this one!

If you're in the CMMI business, or interested in CMMI in general, you should plan on attending the SEI's SEPG 2009 in San Jose', CA in March of 2009.

The theme is "Perform at a Higher Level"

http://www.sei.cmu.edu/sepgna/2009/

Upcoming conferences - come hear Jeff speak!

Dear Readers,

Come join me at some of these great conferences where I'll be speaking in the next few months.

- NOVEMBER -

Great Lakes Software Excellence Conference

"Encapsulated Process Objects"

Grand Rapids, MI

November 5th, 2008

www.GL-SEC.org

 

Agile Development Practices

"Agile CMMI"

Orlando, FL

November 12th, 2008

www.sqe.com/agiledevpractices

 

National Defense Industry Association (NDIA) CMMI User Group

"Encapsulated Process Objects: How Object Orientation and Agility can supercharge your process improvement program"

Denver, CO

November 17th - 20th

www.ndia.org

 

 - 2009 -

SEI's SEPG 2009

"MORE Notes from the Blogosphere"

"Agile and CMMI: Why not embrace both"

"Johnson Controls: an Agile CMMI experience report"

San Jose, CA

March 2009

www.sei.cmu.edu/sepgna2009

Should we select the staged or continuous representation of the CMMI?

I am gearing up to introduce CMMI to my company. Up until yesterday, I was convinced that I should select CMMI-DEV +IPPD using the Continuous strategy.  I just read recently another train of thought that indicated for software development improvements only, w should use the Staged strategy and add RM process area to it. 

If I choose continuous, will it create more difficulty for me when I begin improving other processes? Is it difficult to match capability levels to maturity levels?  I just don't want to do extra work and waste my companies time.

There is no "right" answer for which representation you should choose.  I primarily work with clients who select the "staged" representation (which results in a "maturity level"). But there are numerous examples of companies that choose the continuous representation (resulting in one or more capability levels by process area).

The decision to choose one or the other depends on the goals of the organization.  If you want to work on improving a large "swath" of your organization - from project management to requirements to engineering - the staged representation is appropriate.  If you need to achieve a maturity level of CMMI (say ML2 or ML3) then staged is also appropriate.  If you desire to pick and choose amongst improvements, say, only work on metrics now, and requirements later, then the continuous representation is appropriate.

If you achieve Capability Level Two in all of the Maturity Level Two process areas, then "conversion" from a set of capability levels to a single maturity level is simple.

I see the staged representation as a "fork-lift" for bringining a software organization up to a higher level of performance.  I see the continuous representation as a jack (like the one for your car) that raises performance for one or more parts of your organization.

The downside of staged is that the scope is large and is quite a bit of work.  The downside of continuous is that you'll have processes in your organization performing at all different levels and interfaces between groups will be difficult.

You decide!

Oh, but the way IPPD is optional . .  it's not that it's bad, if you have time it's a great set of practices, but it's an "addition" to the model and is not required.

www.broadswordsolutions.com

Could you explain the meaning behind VER SG2: Perform Peer Reviews

We're not sure we totally understand the meaning behind VER SG2 - Perform Peer Reviews.  Could you shed some light on it for us?

Sure, let me write something up, send it out, and have everyone review it.  After recieving their feeback I'll update the post and release it for everyone to . . . . hey wait!  I'm giving away the answer!!

Peer reviews are a proven method for identifying defects in all types of work products.  One of the common misperceptions of VER SG2 is that it is only performed as part of a linear "VER" process.  I see it as a "stand-alone" utility process that is used throughout a project to identify and drive out defects long before they become part of our product.

SP2.1 - Prepare for peer reviews involves identifying participants, selecting the appropriate "style" of peer review (Fagen inspection or something lighter perhaps), sending our materials, setting up the room, identifying roles, and so on.

SP2.2 - Conduct peer reviews is kind of obvious - conduct it PER THE PLAN (from SP2.1).  Identify and record issues, assign action items, and so on.

SP2.3 - Analyze Peer Review data involves understanding what was discovered during the peer review, both at a project and organizational level and taking appropriate action.

I would recommend you consider peer reviewing all plans, requirements, designs, code, and test results at a minimum.  The life you save may be your own!

www.broadswordsolutions.com

What is considered good evidence for IPM SP1.3?

What are some direct evidence work products for IPM SP1.3 (Establish the project’s work environment)?  What is a good  source for examples of direct and indirect evidence for CMMI Version 1.2 Process Areas and Practices?

There are a number of sources that provide examples of work products that will result when you perform a process based on a CMMI practice(s).  The first one that comes to mind is the CMMI book, or the CMMI technical report (free download from www.sei.cmu.edu/cmmi).

Another source is a SCAMPI PIID worksheet, of which there are many on the web for you to download.  These usually have some suggested work products, although keep in mind that you get what you pay for.

Either way, perhaps you should consider that you're thinking about this in reverse.  The artifacts don't drive the process, the process drives the artifacts.  If you've developed a useful process, it will have useful work products.

For IPM SP1.3, we often see safety and security procedures being implemented, hardware and software being delivered and installed, and contractor or new employee on-boarding being performed.  This usually results in a large amount of documentation that can be used as direct and indirect artifacts.

www.broadswordsolutions.com

And now for something completely different - the CMMI Song!

This has been around for awhile and I've been meaning to post the link.  If you want to have some fun, take a listen to the CMMI song.

https://bscw.sei.cmu.edu/pub/bscw.cgi/d226930/The%20CMMI%20Song%20(audio)

www.broadswordsolutions.com

Are sub-practices required for a CMMI Appraisal?

Are sub-practices located under specific practices expected during CMMI Appraisals (e.g. SAM SP2.4 there are seven sub-practices.  Are Appraisers expecting seven artifacts or artifacts that prove all seven?

The CMMI defines Goals as required, Practices as expected, and everything else as information. So no, you do not need to produce evidence at the sub-practice level - only at the Specific Practice level.

However, as "informative material," sub-practices embody the intention of the authors of the CMMI and further describes and clarifies the meaning of the practice.

It's good to read them and understand what the authors meant by each practice - but it doesn't mean you need to produce evidence for every one.

www.broadswordsolutions.com 

Can I get a list of questions we'll be asked during a SCAMPI Appraisal?

I'm driving my company to CMMi Level 3 certification. We are a very different company in the sense that we are a product development company and the final product is the combined output of electronics, mechanical and software.  Can you please give me some of the example questions, LA will ask for in the appraisal - especially in mechanical.

People often ask me this - some even complain to the SEI that there isn't a "question bank" available that every LA is supposed to ask.  Unfortunately, this betrays a lack of understanding about how the CMMI, and SCAMPI, is used.

In SCAMPI, process performance is verified through a number of means, with interviews being one of them.  There are also other means - documents, presentations, surveys, and others.  SCAMPI finds practices to be strong or weak based on the combination of available evidence.  Depending on the organization, we may not even ask a question about a certain practice if other evidence is available.

The questions we ask also are best if they're tailored to the organization's situation. For instance, if it's a company that makes microprocessors, questions about software may be invalid.  

Another instance is the need to tailor questions to fit the local culture.  If I'm appraising a small, agile organization I'm not going to fill my questions with references to "Change Control Boards," governance infrastructure," "SEPGs" and other words or phrases typical at large companies.  If it's a huge DOD contractor, I'm not going to talk about "sprints," "jolt cola" or skateboards either!

I do have a database of questions I ask, and the list is always growing just like I am always learning.

www.broadswordsolutions.com

How do we practically implement GP2.10

GP2.10 asks us to review status of the process with "higher level management."  Do we really have to review EVERY process with them every week?  I don't want them to micro-manage me that way.

Good question.  It's hard enough to get an upper-manager to take a phone call, let alone review "the process" with us. But think of it this way: the process is the ONLY tool management has to proactively run the business - they should be happy to do this! I know that isn't always practicle though.

I favor a "rotating strategy" where we cycle through the process at monthly meetings.  So in January we review, say, the processes associated with Project Management.  In February we might look at software development.  And so on.

Upper management doesn't HAVE to provide feedback, but it's much better if they do.  After all, these are the "levers" we need to successfully run this business.

www.broadswordsolutions.com

Is CAR applicable to the project or the organization?

Should CAR be applicable only at the organization level and not at the individual project level? I ask this question since CAR is towards resolving common causes which is coming from the QPM, OPP outputs.

The CMMI specifically assigns usage of CAR to the projects, and then tells us it COULD be an input into OID, the organizational PA used to identify and implement improvements.  CAR can address problems at the project level, and can be an input into OID ("Collect Improvement Proposals")

On the other hand, if the "problem" you're trying to fix is an organizational problem, such as say, the process not working, then there is no reason CAR couldn't be applied to the "process project."

www.broadswordsolutions.com 

Do you have a list of documents we need to achieve CMMI?

Hey Jeff!!

I need a help from you. Can you provide me the list of work products needed to achieve CMMI in our organization? 

Let me ask you this.  Should you reach your destination in a car by looking at the white line in the road through your rear-view mirror?  

This kind of "reverse engineering" of the CMMI was never really intended by the authors.  The documents you are referring should be the product produced from performing a process.  

So, if you conduct an estimate using "wide-band delphi" the work products, or "documents," you produce may include the three or more estimates developed by the assigned estimators, the list of tasks in a WBS, and the output of the formal estimate.

If I said, go produce an estimate, you might just sit at your desk and type one up without executing the process.  But you would have "the document."  Does that mean you performed the process? No, it means you typed in your best guess of the estimate.  And it definitely doesn't mean your CMMI ML2.

The CMMI does expect you to produce a "direct artifact" and an "indirect artifact" (in most cases).  This is not the same as saying that all you need is the artifact.  They are merely a consequence of the process being performed. You must also produce evidence that you actually followed a process.

There is a list of "suggested work products" in the CMMI book that you COULD use as a guide, but this list is highly dependant upon your company, methodology, and culture.  

www.broadswordsolutions.com