We're a ML3 company trying for ML4. But we don't understand GP3.2

I’m EPG practitioner of ML3 company and we would like to achieve ML 4. But, I’m not sure for interpreting GP 3.2 in the process. We are so confused whether every process has a measurement metric included in MA.

Because GP 3.2 specifies “collect work product, measures, measurement results, and improvement information derived from planning and performing the process …”.

Could you give me some advice for this?

The more cynical of my Lead Appraiser colleagues might ask “how did you get to ML3 without knowing this?” But I won’t do that. I can think of MANY reasons why this may have occurred!

GP3.2 exists in the model to codify the need to proactively gather information from the project’s use of your process that could be used to improve performance for future projects. This could be “reuse” data, such as useful templates or examples. It could be useful measures that provide the NEXT project with information that would help them. It also could be “lessons learned” gathered from performing retrospectives, post-mortems, and reviews of what worked and what didn’t.

So GP3.2 doesn’t require a metric for every process area, and the Process Area “MA” certainly doesn’t require this.

However, if you’re planning on moving to ML4 you will need to be measuring the performance of selected sub-processes. And to do this you will want to have some granular metrics about the performance of specific sub-processes. One practice that supports this is GP2.8, my favorite, because it requires you to consider these types of measures WAY back in ML2. These measures should help you identify “process levers” that you can manipulate to change the results of a process.

www.broadswordsolutions.com

Follow AskTheCMMIAppraiser on Twitter

To receive Tweets set your Twitter account to follow "CMMIAppraiser."  You'll get daily tips and commentary about the CMMI - all for free!

www.broadswordsolutions.com

If we don't use Process Action Teams (PATs) will we fail an appraisal?

It may seem like a silly question, but it is important for me to understand. I’m not opposed to PATs, but I just need to know if without PATs we will fail the appraisal. We are going for CMMI Appraisal Level 2.

Thanks for your question.  This business just gets curiouser and curiouser (to borrow a word from Winnie the Pooh).

For the benefit of our readers, a Process Action Team, or PAT, is a group of individuals who focus on developing, deploying, and maintaining one or more processes within the organization's set of standard processes.  Here at the Broadsword labs we call them "SIGs" or Special Interest Groups because, well, we do most things differently around here!

Let's separate out your process improvement program and your Appraisal.  An appraisal is nothing more than a user acceptance test to determine if your processes and it's performance have reached ML2 performance.  These are two completely different events.  Your process improvement program is something that you execute in order to develop valuable and useful processes that give you that extra competitive edge, or if your like a lot of my clients, just helps reduce the chaos.  You might use PATs (or SIGs) to help you do that.  You often see SIGs as part of a ML3 effort as a result of executing the OPF and OPD processes.

But you're not doing that.  You're still going for ML2.  While I might recommend SIGs as part of your program at ML2, I would not be appraising their existence or their performance as part of any ML2 appraisal.  There is NOTHING in the model that mandates that PATs be used.

So the short answer is "NO" you don't need PATs for your ML2 appraisal.

Hope that helps!

www.broadswordsolutions.com

How do we apply TS and PI to a pure testing environment?

We provide a pure testing service for one of our clients and we are wondering how to apply TS and PI to these activities.

We are going for CMMI Re-Appraisal (Level 3) during the last part of this month and our Lead Appraiser has clearly said that all process areas, except SAM and those at the organizational level are applicable to even these projects. He wants us to map the direct & indirect evidence against the SP's for TS and PI in the PIID sheet for these two projects.

How does one do that for a projects that is just Testing projects?

Excellent question!  The CMMI, especially the PA's Technical Solutions and Product Integration, are steeped in the language of software development, and it's not always obvious how these might apply to another discipline, such as testing (or paper clip distribution for that matter!)

But first I must digress.  SHAME ON YOUR LEAD APPRAISER for not providing you with some guidance!  Your original message (which I edited because it's my blog not yours) indicated he was insisting on certain things without providing you with some guidance and interpetation of the model.  He should be ashamed.  We're not auditors - the SEI clearly states that a SCAMPI Appraisal is a COLLABORATIVE event, not an audit.  While he can't do the work for you, he should be helping to guide you.  'nuff said.

The good news is it's entirely possible and simple to apply to these two PAs to your situation.

TS: What testing alternatives did you explore?  Did you use a purchased tool, build your own, or reuse something from another project?  Did you write test cases (code)?  Did you reuse them from other projects?  Did you create a test plan with instructions?  I bet you did a lot of this, and these are directly related to practices in TS.

PI: Did you set up and environment for testing?  If you do load, integration, and performance testing you must have determined sequence, developed interfaces. and so on right?  Well, these are good examples of tasks in the PI process area.  By the way, it's some of the SAME evidence you might provide for VAL and VER.

Best of luck on your appraisal!

Jeff speaking at SEPG 2009 in San Jose, CA

The SEI's annual conference, SEPG 2009 is coming soon in San Jose, CA.  SEPG is the big one, the place where process geeks like me and you get to come together, network, learn new things, and make jokes that only we will understand ("so a PPQA auditor comes into a bar....").

Taking place March 23-26, the conference draws 1500 of the world's best practitioners of process improvement and features dozens of great speakers including many of my favorites like Pat O'Tool, Steve Masters, Mike Konrad, David Anderson, Hillel Glazer, and Rick Hefner.

Yours truly will be speaking at least three times: a re-dux of last years #1 presentation (as voted by attendees) "MORE Notes from the Blogosphere" in which we'll examine the most common questions posted to "Ask The CMMI Appraiser" and play a game of "Stump the Appraiser" (with plenty of swag), "Successfully reaching CMMI ML2 with Agile Methods" with my friend and client Ross Timmerman of Johnson Controls, and "CMMI or Agile: Why Not Embrace Both" with Mike Konrad and David Anderson.  Also as a back up I hope to be presenting "Encapsulated Process Objects."

Be there or be square.  We'll have a blast!

www.broadswordsolutions.com

CMMIAppraiser on Twitter

There seems to be redundancy between OPP and QPM. What gives?

I would like to have your viewpoint on the differences between the two PA’s QPM and OPP, especially when the scenario is that the organization has identified sub processes like Cost and Schedule Variance as selected sub processes and all the projects follow this organization wide. Now there seem to be lot of redundancy in that case among various practices of QPM as well as OPP.

Ahhh, the 'ol OPP vs QPM redundancy question!

I'll admit that back in the day I had similar issues with the high maturity process areas, but I've been to the SEI mountain, worshiped at the alter of high maturity, and been fully indoctrinated into the cult-like interpretation of the informative material!  Just kidding, it's great stuff!

I understand that you've identified sub processes that your are monitoring and measuring, and that is great!  But it's not quite what the practices are ALL about.

Think of OPP as setting up an infrastructure and engine for statistical monitoring and analysis - an infrastructure that the projects will later use while the execute processes based upon QPM.  The projects also feed the engine, for OPP is a hungry one!

What kinds of things take place in OPP?  Gaining a solid, statistical understanding of a selected set of sub processes and their performance.  This understanding includes analysis such as the identification of the natural bounds of the process (within control limits) and the use of other methods to understand process performance. We call these "baselines."  It also includes "modeling" which uses the data to predict what will happen (within limits) if certain actions are taken in the future.  We call these "models."

QPM is the CONSUMER of the OPP baselines and models.  When a project starts up, the PM will "compose" the process based on the project's goals/objectives AND the baselines and models made available to her via the OPP-related work.

So, for example, if the project is very high risk, with medium complexity, and a compressed time-frame, the PM might select sub processes from the set of standard processes that will help them succeed given those conditions.  He also will constantly monitor the sub process performance using the tools and methods spawned by OPP-related processes to understand whether or not they are going to meet their objectives.

There is a clear delineation between OPP and QPM - one is a supplier, the other a consumer (and then ultimately a supplier BACK to OPP to feed the baseline data).

www.broadswordsolutions.com

What do I need to do to for our company to become CMMI Maturity Level Three

What does our company need to do to become CMMI ML3?

The CMMI is an integrated process model with +-350 distinct practices that define what a “great company” does, at least within it's engineering, product, software, or services organization.  Consider them a list of requirements for your process.  So, your first step is to become smart about the model.

The CMMI model specification can be downloaded from the SEI’s site at www.sei.cmu.edu/CMMI in PDF format, or a better choice is the CMMI book by Mary Beth Chrissis, Mike Konrad, and Sandy Schrum available on Amazon for about $50.00.  It has case studies as well as background information.

A better choice might be for you and your key project leaders and technical leads to take the “Introduction to CMMI” course either at the SEI or from an SEI Partner in your area.  As an alternative, you might consider a ½ day CMMI Executive Briefing.  I give these often with various clients around the country.  You can also download "Agile CMMI" from our website at www.broadswordsolutions.com/resources.php.

There are also numerous presentations, white papers, and documents on the SEI’s web site for free download.  One I would suggest is “CMMI or Agile: Why not Embrace Both!” co-authored by me and a few other folks in the CMMI / Agile world including Hillel Glazer, David Anderson,  Mike Konrad, and Sandy Schrum.

The next step would be to engage a certified CMMI Lead Appraiser to conduct an informal gap analysis (known as a “SCAMPI C”) and perhaps meet with you and your sponsor to develop an appraisal strategy and plan. 

You would then need a period of time (either on your own or with a qualified CMMI consultant) designing, deploying, and using processes based on the CMMI specification.  Along the way you might conduct some informal appraisals, perhaps a SCAMPI “B,” and then, when you are ready, a SCAMPI “A” appraisal.

A SCAMPI “A” is a rigorous, time consuming, intensive event that takes weeks (or months) to prepare for, and then requires a 1-2 week on-site appraisal conducted by a certified SCAMPI Lead Appraiser with a trained appraisal team.  The Lead Appraiser will lead you and team through the SCAMPI appraisal which will result in you achieving (or not achieving!) maturity level two.

Whew!  There’s a lot in here.  It’s a major project, but well worth doing because it will put you in an excellent position for competing and winning in a difficult economy.

www.broadswordsolutions.com

Can we have a global configuration plan or does each project need one?

I would like to ask you a question about Configuration Management from ML2. At the place I'm currently working at we have implemented a global CM plan applicable to all projects (in addition to that, we also have what we call a 'cover page', which is where we include all the roles with the corresponding team members). Should any of them present any difference with the original CM plan, we have a section destined to these particularities inside an application called Team Foundation Server (TFS).

Is it correct to be doing the afore-described, or should we be using a specific CM plan for each project?"

Well, that depends!

I personally can't stand that answer when I get it, but your post left me with no choice!

It all comes down to "what" is in the plan and the characteristics of your projects.  If your projects all use identical work products and code modules, and there is little or no variation, then yes, one plan could work.  Other factors influencing my response would be company size, project complexity, timing, and methodologies being used.

One approach to consider might be to split the CM Plan that your using into one global document that is "common" to all, and another piece for the "project specific" data.  It sounds like you're doing some of this using TFS, which is great, but I have to believe there is more uniqueness to each project than you've described.  If not, you're probably in decent shape.

www.broadswordsolutions.com

What is the difference between ISO certification and CMMI?

We are wondering what the difference is between CMMI Certification and ISO Certification is. We are ISO certified. Are there different levels of CMMI certificates? Any information you can send us would be very helpful.

If I had a nickel for every . . . . . well, I'd still be poor!

That's a great question though. A lot of organization's have been certified for ISO (9001/9001, 15504, etc) and wish to leverage that success towards the achievement of a level of CMMI.

Remember that ISO is an "audit standard" and that the CMMI is a "process model," so conceptually they're quite a bit different. Think of the CMMI as a large set of related "best practices" gathered from many product engineering and software development organizations. ISO can be very helpful, especially for marketing, HR, and quality, but it is not specifically focused on engineering and project management as is the CMMI. So the depth of CMMI in these areas is far greater.

You've obviously done some homework because you've identified one of the differences already. There are five levels of CMMI "maturity" with the vast majority of companies striving to achieve maturity Level Two or Three. ISO does not have this concept. Also, ISO is a "certification" and CMMI, although it looks and feels like one, is something you "achieve." The difference ends up being minor though (at least from your perspective) but it is a distinction.

ISO also doesn't focus with such depth and breadth on the various process areas (there are 22 process areas in the CMMI) and the appraisal process for CMMI (called SCAMPI) is quite a bit more rigorous. I'm not an ISO auditor, but my colleagues have told me that, while there is some overlap, the process of adopting CMMI is quite a bit more rigorous, as is the appraisal method (SCAMPI).

Your best bet is to get some education on CMMI through an authorized "Introduction to CMMI" training class, and to engage a Certified SCAMPI Lead Appraiser to discuss your particular goals and objectives as they relate to CMMI. If you email info@broadswordsolutions.com you'll be able to start the process of learning more about it.

Best of luck to you!

http://www.broadswordsolutions.com/

Follow "Ask The CMMI Appraiser" on TWITTER!

Dear readers,

Get regular CMMI tips on the micro-blogging site "TWITTER" (http://www.twitter.com/). Sign up to follow "CMMIAppraiser" for your tips . . . always free!

Our Appraisal Expires at the end of this year, what should we do now?

I recently started to work for a cmmi level-3 company and in December 2009 our appraisal expires.

1.     What is the latest time to conduct a SCAMPI ?
2.     What is the minimum number of projects to present in the appraisals?
3.     Is there any significant change brought by the release of CMMI version 1.2?

1. Beginning with CMMI v1.2, the duration between appraisals is three years.  That means if you want to maintain your ML3 rating you will need to complete a SCAMPI "A" by December of 2009.  Depending on your current state of readiness there may be a lot to do.  Many clients I work with commission a SCAMPI "C" to evaluate their readiness before scheduling the higher-cost SCAMPI "A" to renew their rating.

2. SCAMPI v1.2 has clarified the sampling requirements to be (from the SCAMPI MDD): 

In appraisals where the reference model scope includes any project-related

PA, the organizational scope must include at least one focus project. If the

organizational unit includes more than 3 projects, then the organizational

scope must include sufficient focus projects and non-focus projects to

generate at least 3 instances of each practice in each project-related PA in

the model scope of the appraisal.

3.  There was quite a bit of change between v1.1 and v1.2.  Some PA's from ML3 were removed, and others were combined.  All ATMs must now be trained in CMMI v1.2 (there is an upgrade course available), and your LA must also be authorized (now certified) in v1.2

www.broadswordsolutions.com

What is the best way to satisfy Generic Practice GP2.1?

We are just getting started and right away we're struggling! What is the best way to satisfy GP2.1 for all of the process areas?

The Generic Practices are foundational to the success of any or all of the process you deploy, and GP 2.1, "Establish and Maintain an Organizational Policy for planning and performing the process" is one of the most important.

GP2.1 is intended to set the expectation, for all practitioners, that they are to plan for AND perform the process that has been established, and that management wants them to do just that. It is an unambiguous request to plan and perform the process as intended.

Now, this set of policies should mirror your process, not the CMMI, and should clearly let everyone know what management's expectations related to performing the process are.

Not every policy is called "a policy" and some may not even exist in a "policy manual."

I have one client that created such a manual, a 100 page document with a detailed policy (bordering on process) and signatures for all the relevant stakeholders contained within its pages. Did this meet the CMMI requirements? Sure, and then some. The problem with this approach is that a) it's too much work to establish and maintain; b) it's not that valuable and c) no one will read it.

Policies can often be effectively communicated in training materials, on posters, in emails, and via word of mouth. In fact, a combination of all of these is often required to get everyone on the same page!

Good luck,

www.broadswordsolutions.com

Can a project perform PPQA on themselves?

Is it possible to use the project team to achieve PPQA SG1?

We have an idea to use the project team to assure the process using checklists and peer reviews, so the QA just audits work products AND assure that the project team do the checklists or peer reviews.

Is it enough for PPQA SG1?

If you've been following my posts you'll know that I'm not one to say that anything is absolute within the CMMI - local tailoring to accommodate culture and local norms is very important. So, I never say never . . . but . . . .

The PPQA PA specifically calls out the need to "objectively" evaluate processes AND the use of those processes.  It doesn't say "independent" though, but it would be uncommon for an organization to be mature and open enough when first implementing PPQA to perform this objectively on themselves.  Many Lead Appraisers insist on "Independence" to prove objectivity.  This isn't always correct though.

The other problem is that PPQA is the "eyes and ears" of the process deployment effort.  It's job is NOT to be the "process police," it's to uncover problems with the use of the process and make sure that information gets into the hands of the right people and is used appropriately.

One idea I advocate is a "rotating" PPQA responsibility.  PMs and Engineers from other projects serve for a period of time in a PPQA role (say 25% of their time for six months).  This has the hidden benefit of gaining "buy-in" from people who may not be fully engaged, and creates process evangelists. 

www.broadswordsolutions.com

What is the minimum number of projects that need to be appraised?

Is there a minimum requirement for the number of projects that must be appraised for a L3 rating?  I was told that at a minimum 3 projects from the organization must be appraised for all SPs and all the SPs should have been implemented across all the 3 projects. Is that true?

Not exactly.  This seems like an easy question to answer, but like everything else related to the CMMI, it's more complex than it appears.  The SCAMPI MDD (and LA's guide for conducting appraisals) says that:

"In appraisals where the reference model scope includes any project-related PA, the organizational scope must include at least one focus project. If the organizational unit includes more than 3 projects, then the organizational scope must include sufficient focus projects and non-focus projects to generate at least 3 instances of each practice in each project-related PA in the model scope of the appraisal."

It then goes on to say:

- Focus projects must provide objective evidence for every PA within the model scope of the appraisal which addresses model practices applicable to those projects (this includes not only "SPs" but applicable "GPs as well!)

- Non-focus projects must provide objective evidence for one or more PAs within the model scope of the appraisal which address practices performed on projects. 

- Support functions must provide objective evidence for their functional areas (for instance Process audit function, SEPG, management, etc).

So, my reading of the MDD is that if you have three or more active projects, then you must come up with evidence for AT LEAST three instantiations of each practice across the focus and non-focus projects and at least ONE of them has to be a focus project (providing evidence for all PAs).  In this example, you could have a minimum of three focus projects, or you could have one focus project and ten (for example) non-focus projects, as long as you have three examples of evidence for each practice.

The MDD doesn't directly speak to what happens if you have only ONE project (I have several clients in this situation).  My interpretation of the rules here is that if you have only one active project, then that one project is a "focus" project and all of the evidence has to come from that single instantiation.

This scenario is unlikely in ML3 though, because you'll also need to add in "support" groups such as SEPG, QA audit, process management, et al and you will end up with more than one "project" to appraise then.

One more wrinkle.  The sample to be appraised is something that the Lead Appraiser and the Sponsor need to agree upon.  I received an RFP once from a DOD contractor that said they had "already determined the appraisal projects" and that there was no opportunity to discuss it.  That's a no-no.  The LA needs to buy into the sample in order to maintain integrity of the appraisal.

www.broadswordsolutions.com

What is the simplest way to satsify RD3.1 "Establish Operational Concepts and Scenarios?"

I have a problem on how to advice SMEs on the simplest, value added way to implement RD P 3.1 Establish Operational concepts and Scenarios. In which specific work products can we see the development of operational concepts? Is it ok to say that you can see the operational concepts reflected in the following work products as they are refined: non functional requirements, design restrictions, system architecture document, installation manual, deployment diagram, package diagram, component diagram?

First let's take a look at why RD.SP3.1 is even in the model.

Sometimes it's difficult to visualize the feasibility or reasonableness of a requirement without putting it in context. This is one reason prototypes are so popular - they literally paint a picture of what the implementation might look like. When they see this users often say "no, that's not what I meant" or "what would happen if we tried this?" This is a form of requirements validation.

RD.SP3.1 is asking you to validate requirements by putting them into some context.

The simplest and most useful way to do this is with a Use Case, User Story, or Storyboard. They require only a marker and a whiteboard, or software like Visio, to implement. They're extremely valuable at putting requirements in context, and they're low cost, low effort.

Developing a prototype for a more complex requirement, or a proof-of-concept (a "spike") is another way, albeit more costly, to drive out pesky requirements defects.

The other artifacts you've listed seem more like they belong in the Technical Data Package (design documents from TS) rather than in RD, with the exception of non-functional requirements, which need to have RD SP3.1 performed against them to be completed.

http://www.broadswordsolutions.com/

Should process improvement tools be represented in our software process improvement plan?

Our software group is currently revising the Software Process Improvement plan to cover CY2009 process activities. For the lst few years the Software Process Improvement Plan (SPIP) only concentrated on process definition work, however for next year the team has identified tools and applications that will work together with our current processes to improve productivity. The million dollar question is: can we still label this document as "Software Process Improvement Plan" (SPIP) although it contains many tool improvement information. The team prefers to maintain a single document rather than having to maintain multiple documents. What is the industry norm for managing this type of elements (Process + Tool)?

Anyone who has worked with me knows that I scoff at the idea of implementing tools when a process improvement program starts up. Too many companies see a tool as their salvation and end up creating a monster that helps them do the wrong thing – only faster.

<<>>

We had this experience in the Broadsword Labs once when our sales staff, without consulting us, decided they needed a new “tool” for expense processing, went out and bought one, slammed it in, and it wreaked havoc with the client base because . . . you guessed it, they didn’t have a solid process behind it. The clients started seeing all these new forms and reports, no one knew how to use the system, there were errors in the expense reports, it was conusing what the roles were . . . . sounds like some GPs doesn't it?

It’s natural for a company to start to see opportunities where a tool could indeed be helpful after they've deployed some stable processes. Early in the "proces improvement" cycle these tend to be tools related to REQM and CM, then later MA, estimating, and then finally life-cycle management tools like Borland’s CALIBER product line.

I would expect a ML3 company to implement some tools, and to manage that implementation in the SPIP. This is an absolute MUST for ML5 (OID) where piloting and deploying “innovations” is spelled out in the SPs.

However you do it, the deployment of tools should be governed in your SPIP and under the guidelines spelled out in OPF.

http://www.broadswordsolutions.com/