CMMI: Do you know what "done" looks like?

Check out this video to see it all drawn out for you: https://youtu.be/_eM1FVFH-04

About once a week my phone rings with a request to "get a level" of CMMI "as soon as possible."

Some of my friends in the CMMI community would chastise me for even entertaining the question and engaging these callers in a helpful conversation - one in particular (you know who you are)  might even slam down the phone (do people even still do that with cell phones?) while yelling "DUMMY!"

It goes without saying that the true purpose of CMMI is to serve as a model for performance improvement, and many say that companies who chase a certificate or plaque (we call this "plaque buildup") are doing nothing good for themselves, and probably damaging their company in the process.  

But I'm not with them on this.  Here's why.

These callers, often company owners, are in a very tight spot. Often they are qualified for the work, sometimes they're even ALREADY doing the work, but then the government (or customer) throw the CMMI requirement at them as a pre-requisite for keeping their revenue coming in.  Many of them have never even heard of CMMI.  I can excuse them for thinking this a tactic meant to unfairly exclude them from winning work for which their are qualified.

Frankly, asking how fast they can get to CMMI MLx is a fair question.  Especially since they don't really have a good feel for what CMMI even is.  They deserve an answer for how to "get it done."

So what does "done" look like?

We recommend starting with an independent, unbiased assessment.  This is a great way to get at the "single source of truth" and bypass all of the emotions, opinions, and roadblocks.  The CMMI has hundreds of nooks and crannies where performance weaknesses can hide, and we want to make sure you understand exactly what is in front of you and what needs to be done.

Once you have that under you belt, you'll have everything you need to put together a "plan-to-achieve." The data tells us that it's not a short process, but if you treat it like a project, and take a deliverable, purposeful approach, it can be done by any company.  Duration depends on a lot of factors: where you are today, resources commitment, sponsorship, and company culture.  I've seen companies take 5 years to get to ML2, and others take 6 months to get to ML3.

As we like to say in our business, "it depends." It's impossible to know without getting into the details.  Anyone who tells you otherwise ("sure, I can get you there in 2 months...") is lying.  Or unethical. Or both.

Once the plan is defined, there's some work to do.  You'll design some processes, develop some training, deploy both, and have your projects adopt the improvements.  In some cases, you may change the way you're working altogether.  Along the way we usually provide some training (Introduction to CMMI), workshops, and consulting, and when you're ready.....

We plan your appraisal(s) by selecting projects and people to sample, developing a schedule, documenting the plan, and reviewing/approving it with key stakeholders.  In the background we'll register your appraisal with the CMMI Institute, and make other key preparations.

About 30 days before your scheduled "onsite," we'll conduct some Appraisal Team Training with four or more of your team members, and conduct a Readiness Review.  That's where it all starts.  From there there will be same data collection in preparation for the big day arriving!

The "onsite portion of your appraisal (known as "Phase II") will take place over a week or more, and will include interviews, artifact reviews, and characterization of practices (a kind of "scoring"), all leading up to the development of a final findings report that will spell out the result.

Did you "pass?"  Well - that's up to you.  Good luck!

Check out this video to see it all drawn out for you: https://youtu.be/_eM1FVFH-04

Like this blog? Forward to your nearest engineering or software exec!

Jeff Dalton is a Certified SCAMPI Lead Appraiser, Certified CMMI Instructor, author and consultant with years of real-world experience with the CMMI in all types of organizations. Jeff has taught thousands of students in CMMI trainings and has received an aggregate satisfaction score of 4.97 out of 5 from his students.

Visit www.broadswordsolutions.com for more information.

Just the FAQs: What's the value in tailoring and tailoring guidelines?

[Dear Readers, our good friend Pat O’Toole, CMMI expert and seasoned consultant, is collaborating with us on a new monthly series of CMMI-related posts, "Just the FAQs." Our goal with these posts is to provide answers to the most frequently asked questions about CMMI, SCAMPI, engineering strategy and software process improvement. This month Pat discusses the new "Action Plan Re-Appraisal." Take it away! ~ the CMMI Appraiser]

We’re struggling with “tailoring” and “tailoring guidelines.” We think we understand the concepts, but we can’t seem to find the value. What are we missing?

[Note: Jeff addressed much of this in “Just the FAQs #06: What can we learn about process architecture from Justin Bieber?” It’s a terrific article, and if you haven’t watched the YouTube video that he included at the end, stop reading and watch it – it’s a hoot!]

PAT: I actively participate in many of the CMMI‐related Yahoo and LinkedIn forums. I’ve noticed that in many of the exchanges, participants tend to talk past, rather than to, one another, as illustrated in the following fictitious exchange:

A: Our lead appraiser said that we don’t have to perform some of the CMMI practices and we can still get our maturity level.

B: Well THAT’s not right – unless, of course, you are performing alternative practices instead.

A: Nope, he said we are not expected to have alternative practices in place either.

B: Your lead appraiser is dead wrong and should have his certification revoked!

This seemingly meaningful conversation goes back and forth for some time until…

A: Yeah, well, we are going for ML2 and he says that none of the ML3, ML4, or ML5 practices have to be performed – in fact, he won’t even be looking at them during our appraisal.

B: Oh, uh, well, never mind…

The point is that context is important!

Keeping that in mind, consider that “tailoring” can be performed at various levels of process decomposition, and you really have to understand the context in which tailoring is being discussed in order to have a meaningful dialogue. As I see it, there are four layers of tailoring:

First, there’s MEGA‐“T” tailoring – essentially, selecting a life cycle. Whether you are consciously aware of it or not, once you’ve chosen a given development life cycle, a whole lot of rituals associated with all of the other life cycles approved for use have just been tailored out. One could think of this as “tailoring with an ax.”

Next, there is CAPITAL‐“T”, or phase‐level tailoring. In very small projects, for example, the tailoring guidelines may allow the Requirements and Design phases to be combined into a single “Desirements” phase. Similarly, verification and validation activities might also be conducted in an integrated manner.

One could think of this as “tailoring with a knife.” Then there is small‐“t” tailoring – selecting among various methods within a given phase ‐ combining High Level Design and Low Level Design specs, for example, or selecting either formal inspection, desk‐check review, or buddy review for verifying a given document. This is probably what most people think about when they hear the word, “tailoring.” One might think of this as “tailoring with scissors.”

Finally, there is micro‐“t” tailoring – adjustments that are made much closer to the bottom of the food chain. For example, your formal inspection process may indicate that the work product to be inspected must be pre‐published at least 3.17 days in advance of the inspection meeting, thereby giving the participants adequate time to prepare. However, a tailoring guideline may indicate that the item can be pre‐published with less lead time provided each participant explicitly agrees that the truncated review period still provides them adequate preparation time. One might think of this as “tailoring with a needle.”

Until you establish the proper context for a given tailoring discussion, people may be doing a lot of talking, but very little communicating.

. . . . .

OK, now that I’ve (successfully?) argued that context is important to conduct a meaningful conversation about tailoring, let me set the context for what I see as one of the primary stumbling blocks with respect to deriving maximum value from the “tailoring guidelines” implemented by many/most organizations – the source of which is found in the last bit of the CMMI Glossary’s definition: “Tailoring guidelines describe what can and cannot be modified and identify components that are candidates for modification.”

Having conducted a ton of appraisals targeting ML3 or higher, I’ve noticed a couple of recurring themes when it comes to tailoring guidelines – both of which appear to align much too closely to this portion of the Glossary definition.

One of the recurring themes I see is sets of tailoring guidelines that indicate which process elements are mandatory, which are strongly encouraged, and which can be tailored out. These are clearly delineated by CMMI process area to ensure full credit is given to all GP3.1 practices when conducting an appraisal. This aligns much too closely with that bit in the Glossary that indicates “tailoring guidelines describe what can and cannot be modified.”

The other recurring theme I see is use of an Excel spreadsheet that lists every conceivable arrow in the organization’s process quiver. It includes every policy, life cycle description, process document, methodology, workflow, procedure, work instruction, guideline, template, form, checklist, metric, bathroom stall number, etc. that can be selected. The project manager is expected to meticulously mark those elements that the she intends to use. This theme aligns much too closely with that bit in the Glossary that indicates, “Tailoring guidelines… identify components that are candidates for modification.” Well, it’s more “inclusion” than “modification,” but the point still stands – explicit decisions are made regarding the project’s means of conducting its business – its “defined process.”

(Note: organizations having trouble conjuring up uses of formal decision making are likely to use DAR to make the tailoring decisions on each project – thereby bagging an appraisal twofer!)

Having a list of mandatory, suggested, and optional process elements provides some modest value, as does using the 1000+ row Excel checklist to make explicit project decisions regarding process element inclusion. But, like you, I believe there is more value to be had from the concept of tailoring.

Let me try to make the point by relying on everyone’s my favorite analogy – marathon running! Most long distance runners are concerned about the “Five H’s” – hills, heat, humidity, height (altitude), and head winds. Two of the five, hills and height, are attributes of the course and can be considered well in advance. The other three, heat, humidity, and head winds, are race day attributes – and, weather forecasts notwithstanding, you really won’t know what you’re up against until the day of the race.

Magazines like “Runner’s World” have lots of articles dedicated to such topics:

• Heat and How to Handle It
• Don’t Sweat Running in Humidity
• Running Against the Wind

These articles, written by very experienced marathoners, share secrets, hints, and tips about adjustments you may want to consider due to race‐day conditions. The author’s objective is to arm you with information based on their tried‐and‐true experience, but the decision to follow their advice or not is entirely up to you. After all, what works for them might not work for you.

So why not apply this same value‐added concept to process tailoring? In addition to following the standard script for establishing bland and boring tailoring guidelines (the one you know will pass the appraisal), why not write your own “articles” filled with tailoring guidance. Organizations that adopt such a “tips and hints” approach typically find it much more valuable than those that only follow the more mechanistic approaches above. Your project retrospectives, lesson learned sessions, or, my favorite, project post mortems (yet another project has died; let’s examine the corpse and see what we can find) will evolve from generating a repeatable list of what did and didn’t go well on the project, to lively discussions that also explore the question, “If we knew then what we know now, what might we have done differently?”

Like the “Runner’s World” articles, tailoring guidance is intended to provide experiential insight for future projects’ consideration. Codifying the secrets, hints, and tips of how to tailor project activities to overcome the inevitable hills, heat, humidity, height, and head winds will arm the next project team with advice that should make it easier for them to go the distance, and hopefully that’s the value that you were looking for.

Like this post?  Please forward to your nearest engineering or software professional!

© Copyright 2015: Process Assessment, Consulting & Training and Broadsword Solutions

“Just the FAQs” is written/edited by Pat O’Toole and Jeff Dalton. Please contact the authors at PACT.otoole@att.net and jeff@broadswordsolutions.com to suggest enhancements to their answers, or to provide an alternative response to the question posed. New questions are also welcomed!

Just the FAQs: SPECIAL EDITION! CMMI Institute Announces "Action Plan Re-Appraisal!"

[Dear Readers, our good friend Pat O’Toole, CMMI expert and seasoned consultant, is collaborating with us on a new monthly series of CMMI-related posts, "Just the FAQs." Our goal with these posts is to provide answers to the most frequently asked questions about the CMMI, SCAMPI, engineering strategy and software process improvement. This month Pat discusses the new "Action Plan Re-Appraisal." Take it away! ~ the CMMI Appraiser]

I heard that the CMMI Institute just released this thing called an “Action Plan Reappraisal.” What is it and why should I care?

PAT: Imagine that your organization just completed a maturity level 4 (ML4) SCAMPI A appraisal, but because you failed a goal in Organizational Process Performance (OPP) at ML4 and had an “Not Rated” goal in Risk Management (RSKM) at ML3, the SCAMPI A appraisal resulted in an ML2 rating. As Maxwell Smart might say, “You missed it by that much!”

Let’s assume that the ML4 rating was important to your team as a formal recognition of accomplishment and/or because a strategically important client requires it. In either case, you would want to remediate the goal‐impacting weaknesses, institutionalize the associated behavior changes, and be re‐appraised as soon as possible. Prior to the Action Plan Reappraisal being released, your organization would have to undergo another soup‐to‐nuts ML4 SCAMPI A appraisal – a very expensive and disruptive undertaking.

Now, however, the Appraisal Sponsor can elect to conduct an Action Plan Reappraisal (APR) – essentially a “delta appraisal.” An action plan is generated and executed by the organization to remediate the weaknesses and institutionalize the behavior changes, after which the appraisal team re‐evaluates the failed or not rated goals. If any of these goals are now rated “Satisfied,” the maturity level is regenerated. In the happiest of scenarios, both the OPP and RSKM goals would now be rated “Satisfied,” and ML4 will have been achieved!

Any questions?

Is there a time limit in which the Action Plan Reappraisal (APR) must be conducted?
The APR must conclude within 4 months of the SCAMPI A Final Findings presentation.

Why 4 months?
4 months was selected as an appropriate time frame for remediating minor goal‐threatening weaknesses; the APR is not intended for use when remediating major or systemic weaknesses.

Is there a limit regarding how many APRs may be conducted?
Only one APR can be conducted for a given SCAMPI A appraisal.

What happens if our current maturity level expires during this 4‐month period?
An appraisal result that exceeds its 3‐year validity period is no longer published on PARS. If the current SCAMPI A was completed and the associated APR is concluded within the 4‐month time limit, the Appraisal Sponsor can authorize the current results to be published in PARS. Publication in PARS would start after the ARP has been submitted to the CMMI Institute via the SCAMPI Appraisal System (SAS) and the CMMI Institute has concluded its Quality Audit.

How is the 3‐year validity period established for a SCAMPI A that had an APR?
Such an appraisal would expire 3 years after the delivery of the SCAMPI A Final Finding presentation.

Is it now required to conduct an APR if the target maturity level/capability level profile was not achieved in the SCAMPI A appraisal? If not required, who makes the decision to conduct an APR or not?
No, the APR is not required; it is an option available to the Appraisal Sponsor. The Lead Appraiser provides a good‐faith recommendation on whether an APR might be successful if pursued, but the final decision rests with the Appraisal Sponsor. If proceeding with an APR, this decision must be communicated to the CMMI Institute prior to submitting the SCAMPI A appraisal via SAS.

Typically, a goal fails due to one or more practices being characterized as “partially implemented.” or “not implemented.” Does the APR evaluate just these practices? Goals? Process area? Maturity level?
The APR appraises all of the “Unsatisfied” and “Not Rated” goals in the APR appraisal scope in the same manner as in a SCAMPI A: for each practice that supports the goal, documents are reviewed and interviews conducted to determine if that practice is fully, largely, partially, or not implemented on that project. These project‐level characterizations are then used to characterize the practice overall for the organization, or what’s referred to as the “OU practice characterization.” These OU practice characterizations are then used to rate the goal as “Satisfied” or “Unsatisfied.”

Note that it may be determined that additional goals need to be put in scope for the APR to properly evaluate the remediation. For example, if a Project Planning goal was rated “Unsatisfied” in the SCAMPI A, it might be appropriate to evaluate one or more goals from the Project Monitoring and Control process area as well. This decision is made by the Lead Appraiser.

Do we have to evaluate the remediated goals on the same projects, or can we do so on others?
The expectation is that the same projects will be evaluated. However, there may be cases where this is not feasible – for example, the project terminated prior to the remediation being applied, or the project is beyond the point in the lifecycle where the remediated practices are executed. In such cases, replacement projects with the same sampling characteristics may be evaluated instead.

What happens if some of the goals are still rated “Unsatisfied” or “Not Rated” at the end of the ARP?
In the example above, if the previously failed OPP goal was now rated “Satisfied” but the previously unrated RSKM goal was still “Not Rated,” the maturity level would be regenerated but the outcome would be the same – ML2. On the other hand, if the RSKM goal was now rated “Satisfied” and the OPP goal was still rated “Unsatisfied,” the regenerated maturity level rating would be ML3.

Must the organization have all failed and unrated goals re‐evaluated as part of the APR?
No, the Appraisal Sponsor can elect to conduct the APR on a subset of these. In the example above, the sponsor may decide to include only the unrated RSKM goal in the APR scope and, if the goal is successfully remediated, institutionalized, and rerated as “Satisfied,” ML3 will have been achieved.

Can the organization include new process areas in the APR? That is, in the example above, could they expand the model scope to include some or all of the ML5 process areas?
No, the model scope of the APR cannot be greater than that of the original SCAMPI A appraisal.

Do we have to use the same Lead Appraiser for the APR, and do all of the original appraisal team members have to be involved?
Yes, but... The original Lead Appraiser must be involved in all evaluation‐related APR activities, and with the exception of documentation review, these activities must be conducted onsite. In addition, for each failed or unrated goal in the APR model scope, to achieve continuity at least one member of the SCAMPI A mini‐team that evaluated the detailed evidence associated with that goal must also be involved in all of these onsite appraisal activities. In other words, there will be at least two members of the original SCAMPI A appraisal team, and maybe more, involved throughout. The other SCAMPI A appraisal team members must also be involved in OU practice characterizations and goal/capability/maturity level ratings, but they may participate remotely (e.g., via teleconference).

So in our example, if separate mini‐teams were responsible for evaluating the evidence and characterizing the practices for OPP and RSKM, then the Lead Appraiser and at least one member of each of these mini‐teams must participate in all evaluation‐related APR activities. All of the original appraisal team members must be involved in generating the OU practice characterizations and rating the goals.
Special provisions must be made with the CMMI Institute if any of the original appraisal team members, including the Lead Appraiser, cannot participate in the APR.

If there a limit on the number of failed goals that may be evaluated in an APR?
No, however, the Lead Appraiser must make a recommendation to the Appraisal Sponsor regarding the feasibility of achieving the objective. This may include reminding the sponsor that, for a goal to be rated “Satisfied,” the weaknesses have to be rectified AND the behavior changes have to be institutionalized.

Will the appraisal posting on PARS indicate that an APR was conducted?
No, the SCAMPI A and APR are considered a single appraisal event. The posting on PARS only indicates the final outcome of the appraisal; there is no indication that an APR was, or was not, conducted.

What do YOU perceive to be “the good, the bad, and the ugly” regarding Action Plan Reappraisals?
[Author’s note: Prior to this question, all answers were based on the changes introduced in the SCAMPI MDD v1.3b to accommodate the APR. In response to this question, Pat is providing his own opinion, which may or may not align with that of the CMMI Institute or his fellow Lead Appraisers!]

The Good:

Some organizations insist on being “rock solid” going into a SCAMPI A appraisal, as the fear and cost of failure is very high – in both financial and political terms. These organizations may conduct a series of Class C and Class B “dress rehearsals” until they have achieved 99.9% confidence that they will “pass” the SCAMPI A. The Action Plan Reappraisal option will allow such organizations to lower the overall cost by embracing more rating risk. The organization may forego one or more “dry run” appraisals if they know they can address any last potential goal threatening weaknesses over the ensuing four months.

Currently, some Lead Appraisers may struggle with the ethical decision to “flunk” an organization based on one or two marginal practices. They are torn between staying true to the model and method, and forcing the organization to incur the significant cost and disruption of another full SCAMPI A appraisal. The APR option makes the right decision much more palatable (for all “relevant stakeholders!”)

The Bad and the Ugly:

Some perceive that the introduction of the APR “tarnishes the brand.” That is, the SCAMPI A appraisal method is a rigorous evaluation of current organizational outputs and behaviors to determine if a predetermined set of CMMI goals is being satisfied. Appraisals bring focus to organizations that might otherwise be easily distracted. And since the SCAMPI A is typically perceived as the “gold standard” of appraisal methods, anything less, like the APR, is, well, less.

Some organizations may decide to accept too much risk. That is, rather than simply foregoing their 16th pre‐SCAMPI A dry run appraisal, they may throw prudence and caution to the wind and do NO pre‐ appraisal preparation, trusting that they can address any and all issues during their four month “grace period.” Damn the torpedoes, full steam ahead!

Finally, some CMMI‐adopting organizations may fear that an unscrupulous Lead Appraiser may see the Action Plan Reappraisal as a means of extracting even more revenue from them. “Oh gee, you failed a goal, I guess you’ll just have to pay me for another 4 or 5 days of service to get your level!”

Concluding Note
The SCAMPI Method Definition Document v1.3b, Section 4 is the Action Plan Reappraisal’s “official rule book.” If you would like a copy of this section, a copy of the full MDD, or if you have any additional questions or “Good, Bad, and Ugly” insights, please feel free to contact Pat at PACT.otoole@att.net.
Over time, I suspect that what “Dear Abby” said about birth control may also apply to the APR – “It’s better to have and not need than to need and not have.”

© Copyright 2015: Broadsword Solutions and Process Assessment, Consulting & Training
“Just the FAQs” is written/edited by Pat O’Toole and Jeff Dalton. Please contact the authors at pact.otoole@att.net and jeff@broadswordsolutions.com to suggest enhancements to their answers, or to provide an alternative response to the question posed. New questions are also welcomed! 

Do all SCAMPI Appraisal Team Members need to be trained?

Dear Appraiser:

We are currently in the process of going for CMMI L2 & L3 appraisal, my senior management has a question regarding Appraisal Team Members (ATMs) that will under-go the official CMMI training [Introduction to CMMI].

Our consultant proposed 7 ATM’s and will train them. From a sustainability point of view, if some of the ATM’s leave the organization in the future what is the procedure to get additional training  for new members in the future?

Is it mandatory to have ATM’s go through official introduction to CMMI training class?

Thank you in advance for your guidance.  ~Ramesh.

Great question!

Being on a SCAMPI Appraisal Team is serious business, as the ATMs are being asked to gather and evaluate objective evidence, and then develop an inventory of strengths and weaknesses that will be used to improve the performance of the appraised origination.

For obvious reasons, Appraisal Team Members need to be both experienced and trained.  Their experience and domain knowledge helps them understand how the CMMI may be used to improve performance, and "model knowledge" of CMMI helps them understand and apply the practices in a common way that makes sense to everyone on the team.

All prospective Appraisal Team Members must meet basic experience requirements (it's safe to assume no "newbies" or interns will qualify) as defined in the SCAMPI Method Definition Document (MDD), and total team experience must be at least 25 years, not including the Lead Appraiser.

It's not as hard to meet this requirement as you may think.  I'm working on an Appraisal this week with combined team experience of over 130 years!

In addition to experience requirements, there are also training requirements.  Appraisal Team Members must have completed at least one offering of "Introduction to CMMI" (v1.2 or v1.3) in the constellation they are using to appraise their organization (DEV, SVC, of ACQ), and must also have completed a SCAMPI Appraisal Team Training class provided specifically for that appraisal by the Lead Appraiser.

Appraisal Teams that are not well trained, or those with limited experience, will find it difficult to add value to the company during a SCAMPI A Appraisal.  Experienced and well-trained teams usually add significant value - so it's well worth doing!

Good luck - and enjoy your training!

Like this blog? Forward to your nearest engineering or software exec!

Jeff Dalton is a Certified SCAMPI Lead Appraiser, Certified CMMI Instructor, author and consultant with years of real-world experience with the CMMI in all types of organizations. Jeff has taught thousands of students in CMMI trainings and has received an aggregate satisfaction score of 4.97 out of 5 from his students.

Visit www.broadswordsolutions.com for more information about engineering strategy,performance innovation, software process improvement and running a successful CMMI program.

It’s About the Skills, Not Just “Training”

[Dear Readers, our good friend Pat O’Toole, CMMI expert and seasoned consultant, is collaborating with us on a new monthly series of CMMI-related posts, "Just the FAQs." Our goal with these posts is to provide answers to the most frequently asked questions about the CMMI, SCAMPI, engineering strategy and software process improvement. This month Pat talks about the importance of having the necessary skills and competencies, not just meeting a requirement for “training.” Take it away, Pat! ~ the CMMI Appraiser]

In order to address GP2.5 (Train People), we’ve developed 2 hour PowerPoint presentations for each process area and have sign‐up sheets that show we’ve trained all personnel on the processes relevant to their roles. We should be golden in an appraisal, right?

At the risk of being labeled a radical, let me suggest that we forget about the CMMI for a minute…

If I’m the current occupant of the corner office, I don’t care if my people have been “trained,” I care that they have the skills and knowledge to perform the work for which I am paying them. I recognize that in order to achieve a high degree of proficiency, my people need competencies in four skill dimensions:

1. Fundamental skills

2. Domain/product skills

3. Process/tool skills

4. Supplemental/soft skills.

When filling a project management position, the interview process will explore, among other things, the candidate’s knowledge of, and experience in project management. If the person happens to be a Project Management Professional (PMP)® this is pretty easy, as they have achieved an industry-recognized certification that attests to their fundamental skills. They have demonstrated the knowledge and ability to plan and manage projects using such foundational concepts as Work Breakdown Structures, Earned Value, risk management, etc.

OK, so the new hire has fundamental project management skills, but can they apply their skills in our little corner of the world? What do they know about banking systems, EFT, check imaging, and the associated industry standards and governmental regulations? Do they understand the products and services that we offer to meet the needs of our demanding marketplace, and how our customers use our offerings to perform their work? Do they understand the competitors’ product offerings and why we occasionally lose out in a competitive bid situation? 

Without such insight, they will not be able to employ their fundamental skills in an optimal manner on OUR projects. If there are gaps in domain/product knowledge, they need to be addressed for this new hire to become proficient.

In addition, I want to make sure that my people can use the processes and tools that our organization is providing to enable them to perform their work. If they don’t know how we move a project from Point A to Point B, they won’t be able to move their project along in an efficient manner. Such kinks in the system need to be identified and addressed to keep the products and services (and therefore the revenue) flowing seamlessly. 

Better yet, rather than “identified and addressed,” I would prefer that such kinks be “anticipated and avoided,” moving from the reactive state of problem resolution to the proactive state of problem avoidance.

Finally, various roles may require some supplemental skills, a subset of which we typically refer to as "soft skills.” My new project manager may have tremendous fundamental, domain, and process/tool skills, but if they have a cosmic meltdown and a bad case of the sweats every time they present to management or the customer, then we have a problem. Or maybe they write really badly, or they don’t delegate, or they’re not a team player, or… (BTW, an example of a supplemental skill that is NOT a “soft skill” is the ability to read and write Japanese.)

Notice that I didn’t need the CMMI to tell me that my people should have appropriate skills in each of these four dimensions – it’s simply smart, pragmatic management. When interviewing people for an open position, we are trying to match their skill profile to the position’s skill requirements – the more of the required skills they already have covered, the quicker they can contribute to our success.

Rarely do we find the ideal candidate – one that has the perfect combination of fundamental, domain, process/tool, and supplemental skills – so there is nearly always a need to address skill and knowledge gaps. We may use coaching, mentoring, and stretch assignments to address some of these needs. We may place them on a team where they will have the support of other practitioners to help them ramp up quickly. Oh, and we may provide them some formal training as well.

So if management in an organization that doesn’t use the CMMI knows this, shouldn’t management in an organization that uses the model be at least as good? Assuming the answer is “Yes,” doesn’t the 2 hour PowerPoint presentation approach seem a bit underwhelming? Such an approach appears to be much more focused on passing an appraisal than on ensuring employees have the necessary skills and knowledge to perform their work effectively and efficiently.

But let’s be realistic, as soon as you generate an appraisal finding indicating that the 2 hour PowerPoint approach is insufficiently robust, you should fully expect the CMMI Legal Lawyers (who graduated from "Certify U”) to crawl out of the woodwork and strenuously object. “Show me in the model where it says that training is required to cover fundamental skills, domain/product skills, process/tool skills, and supplemental skills!” they protest – and truth be known, given the letter of the law, they are right – the model doesn’t say that.

However, as a top notch defense attorney, you will have already had this discussion with the appraisal sponsor, asking if she is satisfied with a minimalistic approach to “training,” or if she would prefer a more comprehensive approach to competency management – an approach that would provide real value by evolving new hires from marginally adequate to demonstrably proficient. Once she nibbles on the bait, you can tell the CMMI Legal Lawyers that their objection has been overruled.

For those of you that feel more comfortable having model basis for such a finding, remember that

Organizational Process Focus specific practice 1.2 suggests, “Appraise the organization’s processes periodically and as needed to maintain an understanding of their strengths and weaknesses.” Unfortunately, most people interpret this last bit as “… an understanding of their strengths and weaknesses with respect to the CMMI.”

But it needn’t be that way! One could just as easily interpret this as “… an understanding of their strengths and weaknesses in accomplishing organizational and project objectives.” Interpreting the model in this way encourages the lead appraiser community to break free of its own compliance mindset and constraints, thereby empowering us to help our clients appreciate and exploit the value the model is truly intended to provide.

© Copyright 2014: Process Assessment, Consulting & Training and Broadsword Solutions

“Just the FAQs” is written/edited by Pat O’Toole and Jeff Dalton. Please contact the authors at

pact.otoole@att.net and jeff@broadswordsolutions.com to suggest enhancements to their answers, or to provide an alternative response to the question posed. New questions are also welcomed!

Global Adoption of CMMI Performance Improvement Framework Continues to Rise

Reposted from: BusinessWire · Feb. 24, 2014 | Last Updated: Feb. 24, 2014 11:31 AM ET  

CMMI Institute announced today that a record number of Capability Maturity Model Integration (CMMI) appraisals were completed in 2013 as organizations across the globe elevate organizational performance by adopting CMMI. The company also released the findings of a customer satisfaction survey using the Net Promoter® Score (NPS) methodology in which individuals who utilize CMMI rated CMMI as the world class means of improving organizational performance.

CMMI appraisals allow companies to measure their capability maturity against a defined framework of best practices. More than 1,600 appraisals were completed in 2013, the most ever done in one year and an 11% increase over 2012, the previous record year for appraisals. Of the 2013 appraisals nearly 40% were completed in China, where the widespread adoption of CMMI is driving economic growth and development across the region.

For complete story go to: http://t.co/LijVKa7L0Y

Just as a side-note, as a Lead Appraiser at Broadsword, I personally delivered 10 of those SCAMPI A's last year, in addition to about a dozen Class B and fifteen Class C Appraisals.

CMMI ROCKS!

Like this blog? Forward to your nearest engineering or software exec!

Jeff Dalton is a Certified SCAMPI Lead Appraiser, Certified CMMI Instructor, author, and consultant with years of real-world experience with the CMMI in all types of organizations. Jeff has taught thousands of students in CMMI trainings and has received an aggregate satisfaction score of 4.97 out of 5 from his students.

Visit www.broadswordsolutions.com for more information about engineering strategy, performance innovation , software process improvement and running a successful CMMI program.

Do we need measures for every Process Area if we want to be Maturity Level Three?

Dear Appraiser,

When being appraised for CMMI Level 3 do we need to have one or more measures of EVERY PA or can we just measure the things we actually need to? ~Nabeel.

Nabeel,

Thank you for reading!  Interesting question.

Every-time some asks me this I brace myself for the many people who will write in to disagree with me!  This is a subject that people feel very strongly about, and I'm always afraid of starting a religious war….

So, let me go put on my armor, pick pick out my favorite sword, and I'll be right back….

….. OK, I'm back (albeit a little bit heavier)….

I think you're on to something when you say "things we actually need."  It's important that a company establish measures that reflect the information needs that are required to be a great company.  In fact, the CMMI explicitly states that you should "Develop and Sustain a Measurement Capability Used to Support Management Information Needs."

What does this mean?

It means that only you and your management can determine what your company can benefit from.  Not a consultant.  Not a model.  Not an instructor.  So, I'll agree *somewhat* with the statement implied by your question.

However…..

In order to fully grasp the intent of the CMMI, you need to understand that it was developed as a research project.  And that project examined the successful implementations of many projects at many companies, and came to some conclusions about patterns and practices.

In my experience, many companies (and their management), don't really have any idea WHAT their management information needs are.  As a CIO myself, I often learned about some new technique, measure, or process that I had not been using, only to say "Wow, that is something I REALLY need to do!"

So goes the CMMI.  It catalogues many of the attributes of a measurement "program" that were seen at other great companies, and makes suggestions for how you might use them.

Note that the CMMI doesn't tell you that you need to measure EVERYTHING.  It starts by asking the question, "what are your objectives?" (MA SP1.1).  This should drive everything.  Typical objectives are "projects need to be on time, on budget," "risks should be kept to a minimum," "growth should be at n%," recalls should be below n%," and so on.  The more you can quantify it, the better.

But the Process Areas also give you a hint into what you should measure.  Look at VER SP2.3 - which discusses analyzing the results of peer reviews.  That implies a measure.  Look at GP2.8, "Monitor and Control….", that implies one also (and appears in every process area).

Let's take that one on for a second.  You didn't say it, but I suspect GP2.8 is the one you are referring to when you asked your question. There are many Lead Appraisers who insist that this practice means you must have a measure for each process area.  This isn't exactly true.  The CMMI isn't a requirements document - it's supposed to be interpreted so that it provides the most value for the user. There was a time when this measurement requirement was the conventional approach (back in the SW-CMM days), and even early in the CMMI life-cycle it was carried over, but this was clarified in version 1.3 of the model to read:

"GP2.8 Monitor and Control the Process"

It goes on to say "…. can involve measuring appropriate attributes…."  Neither the  Goal (required) or the practice (expected) say "must."

The sub-practices provide additional data with phrases like "evaluate actual progress," "review accomplishments," "review activities," "identify problems," etc…

I would start by turning each instance of GP2.8 into a question: "How do I know our requirements change process is working well?"  "How do I know that our estimating process is effective?" And so on.

If the answer requires a metric, then so be it.  If you have another creative solution, then by all means use that too!

In other words, measurement should be PART of the toolset you use to monitor and control your process, and there is a very high likelihood that you will have a healthy collection of process performance measures, but it's not the only tool you can use.

In the end it's not important exactly what the mix of tools is - but I WILL ask you to answer that question when I appraise your organization, so be ready!

Good luck!

Like this blog? Forward to your nearest engineering or software exec!

Jeff Dalton is a Certified SCAMPI Lead Appraiser, Certified CMMI Instructor, ScrumMaster, author, and consultant with years of real-world experience with the CMMI in all types of organizations. Jeff pioneered agileCMMI, the leading methodology for incremental and iterative process improvement. He has taught thousands of students in CMMI trainings and has received an aggregate satisfaction score of 4.97 out of 5 from his students.

Visit www.broadswordsolutions.com for more information about engineering strategy, performance innovation , software process improvement and running a successful CMMI program.

To download eBooks about CMMI, visit Jeff’s Author Page on Amazon.

What kinds of questions are we asked during a SCAMPI C? What can the interviewees expect?

Dear Appraiser,

What kinds of questions are we asked during a SCAMPI C?  What can the interviewees expect? ~Gary

Great question!  I get this one a lot, and the first answer is "don't worry!" 

A SCAMPI C is simply a gap analysis to determine the current state of the process at your company.  We want to understand how you do the things you do everyday.  

There are four main categories: Project Management, Process Management, Engineering, and Support (tools, measurement, quality, etc).  We don't ask them by category, per se, but we try to stick with project "threads."

We will ask you to describe:

- how did you estimate the scope of that project?

- what happens when a customer asks to change the requirements/features?

- how do you ensure the quality of the designs, code, or specification for your end product?

- how are your teams organized?

- what measures are you capturing?

- what process/approach/tools are you using to ensure document/code/designs are managed?

… and so forth.

There is no "right" answer to any question.  You will not need to know the CMMI, or any of our buzzwords, to have a successful outcome.  We will analyze all of the information, from all of the interviews, and compare it to the practices in the CMMI-DEV model.  That, along with other research we do on your company, will lead to a mapping of strengths and weaknesses weaknesses that can be used to implement improvements at your company.

Interviews are usually held in small groups, either project leadership (PM, Lead Engineer, Lead Tester, etc), or in "functional groups" (engineers, testers, etc).  We usually speak with each group for 1-2 hours.

For more information about the areas of the CMMI, head over to our "What is CMMI" page.

I hope this helps.  Feel free to share this information, or to ask any any other questions you may have.  And have fun!

Like this blog? Forward to your nearest engineering or software exec!

Jeff Dalton is a Certified SCAMPI Lead Appraiser, Certified CMMI Instructor, ScrumMaster, author, and consultant with years of real-world experience with the CMMI in all types of organizations. Jeff pioneered agileCMMI, the leading methodology for incremental and iterative process improvement. He has taught thousands of students in CMMI trainings and has received an aggregate satisfaction score of 4.97 out of 5 from his students.

Visit www.broadswordsolutions.com for more information about engineering strategy, performance innovation , software process improvement and running a successful CMMI program.

To download eBooks about CMMI, visit Jeff’s Author Page on Amazon.

When is Broadsword speaking at SEPG NA 2013?

Hey, CMMI Appraiser, 

Last year I heard you speak at SEPG NA in New Mexico. You talked about how going through the CMMI Appraisal process is more about organizational performance improvement and less about getting a “CMMI certificate,” which doesn’t exist anyway.  Will you be speaking again at the SEPG conference in October? If so, when? I'd like my managers to attend. ~ Mary S.

Hey, Mary,

Yes, I’ll be there! For the 8th consecutive year, Broadsword has been selected to speak at the SEPG North America 2013 Conference on October 1-2, 2013 in Pittsburgh, PA.

This year promises to be even better than ever. I will be introducing a concept I call “Agile Resiliency” – a strategy for helping engineering and software professionals be more Agile, and make Agile even stronger with the CMMI -- and there will be presentations by some of the most influential thinkers in the industry.

So invite your managers. Invite your clients. Spread the word!

I’m thrilled that you remembered the key take-away from my speech last year, Mary. In many companies, as you are experiencing, there is a tendency among executives to focus more on “CMMI certification.” As you said, this is a misnomer – there’s no official CMMI certificate.

But we can't blame them.  They are simply using the phrase they hear from clients, who say things like, “We require you to have a CMMI certification.” They don’t hear anybody saying, “We want you to be a great company.”

Well, the CMMI Institute wants you to be a great company. They want your clients to be great companies, too.  That's why they worked so hard to put together an outstanding slate of speakers this year. The itinerary has not been released yet, but whichever sessions your executives attend, they will gain valuable perspective on the CMMI. You’ll be able to go back to your company afterward and start to drive the changes you are seeking.

Here’s the registration link for SEPG NA 2013: http://sepgconference.org/register-sepg

Looking forward to seeing you and your executives in Pittsburgh, Mary – and please be sure to introduce yourselves!

What’s the difference between SCAMPI A, SCAMPI B, SCAMPI C?

Dear CMMI Appraiser:

What is the difference between a CMMI SCAMPI A, SCAMPI B and SCAMPI C appraisal? Do they correspond to the CMMI Maturity Level ratings? For example, do we slot to the SCAMPI C, since we want to be a CMMI Level 3 organization? ~ Lisa A.

Hi, Lisa. No, there is no correspondence between the two. An organization can be appraised at Maturity Level 2, Maturity Level 3, ML4 or ML5. But the only class of appraisal that results in a “maturity” rating is a SCAMPI A appraisal.

However, You can think of the three classes of SCAMPI in software terms, as follows:

• SCAMPI C is like a Sprint Demo
• SCAMPI B is like a Systems Test
• SCAMPI A is like a User Acceptance Test

Below are more distinctions between SCAMPI A, SCAMPI B and SCAMPI C ... just in time for lunch!

SCAMPI A , as I said above, is the only class of the method that results in a "maturity" or "capability" rating (such as CMMI Maturity Level 2 or CMMI Capability Level 3 in Project Planning). What’s more, a SCAMPI A is a team-based event which is led by an external "SCAMPI Lead Appraiser" who has been certified by the CMMI Institute and Software Engineering Institute (SEI) to perform SCAMPI Appraisals.

SCAMPI B is also a formal appraisal – albeit less so. This CMMI appraisal serves as a tool to give you the information you need to completely understand your current state in relation to the CMMI. In particular, you’ll have enough information to decide whether you are likely to succeed in the formal SCAMPI A. To be clear, SCAMPI B is an important and highly useful appraisal; it just doesn't result in a rating.

SCAMPI C doesn’t result in a rating either. But it is tremendously useful too. The SCAMPI C is first and foremost a way for your company to find out about itself and how you stack up against the CMMI model. It’s also an opportunity for your CMMI Appraiser to learn as much about your company as he can, and for your company to learn about the CMMI Appraiser. This is important because you will be spending a lot of time together, generally a year or two, making decisions that will have an serious impact on the behaviors of your people. And so, helping both sides feel more comfortable with each other is one of the most beneficial aspects of the SCAMPI C.

As a SCAMPI Lead Appraiser, it's my job to lead a team that evaluates process performance across your organization against the required components of the Model (the Goals) and the expected components (the Practices).

Either way, you should start working with a certified Lead Appraiser early - long before you desire to conduct and appraisal.  That way, you’ll have a track record together when you are ready to begin your SCAMPI A journey. Because that’s what this is, a journey. CMMI isn't a paper-chase. It’s not about getting a so-called “CMMI certification.” It's about making your company great.

Being CMMI Maturity Level 2 or CMMI Maturity Level 3 doesn’t in itself make you a great company. But it does mean that you are sufficiently equipped with the infrastructure and tools you need to become the great company you want to be.

The rest is up to you.

Like this blog? Forward to your nearest engineering or software exec!

Jeff Dalton is a Certified SCAMPI Lead Appraiser, Certified CMMI Instructor, author, and consultant with years of real-world experience with the CMMI in all types of organizations. Jeff has taught thousands of students in CMMI trainings and has received an aggregate satisfaction score of 4.97 out of 5 from his students.

Visit www.broadswordsolutions.com for more information about engineering strategy, performance innovation , software process improvement and running a successful CMMI program.

To download eBooks about CMMI, visit Jeff’s Author Page on Amazon.