What conclusions can we draw about CMMI Users’ experience?

Welcome back to Ask the CMMI Appraiser for the final installment of CMMI User Stories.

If you’ve been following these updates, you are probably aware of the impetus for sharing them. In May, the industry received a breath of fresh air when the SEI announced that the CMMI would no longer be considered a "research" project, and was now “mature”. While it was exciting to learn that the CMMI would be moved to a new business unit within Carnegie Mellon University, and headed up by former Confluence CEO Kirk Botula, many questions remained about the way CMMI Users were experiencing the Model.

Fortunately, Broadsword was in a unique position to provide some answers. Having recently completed our new study of what CMMI Users really think about the CMMI, we had a lot of interesting data at our fingertips. We were pleased to share excerpts from our report.

What conclusions can we draw about the CMMI Experience?

As a result of this study, we have come to several conclusions:

• The vast majority of companies that adopt CMMI received value from the experience. According to 90% of CMMI Users polled, the experience was worthwhile. 

• Only slightly more than half of the CMMI Users said they were motivated by receiving a “CMMI certificate.” Not surprisingly, there were overlapping responses. Many CMMI Users seeking a “CMMI level” told us they also valued continuous improvement. Many CMMI Users who were motivated to be a great company told us they also planned to take advantage of the competitive advantages associated with being “CMMI certified.”  And some said they started with a "certification" approach but as they learned, they transitioned to the more healthy "great company" approach.

• Calculating ROI continues to be difficult. Almost the same percentage of CMMI Users responded that they had received an ROI as those who did not know. This appears to reinforce what CMMI Appraisers and CMMI sponsors alike have suspected for years: that a direct correlation between cost and value of CMMI is difficult to establish. It’s much easier to draw a connection between companies that intend to use the CMMI to radically improve performance, and those that do not.

• Adopting CMMI continues to be a difficult, but worthwhile, effort. One of the biggest challenges facing the new business unit will be to continue setting expectations among CMMI Users that it is not easy to adopt the Model, but well worth the effort.

• Most companies reported a positive experience with their Lead Appraisers. Because the cost of a good Lead Appraiser is a fraction of the reward of a successful process improvement program that puts companies on the path to greatness, it makes sense that most CMMI Users were satisfied.

• An overwhelming majority responded that they will be renewing their rating when it expires. They said the real performance breakthroughs came when they stopped being concerned with "CMMI certification" and thought more about what it took to be a professional company. By contrast, CMMI Users who did not think this way about the CMMI, we found, were among the 5% who reported that they did not intend to renew.

This concludes our CMMI User Story analysis. We hope this effort provides useful insight to anyone using the CMMI. Whether you are an engineering executive leading the CMMI effort in your company, a Lead Appraiser, an SEI Partner or a member of the new CMU business unit, you are now armed with data that can help make the CMMI a better Model for everyone.

Thank you for following our CMMI User Story analysis. For additional tips, techniques, insights and analysis on a changing industry, be sure to return to Ask the CMMI Appraiser.

We’ve also made the information available in an eBook. If you would like to receive the complete set of CMMI User stories in one report, simply subscribe at the top of this page to receive eBooks. Provided that you are using a valid company email address (not a Yahoo, AOL or Gmail account), we will be happy to add you to the distribution list, and send you the eBook, along with others, in coming weeks.

Like this blog? Forward to your nearest engineering or software exec!

Jeff Dalton is a Certified SCAMPI Lead Appraiser, Certified CMMI Instructor, author, and consultant with years of real-world experience with the CMMI in all types of organizations. Jeff has taught thousands of students in CMMI trainings and has received an aggregate satisfaction score of 4.97 out of 5 from his students.

Visit www.broadswordsolutions.com for more information about running a successful CMMI program.

Do you intend to renew your CMMI Maturity Level?

Welcome back to Ask the CMMI Appraiser for today’s installment of CMMI User Stories.

Since the SEI’s announcement about the CMMI raised a call for more detailed information about how people are experiencing the CMMI, Broadsword has been sharing excerpts from our new study of what CMMI users really think about the CMMI.

A quick recap of the questions we’ve asked to date:

What do companies THINK about the CMMI?
After adopting the CMMI, did you experience an increase in quality or performance?
After adopting the CMMI, did you experience a decrease in defects?
Adopting the CMMI ... was it worth it?
Do CMMI Users actually get an ROI?
Adopting the CMMI - how hard is it?
What was your motivation for adopting the CMMI?
Are you satisfied with your CMMI Appraiser?

Today’s post concerns the final question we asked CMMI users:

Do you intend to renew your CMMI Maturity Level? 

Renewal of a CMMI Maturity or Capability Level rating must occur every three years. We wanted to understand what percentage of current adopters intended to renew.

As the chart reveals, the vast majority of CMMI Users intended to renew.

What does this data tell us? Consistent with the previous question, “Are you satisfied with your CMMI Appraiser?” in which over 90% of respondents said that they were happy with their Certified SCAMPI Lead Appraisers, over 90% plan to renew.

Most CMMI Users told us they were aware that the duration between SCAMPI “A” appraisals is three years. However, not every CMMI User is aware that, depending on their company’s current health and readiness, there may be a lot to do before the renewal. For example, this CMMI Appraiser works with many companies that commission a SCAMPI "C" to evaluate their health and readiness before scheduling the higher-cost SCAMPI "A" to renew their rating. Regardless of when they achieved their ML rating, it makes sense for CMMI users to assume they need to start their renewal process ASAP.

A final point among CMMI Users who intended to renew: Many told us that they had the best success with the CMMI when their team stopped looking at the CMMI as something they had to “do,” and started looking at it as a set of best-practices of many successful organizations that could serve as a model for them to be successful. As a result, they stopped focusing on what documents they produced, and focused more on how they were acting. They said the real performance breakthroughs came when they stopped being concerned with "CMMI certification" and thought more about what it took to be a professional company.

By contrast, CMMI Users who did not think this way about the CMMI, we found, were among the 5% who reported that they did not intend to renew.

For our final blog post on CMMI User Story analysis, check back soon right here on Ask the CMMI Appraiser.

We’ve also made the information available in an eBook. If you would like to receive the complete set of user stories all at once, simply subscribe at the top of this page to receive eBooks. Provided that you are using a valid company email address (not a Yahoo, AOL or Gmail account), we will be happy to add you to the distribution list, and send you the eBook, along with others, in coming weeks.

Like this blog? Forward to your nearest engineering or software exec!

Jeff Dalton is a Certified SCAMPI Lead Appraiser, Certified CMMI Instructor, author, and consultant with years of real-world experience with the CMMI in all types of organizations. Jeff has taught thousands of students in CMMI trainings and has received an aggregate satisfaction score of 4.97 out of 5 from his students.

Visit www.broadswordsolutions.com for more information about running a successful CMMI program.

Are you satisfied with your CMMI Appraiser?

Welcome back to Ask the CMMI Appraiser for today’s installment of CMMI User Stories.

As you know from our earlier posts, the SEI’s recent announcement about the CMMI raised a call for more detailed information about how people are experiencing the CMMI. In response, Broadsword decided to release excerpts from our new study of what CMMI users really think about the CMMI. Today’s post concerns the seventh question we asked CMMI users:

Were you satisfied with your experience working with your CMMI appraiser?

Organizations often report different experiences with different lead appraisers. We wanted to understand the level of satisfaction with the lead appraiser experience.

Very few responded that they were not satisfied, or did not know.

What does this data tell us? The market seems to be responding to the fact that most (not all) Certified SCAMPI Lead Appraisers are very experienced. Many have not only been practicing engineers/software developers/project managers with advanced degrees, but have also been executives, leaders, authors, and successful consultants for many years.

For example, this CMMI Appraiser has been a software developer, engineer, project manager, software architect, line manager, Chief Technology Officer, VP of Consulting, and CIO - all before I was a CMMI Appraiser.

So Certified Lead Appraisers are not your typical “CMMI consultants." Many of them can add significant strategic value to companies adopting the CMMI, due to their depth of experience. And because the cost of a Lead Appraiser is a fraction of the reward of a successful process improvement program that puts companies on the path to greatness, it makes sense that most CMMI users were satisfied.

For more CMMI User Story analysis, check back soon right here on Ask the CMMI Appraiser.

We’ve also made the information available in an eBook. If you would like to receive the complete set of user stories all at once, simply subscribe at the top of this page to receive eBooks. Provided that you are using a valid company email address (not a Yahoo, AOL or Gmail account), we will be happy to add you to the distribution list, and send you the eBook, along with others, in coming weeks.

Like this blog? Forward to your nearest engineering or software exec!

Jeff Dalton is a Certified SCAMPI Lead Appraiser, Certified CMMI Instructor, author, and consultant with years of real-world experience with the CMMI in all types of organizations. Jeff has taught thousands of students in CMMI trainings and has received an aggregate satisfaction score of 4.97 out of 5 from his students.

Visit www.broadswordsolutions.com for more information about running a successful CMMI program.

What was your motivation for adopting the CMMI?

Welcome back to Ask the CMMI Appraiser for today’s installment of CMMI User Stories.

If you are like the majority of CMMI practitioners, you've heard the SEI’s recent announcement about the (to be named) new business unit that will be operating the CMMI. The announcement has sparked a desire for more information about how people are currently using the CMMI in their businesses. We here at Broadsword decided to answer the call by sharing excerpts of our new study of what CMMI users really think about the CMMI.

Since our first post on this topic (“What do companies THINK about using the CMMI?”), we’ve received lots of great feedback and requests for more excerpts. So here is the sixth question we asked in our report of CMMI users:

Were you initially motivated to get a “certificate”? 

Now, before anyone gets upset by our choice of words, we do know that the SEI does not recognize the phrase “CMMI certificate.” However, a vast majority of adopters DO use that phrase, whether we like it or not, and it's become part of the lexicon in the community.  As the great Nelson Mandela said, “If you talk to a man in a language he understands, it goes to his head. If you talk to him in his language, it goes to his heart.”  So, that's the word we chose.

By the way, the correct term is "CMMI Rating" or "Appraised at CMMI Maturity Level x."  'nuff said.

Word choice aside, there is much debate in our industry about the motivations behind adopting CMMI. Some adopt the Model strictly for marketing reasons. Some are responding to mandates from their customers. And some focus on following a proven set of guidelines for becoming a great company. We wanted to understand the motivation driving those who adopted CMMI.

We found that less than half said, yes, they were primarily motivated to get a “CMMI certificate.” However, for those in this group that was focused only on "the piece of paper," the promise of marketing advantages did not necessarily translate to new business. One sponsor revealed the irony of his situation by answering, “Yes. Everyone hated it and we didn’t even win the deal.”  But that doesn't tell the whole story.

What does this tell us? It confirms what we here at "Ask the CMMI Appraiser" have been saying all along - "STOP FOCUSING ONLY ON A CERTIFICATE!" The reasons companies choose to adopt the CMMI are fairly evenly split. Not surprisingly, there was a lot of overlap. Several companies told us they were initially motivated to achieve a “CMMI level,” or to “get CMMI certified,” but that they also valued continuous improvement. Others told us that they were initially motivated to be a great company – but that they planned to take advantage of the competitive advantages associated with being “CMMI certified.”

Based on this anecdotal evidence, it seems that regardless of the motivation behind initially adopting the CMMI, those companies that desire to be great are more likely to reach their goals.

For more CMMI User Story analysis, check back soon right here on Ask the CMMI Appraiser.

We’ve also made the information available in an eBook. If you would like to receive the complete set of user stories all at once, simply subscribe at the top of this page to receive eBooks. Provided that you are using a valid company email address (not a Yahoo, AOL or Gmail account), we will be happy to add you to the distribution list, and send you the eBook, along with others, in coming weeks.

Like this blog? Forward to your nearest engineering or software exec!
Jeff Dalton is a Certified SCAMPI Lead Appraiser, Certified CMMI Instructor, author, and consultant with years of real-world experience with the CMMI in all types of organizations. Jeff has taught thousands of students in CMMI trainings and has received an aggregate satisfaction score of 4.97 out of 5 from his students.

Visit www.broadswordsolutions.com for more information about running a successful CMMI program.