What was your motivation for adopting the CMMI?

Welcome back to Ask the CMMI Appraiser for today’s installment of CMMI User Stories.

If you are like the majority of CMMI practitioners, you've heard the SEI’s recent announcement about the (to be named) new business unit that will be operating the CMMI. The announcement has sparked a desire for more information about how people are currently using the CMMI in their businesses. We here at Broadsword decided to answer the call by sharing excerpts of our new study of what CMMI users really think about the CMMI.

Since our first post on this topic (“What do companies THINK about using the CMMI?”), we’ve received lots of great feedback and requests for more excerpts. So here is the sixth question we asked in our report of CMMI users:

Were you initially motivated to get a “certificate”? 

Now, before anyone gets upset by our choice of words, we do know that the SEI does not recognize the phrase “CMMI certificate.” However, a vast majority of adopters DO use that phrase, whether we like it or not, and it's become part of the lexicon in the community.  As the great Nelson Mandela said, “If you talk to a man in a language he understands, it goes to his head. If you talk to him in his language, it goes to his heart.”  So, that's the word we chose.

By the way, the correct term is "CMMI Rating" or "Appraised at CMMI Maturity Level x."  'nuff said.

Word choice aside, there is much debate in our industry about the motivations behind adopting CMMI. Some adopt the Model strictly for marketing reasons. Some are responding to mandates from their customers. And some focus on following a proven set of guidelines for becoming a great company. We wanted to understand the motivation driving those who adopted CMMI.

We found that less than half said, yes, they were primarily motivated to get a “CMMI certificate.” However, for those in this group that was focused only on "the piece of paper," the promise of marketing advantages did not necessarily translate to new business. One sponsor revealed the irony of his situation by answering, “Yes. Everyone hated it and we didn’t even win the deal.”  But that doesn't tell the whole story.

What does this tell us? It confirms what we here at "Ask the CMMI Appraiser" have been saying all along - "STOP FOCUSING ONLY ON A CERTIFICATE!" The reasons companies choose to adopt the CMMI are fairly evenly split. Not surprisingly, there was a lot of overlap. Several companies told us they were initially motivated to achieve a “CMMI level,” or to “get CMMI certified,” but that they also valued continuous improvement. Others told us that they were initially motivated to be a great company – but that they planned to take advantage of the competitive advantages associated with being “CMMI certified.”

Based on this anecdotal evidence, it seems that regardless of the motivation behind initially adopting the CMMI, those companies that desire to be great are more likely to reach their goals.

For more CMMI User Story analysis, check back soon right here on Ask the CMMI Appraiser.

We’ve also made the information available in an eBook. If you would like to receive the complete set of user stories all at once, simply subscribe at the top of this page to receive eBooks. Provided that you are using a valid company email address (not a Yahoo, AOL or Gmail account), we will be happy to add you to the distribution list, and send you the eBook, along with others, in coming weeks.

Like this blog? Forward to your nearest engineering or software exec!
Jeff Dalton is a Certified SCAMPI Lead Appraiser, Certified CMMI Instructor, author, and consultant with years of real-world experience with the CMMI in all types of organizations. Jeff has taught thousands of students in CMMI trainings and has received an aggregate satisfaction score of 4.97 out of 5 from his students.

Visit www.broadswordsolutions.com for more information about running a successful CMMI program.

We need the CMMI because we need one standard process, right?

Hey, CMMI Appraiser, I am an engineering executive in Virginia serving healthcare organizations. We use Agile methods for most of our internal projects, but we now need to become more disciplined. A Virginia CMMI consultant recently told us that our organization needs one standard process. This sounds correct, but while we evaluate our options, what is your opinion? ~ Douglas F.

Hey, Douglas,

I hear that all the time.  Consultants love talking about the CMMI as being all about everybody doing the same thing every time. According to them, the CMMI is repeatable and predictable, and that means that everybody has to have the same behavior. One size fits all.

It sounds correct. But is it?

Unfortunately, the CMMI says nothing of the kind.  What needs to be repeatable are the OUTCOMES.  Many projects are unique and require there own "way of doing things" (otherwise known as the PROCESS) - even though they will have repeatable outcomes (high quality software, efficient production, happy customers).

This is not to downplay the CMMI’s ability to help you bring more discipline to your Agile methods. It is a terrific tool for that purpose. As a matter of fact, it is THE TOOL for that.  However, it does not say you need a single standard process. The CMMI says you should have a SET of standard processes from which each project can derive their unique process.  A “set” means more than one. And “more than one” means you need some flexibility and agility to decide how you are going to do something.

So, to be successful in adopting the CMMI in your Agile environment, you’ll need to understand what the CMMI says and doesn’t say. You also need to know that, in the CMMI, flexibility and agility outweigh rigid compliance.  See my Agile Process Manifesto....

For example, let’s say your development team has a new project. Do they want to use Planning Poker to estimate? Do they want to use Wideband Delphi or Cocomo? Or do they want to use something else? The answer depends on what they are building and the constraints they are dealing with.

"Constraint-based process."  I like it.  Look for more of that in the future!

The ability to provide flexibility is the most important thing you can do when it comes to improvement and deploying process. Rigid compliance hurts you, it doesn’t help you.

The great thing about the CMMI is that it is extremely agile. It gives you the ability – in fact, it mandates – that you have choices and agility. It's right in the book ("Integrated Project Management").

I’ll say it again. A set means more than one, and more than one means there is room for flexibility and agility.

Now, there are some excellent consultants who specialize in the CMMI out there. But there are also many (too many) who can barely spell “CMMI.”

The adoption of CMMI carries with it many complex emotions. Some software and engineering executives feel strongly that they want to improve organizational performance. Sometimes they covet "CMMI certification" – and sometimes they want both.

Whatever your goals, Douglas, it can be a challenge understand how to interpret the true meaning of the CMMI at first. Turning to a qualified CMMI consultant can help alleviate the pain, remove roadblocks, and set you on the right path. But beware! A person promising to relieve you of the baggage by promising “one standard process” may not have the skills and experience you need - so tread carefully.

Anyone can claim to be a CMMI consultant. But only a small amount of people in the world have the requisite skills, training, experience, and certifications that qualify them for the task - and an even smaller group of them have a proven track record.  Think about it - it's not just "CMMI."  It's organizational change, communications, successfully giving (and getting ) advice.  It's complicated!

Good CMMI consultants have a deep holistic understanding of the entire model, how it works, and useful ideas on how to use it to improve your company. They will recommend an approach that bridges agility and discipline, in a manner that fits the way your organization is already doing things.

Watch out for CMMI consultants tell tell you one size fits all.  Next time, just say, "NO!"

Like this blog? Forward to your nearest engineering or software exec!

Jeff Dalton is a Certified SCAMPI Lead Appraiser, Certified CMMI Instructor, author, and consultant with years of real-world experience with the CMMI in all types of organizations. Jeff has taught thousands of students in CMMI trainings and has received an aggregate satisfaction score of 4.97 out of 5 from his students.

Visit www.broadswordsolutions.com for more information about running a successful CMMI program.

Adopting the CMMI - how hard is it?

Welcome back to Ask the CMMI Appraiser for today’s installment of CMMI User Stories.
If you are just joining us, we are in the middle of sharing our new study of what CMMI users really think about the CMMI. (Here’s the post that launched this discussion: “What do companies THINK about using the CMMI?”) Our decision to release this information was prompted by the SEI’s recent announcement about the CMMI, which raised a call for more detailed information about how people are experiencing the tool.

On with our analysis. Here’s the fifth question we asked CMMI users:

Was your experience with the CMMI harder than you expected? 

Why ask this question? Despite the fact that a plurality of respondents said that they had received an ROI after adopting the CMMI (see the CMMI Appraiser's most recent post), many also reported that adoption of the CMMI can be quite complex and difficult. We wanted to understand if that difficulty exceeded the expectations of the organizations that adopt the CMMI.

It was interesting to find that 39% of respondents said the experience was harder than expected, while almost two-thirds said they were prepared for the challenge, and did not find adopting the CMMI harder than expected.

What conclusions can we draw? It appears that the SEI and its Partners have done a good job setting expectations, for the most part. The greater percentage of respondents felt that the adoption was difficult, but no more difficult than anticipated.

By the end of the year, the CMMI constellations (as well as other "mature technologies" that are no longer considered "research" projects) will be moved to the new business unit within Carnegie Mellon University (yet to be named). One of the challenges facing this new business unit will be to continue setting expectations among CMMI Users that it is not easy to adopt the Model, but well worth the effort.

For more CMMI User Story analysis, check back soon right here on Ask the CMMI Appraiser.

We’ve also made the information available in an eBook. If you would like to receive the complete set of user stories all at once, simply subscribe at the top of this page to receive eBooks. Provided that you are using a valid company email address (not a Yahoo, AOL or Gmail account), we will be happy to add you to the distribution list, and send you the eBook, along with others, in coming weeks.

Like this blog? Forward to your nearest engineering or software exec!
Jeff Dalton is a Certified SCAMPI Lead Appraiser, Certified CMMI Instructor, author, and consultant with years of real-world experience with the CMMI in all types of organizations. Jeff has taught thousands of students in CMMI trainings and has received an aggregate satisfaction score of 4.97 out of 5 from his students.

Visit www.broadswordsolutions.com for more information about running a successful CMMI program.

Do CMMI Users actually get an ROI?

Welcome back to Ask the CMMI Appraiser for today’s installment of CMMI User Stories.

In an industry full of inquisitive problem-solvers, it was no surprise that the SEI’s announcement about the CMMI triggered so many requests for more information about how people are using the CMMI. After all, folks like you and me need data to help us evaluate where we are in our ongoing quest to become -- and remain -- great companies.

As you know if you are a regular visitor at Ask the CMMI Appraiser, over the past several days, we’ve been sharing excerpts from our brand new study of what CMMI users from 50 companies really think about the CMMI.

Today, we present the fourth question we asked CMMI users:

Did you experience an ROI?

As described in our last Ask the CMMI Appraiser post, adopting the CMMI requires time and effort. And while 90% of CMMI Users said it was worth it, the CMMI can be quite costly. How do CMMI Users feel about their return on Investment?

We learned from many CMMI Users that it can be difficult to ascertain an aggregate ROI, due to the wide degree of variability in products and services delivered by the adopters of CMMI. Even so, we wanted to understand the market’s perception of CMMI, and whether sponsors thought they had received an ROI (internally, we referred to this metric as the “cost factor”).

Over 40% said yes, they had seen an ROI; 40% said they had no idea; 15% said no and 3% said "no comment."

Among those who tracked ROI, most were positive. One respondent told us: “it was one of our most profitable divisions.”

What can we glean from this information? It’s interesting that the same percentage responded that they had received an ROI as those who did not know. This appears to reinforce what CMMI Appraisers and CMMI sponsors alike have suspected for years: that a direct correlation between cost and value of CMMI is difficult to establish. It’s much easier to draw a connection between companies that intend to use the CMMI to radically improve performance, and those that do not.

It’s also interesting to reflect on those who said “no comment.” Were they really in the “I don’t know” category, but unwilling to admit it?  Or were they concerned that they may appear to have sipped the kool-aid?

For more CMMI User Story analysis, check back soon right here on Ask the CMMI Appraiser.

We’ve also made the information available in an eBook. If you would like to receive the complete set of user stories all at once, simply subscribe at the top of this page to receive eBooks. Provided that you are using a valid company email address (not a Yahoo, AOL or Gmail account), we will be happy to add you to the distribution list, and send you the eBook, along with others, in coming weeks.

Like this blog? Forward to your nearest engineering or software exec!

Jeff Dalton is a Certified SCAMPI Lead Appraiser, Certified CMMI Instructor, author, and consultant with years of real-world experience with the CMMI in all types of organizations. Jeff has taught thousands of students in CMMI trainings and has received an aggregate satisfaction score of 4.97 out of 5 from his students.

Visit www.broadswordsolutions.com for more information about running a successful CMMI program.

Continuously improve? What the $%^&! does THAT mean?

Hey, CMMI Appraiser – we are an electronics parts manufacturer in Virginia that recently merged with an organization that follows the CMMI. Even though they acquired us because of our Agile approach, my new manager doesn’t understand Scrum at all. He keeps asking me to push my team to “continuously improve” and “get better” at what they do, and “do more with less.” What the heck does that mean? We’re just trying to build great products, same as always. ~ Dan L.

Hey, Dan, so, your best isn’t good enough, you don’t know why,  and you don’t know what your manager's expectations are for what improvement looks like.  Hasn't he trained you to read his mind?  That would fall under GP2.5 by the way . . . .

I remember earlier in my career when I was the CIO of a company and I heard the same thing.  "Get better!  Make sure your team is continually improving!"  Like you, I said, What does that mean? Come in earlier? Work harder? Be more serious? (They actually said that to me one time. I responded by being LESS serious).

So you are not alone with this problem, my friend. The cool thing is, since then, I’ve done a lot of work with companies that have embraced the CMMI and Scrum in an effort to get better, and I've learned to help demystify things a bit.

In my opinion, the clearest way to think about Scrum and CMMI is in terms of process improvement, not running teams or projects. That keeps us focused on the only project we really care about, the one called, “Making our company Great.” Scrum can help companies adopt things like CMMI, ISO and continuous improvement programs in general.

Now, I don’t claim to be a mind-reader, but that could be exactly what your manager wants: Taking Agile and making it better by applying the lessons of the CMMI.  And conversely, taking the CMMI and making it better with Agile.  If you're getting a recursion headache now, don't worry.  It'll subside.

If so, he's onto something, although maybe not articulating it very well.  This is an idea I’m really passionate about. I believe all organizations can benefit from rethinking the CMMI and Scrum and how they work. It boils down to three simple concepts:

First, the CMMI is a behavioral model, not a process. It was intended to help make things better, a guide to continuously improve. It describes how great companies perform.

Second, as such, the CMMI doesn’t tell you HOW to get better. In fact, the CMMI doesn’t tell you how to do anything. It says, “Here’s what great companies have told us that they do.”  Your job is to apply these lessons to what you are doing in your context.  Tricky?  Yes, but well worth it.

Finally, the truth about CMMI and Scrum is that they are both the "same thing."  They are tools to help solve business problems. They help us improve  requirements churn and volatility, they help us meet schedule and budget, and they help us perform the work that we do every day.  They're not overhead - they're "underhead."

So relax, Dan.  You’ve got everything you need.  Leverage Scrum to get the best of the CMMI. And leverage the CMMI to get the best of Scrum.

If that’s what your manager is thinking, then I see a happy, productive company in your future.

Like this blog? Forward to your nearest engineering or software exec!

Jeff Dalton is a Certified SCAMPI Lead Appraiser, Certified CMMI Instructor, author, and consultant with years of real-world experience with the CMMI in all types of organizations. Jeff has taught thousands of students in CMMI trainings and has received an aggregate satisfaction score of 4.97 out of 5 from his students.

Visit www.broadswordsolutions.com for more information about running a successful CMMI program.

Adopting the CMMI ... was it worth it?

Welcome back to Ask the CMMI Appraiser for today’s installment of CMMI User Stories.
As you know by now, in response to the SEI’s announcement about the CMMI, we decided to provide more information about the experiences CMMI users are having.  Below is an excerpt from our brand new study of what CMMI users from 50 companies really think about the CMMI.

The study consisted of calling CMMI users and asking them about their experiences with the CMMI, good and bad. Here’s our third question:

Was adopting the CMMI worth doing?

Why ask this question? Because, in the early years of the CMMI, the US Department of Defense (DoD) and other early adopters of CMMI, like Lockheed Martin, Rockwell Collins, Boeing and Air Force, were known for conducting top-down, document-heavy, “command and control" software projects. This reputation carried over as these same organizations embraced CMMI. As a result, many smaller, nimbler businesses thought that adopting the CMMI was not a "good fit" for them and was not worth it.

Since then, perception of the CMMI model has changed. The CMMI has evolved beyond the "oppressive interpretation."  A more flexible, agile approach to the model is becoming the new standard.

Despite this change, the CMMI is still time-consuming and resource-intensive to adopt (but well worth it).  Change like this can be disruptive to the organization.  And so we wanted to understand if sponsors thought the overall effort was worth it (internally, we referred to this metric as the “hassle factor.”)

90% said the CMMI effort was worth doing.

10% said the CMMI effort was not worth doing.

What can we take away from this response? Most companies did not experience the CMMI as being oppressive. It may have required a concerted effort to adopt the CMMI, but the vast majority of CMMI users said that adopting it was worthwhile.

Be sure to check back regularly as we share results of the CMMI User Story study right here on Ask the CMMI Appraiser.

We’ve also made the information available in an eBook. If you would like to receive the complete set of user stories all at once, simply subscribe at the top of this page to receive eBooks. Provided that you are using a valid company email address (not a Yahoo, AOL or Gmail account), we will be happy to add you to the distribution list, and send you the eBook, along with others, in coming weeks.

Like this blog? Forward to your nearest engineering or software exec!
Jeff Dalton is a Certified SCAMPI Lead Appraiser, Certified CMMI Instructor, author, and consultant with years of real-world experience with the CMMI in all types of organizations. Jeff has taught thousands of students in CMMI trainings and has received an aggregate satisfaction score of 4.97 out of 5 from his students.

Visit www.broadswordsolutions.com for more information about running a successful CMMI program.

After adopting the CMMI, did you experience a decrease in defects?

Welcome back to Ask the CMMI Appraiser for today’s installment of CMMI User Stories.

As you are probably aware by now (especially if you read my post, “What do companies THINK about using the CMMI?”), the SEI’s announcement about the CMMI created tremendous demand for more information about the experiences CMMI users are having. In response, we are sharing our brand new study of what CMMI users from 50 companies really think about the CMMI.

Here’s the second question we asked CMMI users about their experiences with the CMMI:

Did you experience a decrease in defects?

Why this question? Because sometimes problems occur again and again and that leads software engineering organizations to CMMI (or other process model) adoption. For example, when we first start working with companies, we see that they struggle with late projects, over budget projects, unhappy customers and unhappy employees. These problems are usually coupled with excessive defects in the software they are developing.

The CMMI provides guidelines for deploying processes that improve product quality and reduce defects. We wanted to understand whether organizations actually experienced these benefits through their adoption of the CMMI.

Fully half of respondents said that defects had decreased, sometimes dramatically. Fewer than half said, no, defects had not decreased. 12% had no idea (no idea?  Don't get me started on THAT one!),

What can we take away from this response?  In our conversations with those who said, “No, we did not experience a decrease in defects,” we were surprised - but perhaps they were ALREADY doing the very things that the CMMI guides us to do (a common occurrence).  It makes sense though - great companies often do great things without adopting a process model - that's why they're great companies, and that's kind of the point!

Be sure to check back soon here on Ask the CMMI Appraiser for more insight from our CMMI User Stories.

We’ve also made the information available in an eBook. If you would like to receive the complete set of user stories all at once, including the narratives from each interview, simply subscribe at the top of this page to receive eBooks. Provided that you are using a valid company email address (not a Yahoo, AOL or Gmail account), we will be happy to add you to the distribution list, and send you the eBook, along with others, in coming weeks.

Like this blog? Forward to your nearest engineering or software exec!

Jeff Dalton is a Certified SCAMPI Lead Appraiser, Certified CMMI Instructor, author, and consultant with years of real-world experience with the CMMI in all types of organizations. Jeff has taught thousands of students in CMMI trainings and has received an aggregate satisfaction score of 4.97 out of 5 from his students.

Visit www.broadswordsolutions.com for more information about running a successful CMMI program.

 

After adopting the CMMI, did you experience an increase in quality or performance?

Welcome back to Ask the CMMI Appraiser for today’s installment of CMMI User Stories.

As described in my last post (What Do Companies THINK about the CMMI?), in response to the demand for information created by the SEI’s announcement about the CMMI (in a nutshell, the CMMI is being graduated from "research" status to “mature” status), we are sharing our study of what CMMI users really think about the CMMI.

The study consisted of calling CMMI users and asking them about their experiences with the CMMI, good and bad. Here’s our first question:

Did you experience an increase in quality or performance?

Why do we start with this question? Because different companies have different expectations of the CMMI. Some seek CMMI “levels” for the marketing advantages, or to satisfy customer requirements. Others strive for continuous process and performance improvement; they see the CMMI as the tool they use for guidance.

In both cases, the strength of the CMMI is that it provides organizations with guidelines for increasing performance and improving the quality of products and services. We wanted to know whether organizations actually achieved improvements in performance and quality.

81% of respondents said they improved in these areas.

 

What can we take away from this response? Of those who responded “I don’t know,” many told us that they weren’t tracking quality or performance increases. Many of those who said, “No,” indicated that they were already operating at a high level and the CMMI helped them standardize their excellence in performance and quality.

Most of CMMI Users experienced an increase in quality and performance if they were looking for it.

Be sure to check back regularly as we share results of the CMMI User Story study right here on Ask the CMMI Appraiser.

We’ve also made the information available in an eBook. If you would like to receive the complete set of user stories all at once, simply subscribe at the top of this page to receive eBooks. Provided that you are using a valid company email address (not a Yahoo, AOL or Gmail account), we will be happy to add you to the distribution list, and send you the eBook, along with others, in coming weeks.

Like this blog? Forward to your nearest engineering or software exec!

Jeff Dalton is a Certified SCAMPI Lead Appraiser, Certified CMMI Instructor, author, and consultant with years of real-world experience with the CMMI in all types of organizations. Jeff has taught thousands of students in CMMI trainings and has received an aggregate satisfaction score of 4.97 out of 5 from his students.

Visit www.broadswordsolutions.com for more information about running a successful CMMI program.

What do companies THINK about the CMMI?

Hey, CMMI Appraiser, now that the CMMI is moving to a new business unit within CMU, wouldn’t it be great if we knew what people really thought about using the CMMI? ~ a fellow CMMI Appraiser 

Hey, fellow CMMI Appraiser,

Yes, that’s the million dollar question, isn’t it? A lot of people are keenly interested in the perception of the CMMI in the market today. Especially now that the SEI has announced that the CMMI constellations (as well as other "mature technologies") will no longer be considered "research" projects, and will be moved to a new business unit within Carnegie Mellon University (CMU).

As I’ve said, I interpret this as a good … no, a great thing. The Department of Defense, long a sponsor of the SEI, was very involved (along with CMU) in the decision. The new business unit will be unencumbered by the constraints of the DoD, and will take the CMMI in new, more flexible directions, and to new markets.

And we can’t say no one saw this coming. For the past few years, at SEPG NA conferences, several of my fellow Lead Appraisers and I have been talking about the need for change in the market. We saw CMMI adoption increasing for large and small companies alike. We thought our clients were reaping tremendous rewards. But we had no idea what other CMMI users, those we didn't know, actually thought about using the tool.

In the months leading up to the SEI’s CMMI announcement, I decided to commission a User Story study. My company, Broadsword, selected 50 companies randomly from the SEI’s Published Appraisal Results (PARs). The companies had achieved CMMI Maturity Level 2 or 3 between 2007 and 2010.

The reason we chose this timeframe was to give the CMMI users some period of time with their adoption, allowing for more objectivity.

Next, we retained two call center representatives, who picked up the phone and started calling.

Over several days, we spoke with CEOs, VPs and Quality Assurance Managers of North American companies, large and small. Companies were in the aerospace, defense, finance, transportation, energy and manufacturing industries, and they were all using CMMI-DEV.

We listened to the stories they told about using the CMMI. We captured the data, and analyzed the results.

As far as we know, this is the first study of its kind that builds its case on anecdotal data from actual end users. We gathered stories about what’s working and what’s not working with the CMMI, directly from the users themselves.

I hope the timeliness of this information will prove valuable to you, whatever your role with the CMMI. It was finalized literally days before the SEI announcement.

And so I invite you to check back regularly as we share results of the CMMI User Story study right here on Ask the CMMI Appraiser.

We’ve also made the information available in an eBook. If you would like to receive the complete set of user stories all at once, simply subscribe to reveive eBooks at the top of this page.  Provided that you are using a valid company email address (not a yahoo, aol or gmail account), we will be happy to add you to the distibution list, and send you the eBook, along with others, in coming weeks.

Like this blog? Forward to your nearest engineering or software exec!

Jeff Dalton is a Certified SCAMPI Lead Appraiser, Certified CMMI Instructor, author, and consultant with years of real-world experience with the CMMI in all types of organizations. Jeff has taught thousands of students in CMMI trainings and has received an aggregate satisfaction score of 4.97 out of 5 from his students.

Visit www.broadswordsolutions.com for more information about running a successful CMMI program.