What conclusions can we draw about CMMI Users’ experience?

Welcome back to Ask the CMMI Appraiser for the final installment of CMMI User Stories.

If you’ve been following these updates, you are probably aware of the impetus for sharing them. In May, the industry received a breath of fresh air when the SEI announced that the CMMI would no longer be considered a "research" project, and was now “mature”. While it was exciting to learn that the CMMI would be moved to a new business unit within Carnegie Mellon University, and headed up by former Confluence CEO Kirk Botula, many questions remained about the way CMMI Users were experiencing the Model.

Fortunately, Broadsword was in a unique position to provide some answers. Having recently completed our new study of what CMMI Users really think about the CMMI, we had a lot of interesting data at our fingertips. We were pleased to share excerpts from our report.

What conclusions can we draw about the CMMI Experience?

As a result of this study, we have come to several conclusions:

• The vast majority of companies that adopt CMMI received value from the experience. According to 90% of CMMI Users polled, the experience was worthwhile. 

• Only slightly more than half of the CMMI Users said they were motivated by receiving a “CMMI certificate.” Not surprisingly, there were overlapping responses. Many CMMI Users seeking a “CMMI level” told us they also valued continuous improvement. Many CMMI Users who were motivated to be a great company told us they also planned to take advantage of the competitive advantages associated with being “CMMI certified.”  And some said they started with a "certification" approach but as they learned, they transitioned to the more healthy "great company" approach.

• Calculating ROI continues to be difficult. Almost the same percentage of CMMI Users responded that they had received an ROI as those who did not know. This appears to reinforce what CMMI Appraisers and CMMI sponsors alike have suspected for years: that a direct correlation between cost and value of CMMI is difficult to establish. It’s much easier to draw a connection between companies that intend to use the CMMI to radically improve performance, and those that do not.

• Adopting CMMI continues to be a difficult, but worthwhile, effort. One of the biggest challenges facing the new business unit will be to continue setting expectations among CMMI Users that it is not easy to adopt the Model, but well worth the effort.

• Most companies reported a positive experience with their Lead Appraisers. Because the cost of a good Lead Appraiser is a fraction of the reward of a successful process improvement program that puts companies on the path to greatness, it makes sense that most CMMI Users were satisfied.

• An overwhelming majority responded that they will be renewing their rating when it expires. They said the real performance breakthroughs came when they stopped being concerned with "CMMI certification" and thought more about what it took to be a professional company. By contrast, CMMI Users who did not think this way about the CMMI, we found, were among the 5% who reported that they did not intend to renew.

This concludes our CMMI User Story analysis. We hope this effort provides useful insight to anyone using the CMMI. Whether you are an engineering executive leading the CMMI effort in your company, a Lead Appraiser, an SEI Partner or a member of the new CMU business unit, you are now armed with data that can help make the CMMI a better Model for everyone.

Thank you for following our CMMI User Story analysis. For additional tips, techniques, insights and analysis on a changing industry, be sure to return to Ask the CMMI Appraiser.

We’ve also made the information available in an eBook. If you would like to receive the complete set of CMMI User stories in one report, simply subscribe at the top of this page to receive eBooks. Provided that you are using a valid company email address (not a Yahoo, AOL or Gmail account), we will be happy to add you to the distribution list, and send you the eBook, along with others, in coming weeks.

Like this blog? Forward to your nearest engineering or software exec!

Jeff Dalton is a Certified SCAMPI Lead Appraiser, Certified CMMI Instructor, author, and consultant with years of real-world experience with the CMMI in all types of organizations. Jeff has taught thousands of students in CMMI trainings and has received an aggregate satisfaction score of 4.97 out of 5 from his students.

Visit www.broadswordsolutions.com for more information about running a successful CMMI program.

Is adopting the CMMI more of a science or an art?

Hey, CMMI Appraiser – settle a bet between me and a fellow Michigan software engineering executive. Is adopting the CMMI more of a science or an art? I say science, because of its rigor and discipline, and he says art, because, he says, the Model can be interpreted differently for each company that adopts it. Who’s right? ~ Isaac F.

Yes!

Isaac, it’s both. Taking the journey to become a great company by adopting the CMMI is both an art and a science. So you both win.

Most people think art and science are polar opposites. Whereas art is expressive, emotive, decorative, or conceptual, science is logical, ordered, practical and functional. How could they possibly overlap?

To illustrate this dynamic, let me share a personal passion of mine.

This CMMI Appraiser started his career journey as a classical musician. I hold degrees in both Music and Computer Science. Today, in addition to running an Agile process innovation firm, Broadsword, I am a Certified Lead Appraiser, CMMI Instructor, ScrumMaster and author of “agileCMMI,” Broadsword’s leading methodology for incremental and iterative process improvement.  And I also play music semi-professionally.

Here’s my secret: I see most things as a musical orchestration. When I wrote software, it was like playing the computer – composing a piece and playing it on a complex instrument.

Today, when I walk into a company, I see an orchestra and a composition that is in need of arrangement and rehearsal. I see my job as a behind-the-scenes conductor or coach to the orchestra, a sectional coach, if you will, who goes around making sure all of the different sections (brass section, string section, etc.) are working together. Then I tap my baton on the stand and say, “Time to play,” and you hear beautiful music come out the other side.

Sounds like art … but there’s a whole lot of science that goes into a CMMI-based process improvement program before you can synthesize something new.

Charlie "Bird" Parker, my favorite jazz musician, did that. Charlie was known for his genius in playing fast and technically accurate.  For Bird (and for me) music is about engineering the craft. You learn the theory, history and all of the academics, which includes theory, drilling and finger exercises. After you master that, only then you can be creative.

A perfect example of an organization that does this is CutTime Productions (www.cuttime.com). Led by Detroit Symphony Orchestra bassist Rick Robinson, CutTime Productions is an artistic venture committed to connecting curious but inexperienced music lovers with the beauty and power of classical music in new and innovative ways. Broadsword recently became a supporter of CutTime, helping them cover the startup costs of traveling the country and introducing their music and methods nationally.

CutTime has the ability to apply science and discipline, and layer creativity on top of it. In that sense, CutTime Productions does for classical music what Charlie Parker did for Jazz, and what Broadsword does for software engineering. We are all after the same wonderful, paradoxical goal:

Applying science and art on the path to greatness.

Like this blog? Forward to your nearest engineering or software exec!

Jeff Dalton is a Certified SCAMPI Lead Appraiser, Certified CMMI Instructor, author, and consultant with years of real-world experience with the CMMI in all types of organizations. Jeff has taught thousands of students in CMMI trainings and has received an aggregate satisfaction score of 4.97 out of 5 from his students.

Visit www.broadswordsolutions.com for more information about running a successful CMMI program.

It’s time for another Webinar! CMMI – Everything You Need to Know

Need to learn EVERYTHING about CMMI? Now's your chance . . . and it's FREEEEEEE!

Join us for “CMMI - Everything You Need to Know” on Friday, July 20, 2012 at 2PM EDT, a free Webinar hosted by Broadsword Solutions Corporation.

Who should participate? If you are an executive, engineer and/or business professional who is trying to create an environment in which your organization can manage its uniqueness in a structured way, this Webinar is designed specifically for you.

Because what is the CMMI all about, anyway? The CMMI is about the transformation of the culture of your company. It’s about improving and changing the way your company behaves, so that you build better products, win new business and retain the customers you have. This fast-paced, indepth Webinar shows how to use the CMMI to reach these outcomes, and more.

Even if you have been working with CMMI for a while, you’ll want to check out the Webinar. You are bound to hear some ideas that you haven't considered before, that will help you get better at what you are ALREADY doing.

Your speaker is Jeff Dalton of "Ask the CMMI Appraiser." Jeff is a Certified Lead Appraiser and CMMI Instructor. He has conducted hundreds of CMMI Appraisals and taught thousands of students in his CMMI Training classes. He is a process innovation thought-leader and CMMI Consultant who has been a guest speaker at conferences and workshops around the world.

Don't miss this informative event! Register here. 

Like this blog? Forward to your nearest engineering or software exec!

Jeff Dalton is a Certified SCAMPI Lead Appraiser, Certified CMMI Instructor, author, and consultant with years of real-world experience with the CMMI in all types of organizations. Jeff has taught thousands of students in CMMI trainings and has received an aggregate satisfaction score of 4.97 out of 5 from his students.

Visit www.broadswordsolutions.com for more information about running a successful CMMI program.

Do you intend to renew your CMMI Maturity Level?

Welcome back to Ask the CMMI Appraiser for today’s installment of CMMI User Stories.

Since the SEI’s announcement about the CMMI raised a call for more detailed information about how people are experiencing the CMMI, Broadsword has been sharing excerpts from our new study of what CMMI users really think about the CMMI.

A quick recap of the questions we’ve asked to date:

What do companies THINK about the CMMI?
After adopting the CMMI, did you experience an increase in quality or performance?
After adopting the CMMI, did you experience a decrease in defects?
Adopting the CMMI ... was it worth it?
Do CMMI Users actually get an ROI?
Adopting the CMMI - how hard is it?
What was your motivation for adopting the CMMI?
Are you satisfied with your CMMI Appraiser?

Today’s post concerns the final question we asked CMMI users:

Do you intend to renew your CMMI Maturity Level? 

Renewal of a CMMI Maturity or Capability Level rating must occur every three years. We wanted to understand what percentage of current adopters intended to renew.

As the chart reveals, the vast majority of CMMI Users intended to renew.

What does this data tell us? Consistent with the previous question, “Are you satisfied with your CMMI Appraiser?” in which over 90% of respondents said that they were happy with their Certified SCAMPI Lead Appraisers, over 90% plan to renew.

Most CMMI Users told us they were aware that the duration between SCAMPI “A” appraisals is three years. However, not every CMMI User is aware that, depending on their company’s current health and readiness, there may be a lot to do before the renewal. For example, this CMMI Appraiser works with many companies that commission a SCAMPI "C" to evaluate their health and readiness before scheduling the higher-cost SCAMPI "A" to renew their rating. Regardless of when they achieved their ML rating, it makes sense for CMMI users to assume they need to start their renewal process ASAP.

A final point among CMMI Users who intended to renew: Many told us that they had the best success with the CMMI when their team stopped looking at the CMMI as something they had to “do,” and started looking at it as a set of best-practices of many successful organizations that could serve as a model for them to be successful. As a result, they stopped focusing on what documents they produced, and focused more on how they were acting. They said the real performance breakthroughs came when they stopped being concerned with "CMMI certification" and thought more about what it took to be a professional company.

By contrast, CMMI Users who did not think this way about the CMMI, we found, were among the 5% who reported that they did not intend to renew.

For our final blog post on CMMI User Story analysis, check back soon right here on Ask the CMMI Appraiser.

We’ve also made the information available in an eBook. If you would like to receive the complete set of user stories all at once, simply subscribe at the top of this page to receive eBooks. Provided that you are using a valid company email address (not a Yahoo, AOL or Gmail account), we will be happy to add you to the distribution list, and send you the eBook, along with others, in coming weeks.

Like this blog? Forward to your nearest engineering or software exec!

Jeff Dalton is a Certified SCAMPI Lead Appraiser, Certified CMMI Instructor, author, and consultant with years of real-world experience with the CMMI in all types of organizations. Jeff has taught thousands of students in CMMI trainings and has received an aggregate satisfaction score of 4.97 out of 5 from his students.

Visit www.broadswordsolutions.com for more information about running a successful CMMI program.

What’s the fastest way to get buy-in of new processes?

Hey, CMMI Appraiser – just wanted to update you since my last question to “Ask the CMMI Appraiser.” You may recall that our SEPG took your Introduction to CMMI training last fall.  Now we are designing new processes. What’s the fastest way to get buy-in for these new processes, company-wide? ~ Ross O.

Hey, Ross – great to hear from you again! Helping people transform company culture with CMMI is an area of expertise of Laura Adkins, a Senior CMMI Consultant with Broadsword. Laura does amazing work with companies who are trying to institutionalize new processes within the Agile and CMMI frameworks. Take it away, Laura! ~ The CMMI Appraiser

Thanks, CMMI Appraiser!

Ross, since you’ve been to our CMMI trainings, you are familiar with Broadsword’s vision for using the CMMI to change the way your company behaves. The goal is to make this transition a positive experience, so that everyone buys in. One way to accelerate buy-in – or “institutionalize” – is with pilot testing.

Why does pilot testing accelerate buy-in? There are many reasons. The first reason is: everyone who participates in a pilot test becomes a champion.

Like the athletes in this picture, champions see the value of processes that work. They have been trained and mentored and can spread the word on why they love the new processes. This is important because, the more people you have supporting your new processes, the better. It’s how you create early buy-in.

Having the support and endorsement of champions means a lot. We’ve found that users are far more likely to try out a new process if they know it was pilot tested first. For example, a project manager who has used the new processes in pilot testing will spread the word to other project managers. And he will talk to his peers in a different manner from how the SEPG will be rolling the new processes out. He’ll say things like, “Hey, it really wasn’t that bad. It was easier than you think.” So having champions really helps disseminate the new processes and knowledge among others, in a way that they will trust.

The second reason pilot testing accelerates institutionalization is the simple fact that you’ve already trained people for the pilot. Not only are they ready for the new processes ahead of time, but you now have fewer people left to train. That helps accelerate adoption.

The third reason pilot testing gets everyone to buy in faster is that it decreases the risk of unintended consequences. Sometimes we make assumptions when we develop processes, assumptions about what types of projects are using the processes, or what will work. If you do this in a vacuum, where you haven’t solicited everyone’s feedback, you may be in for some unpleasant surprises. You don’t know what you don’t know, as they say.

Pilot testing helps you know. Use pilot testing to validate that you have made correct assumptions, and that the new processes will be a good fit for your organization.

That’s the best way to get buy-in quickly.

Like this blog? Forward to your nearest engineering or software exec!

Laura Adkins is a Senior CMMI Consultant with Broadsword Solutions Corporation. She has years of real world experience using pilot testing strategies and tactics to help her clients achieve their goals. Laura also uses the CMMI, in partnership with her clients, to set-up, monitor, and sustain process improvement programs.

Visit www.broadswordsolutions.com for more information about running a successful CMMI program.

Posted by Jeff Dalton at 1:39 PM 0 comments 

Does the Agile Manifesto "forbid" writing down requirements?

Hey, CMMI Appraiser – I’m a member of an an agile team for a Maryland contractor to the National Institutes of Health. We just started designing a fairly complex financial system for NIH, but one of my teammates is adamantly against writing down changes to the requirements. He says the Agile Manifesto forbids it – but our boss clearly thinks it’s important. Does the Agile Manifesto really forbid writing down the requirements? ~ Carol W.

Carol, I had the same conversation with an XP team a couple of weeks ago. They said, “We don’t see any value in writing down the changes to the requirements. Business value is in our software, not in documents.”

Suffering through unclear requirements is not a question of "if" but "when" on any project team.  In my experience, anyone who says that has never suffered from the confusion and chaos that inevitably occurs when a team loses track of the current state of requirements (just ask any tester or customer).  In the case of your colleague, I strongly suspect he is mistaking the Agile Manifesto, a very well thought out, disciplined set of guidelines, for the "Lazy Manifesto."  Only one of those approaches is going to help make your company great (hint: it's not the Lazy Manifesto).  Let's take a look at what the Agile Manifesto actually says (with compliments to Sutherland, Jeffries, Beck, Schwaber, Beedle,  Cockburn, et al):

No doubt you've seen this before.  At this point, the Agile Manifesto is so familiar that a mythology has been built up around it. After listening to so many people talk about it I get this mental image of a gang of brainy geeks climbing up a mountain-side, complete with crampons and ice picks, setting up a smoke lodge, and sitting there on the mountaintop for a couple days. Smoke fills the teepee, and on the last day, out comes a puff of white smoke along with the Agile Manifesto, along with a torrent of hundreds of conferences and books!  Everyone in the teepee goes on to become a superstar and software is defect free and on-time forever!  Did I mention the rainbows and unicorns?

It turns out these folks got many things right … and one thing REALLY right.  Collaboration and communication  (and the structure to encourage them) are pivotal for creating great software.

They were also smart enough, as the founding fathers and mothers of "Agile," to say about the stuff on the right: “This stuff is important too.”

But somewhere along the way, too many people took a paper cutter to this document, and focused only on the stuff on the left.  This may be why your colleague is confused.  This is where comments like "business value only exists in code" came from.  That is ONE of the places business value exists, perhaps the most important, but not the only place.

So, IMHO, it is important to write down requirements.  Does it have to be a huge, heavy specification?  No, it can be as simple as sticky notes or maybe in a tool like iScrum or Process Focus.  Applying discipline to your Agile methods will help your team, customer, and boss understand the business issues at stake. This will help make your team more successful and put you in position to take on larger, more complex projects.

To be clear, when I say "apply discipline," I don't mean rigidity and useless documentation.  Of course we care more about the stuff on the left.  Just remember the stuff on the right is really important too.

Sounds like your colleague is about to get his first experience with that.

Like this blog? Forward to your nearest engineering or software exec!

Jeff Dalton is a Certified SCAMPI Lead Appraiser, Certified CMMI Instructor, author, and consultant with years of real-world experience with the CMMI in all types of organizations. Jeff has taught thousands of students in CMMI trainings and has received an aggregate satisfaction score of 4.97 out of 5 from his students.

Visit www.broadswordsolutions.com for more information about running a successful CMMI program.

Are you satisfied with your CMMI Appraiser?

Welcome back to Ask the CMMI Appraiser for today’s installment of CMMI User Stories.

As you know from our earlier posts, the SEI’s recent announcement about the CMMI raised a call for more detailed information about how people are experiencing the CMMI. In response, Broadsword decided to release excerpts from our new study of what CMMI users really think about the CMMI. Today’s post concerns the seventh question we asked CMMI users:

Were you satisfied with your experience working with your CMMI appraiser?

Organizations often report different experiences with different lead appraisers. We wanted to understand the level of satisfaction with the lead appraiser experience.

Very few responded that they were not satisfied, or did not know.

What does this data tell us? The market seems to be responding to the fact that most (not all) Certified SCAMPI Lead Appraisers are very experienced. Many have not only been practicing engineers/software developers/project managers with advanced degrees, but have also been executives, leaders, authors, and successful consultants for many years.

For example, this CMMI Appraiser has been a software developer, engineer, project manager, software architect, line manager, Chief Technology Officer, VP of Consulting, and CIO - all before I was a CMMI Appraiser.

So Certified Lead Appraisers are not your typical “CMMI consultants." Many of them can add significant strategic value to companies adopting the CMMI, due to their depth of experience. And because the cost of a Lead Appraiser is a fraction of the reward of a successful process improvement program that puts companies on the path to greatness, it makes sense that most CMMI users were satisfied.

For more CMMI User Story analysis, check back soon right here on Ask the CMMI Appraiser.

We’ve also made the information available in an eBook. If you would like to receive the complete set of user stories all at once, simply subscribe at the top of this page to receive eBooks. Provided that you are using a valid company email address (not a Yahoo, AOL or Gmail account), we will be happy to add you to the distribution list, and send you the eBook, along with others, in coming weeks.

Like this blog? Forward to your nearest engineering or software exec!

Jeff Dalton is a Certified SCAMPI Lead Appraiser, Certified CMMI Instructor, author, and consultant with years of real-world experience with the CMMI in all types of organizations. Jeff has taught thousands of students in CMMI trainings and has received an aggregate satisfaction score of 4.97 out of 5 from his students.

Visit www.broadswordsolutions.com for more information about running a successful CMMI program.

What was your motivation for adopting the CMMI?

Welcome back to Ask the CMMI Appraiser for today’s installment of CMMI User Stories.

If you are like the majority of CMMI practitioners, you've heard the SEI’s recent announcement about the (to be named) new business unit that will be operating the CMMI. The announcement has sparked a desire for more information about how people are currently using the CMMI in their businesses. We here at Broadsword decided to answer the call by sharing excerpts of our new study of what CMMI users really think about the CMMI.

Since our first post on this topic (“What do companies THINK about using the CMMI?”), we’ve received lots of great feedback and requests for more excerpts. So here is the sixth question we asked in our report of CMMI users:

Were you initially motivated to get a “certificate”? 

Now, before anyone gets upset by our choice of words, we do know that the SEI does not recognize the phrase “CMMI certificate.” However, a vast majority of adopters DO use that phrase, whether we like it or not, and it's become part of the lexicon in the community.  As the great Nelson Mandela said, “If you talk to a man in a language he understands, it goes to his head. If you talk to him in his language, it goes to his heart.”  So, that's the word we chose.

By the way, the correct term is "CMMI Rating" or "Appraised at CMMI Maturity Level x."  'nuff said.

Word choice aside, there is much debate in our industry about the motivations behind adopting CMMI. Some adopt the Model strictly for marketing reasons. Some are responding to mandates from their customers. And some focus on following a proven set of guidelines for becoming a great company. We wanted to understand the motivation driving those who adopted CMMI.

We found that less than half said, yes, they were primarily motivated to get a “CMMI certificate.” However, for those in this group that was focused only on "the piece of paper," the promise of marketing advantages did not necessarily translate to new business. One sponsor revealed the irony of his situation by answering, “Yes. Everyone hated it and we didn’t even win the deal.”  But that doesn't tell the whole story.

What does this tell us? It confirms what we here at "Ask the CMMI Appraiser" have been saying all along - "STOP FOCUSING ONLY ON A CERTIFICATE!" The reasons companies choose to adopt the CMMI are fairly evenly split. Not surprisingly, there was a lot of overlap. Several companies told us they were initially motivated to achieve a “CMMI level,” or to “get CMMI certified,” but that they also valued continuous improvement. Others told us that they were initially motivated to be a great company – but that they planned to take advantage of the competitive advantages associated with being “CMMI certified.”

Based on this anecdotal evidence, it seems that regardless of the motivation behind initially adopting the CMMI, those companies that desire to be great are more likely to reach their goals.

For more CMMI User Story analysis, check back soon right here on Ask the CMMI Appraiser.

We’ve also made the information available in an eBook. If you would like to receive the complete set of user stories all at once, simply subscribe at the top of this page to receive eBooks. Provided that you are using a valid company email address (not a Yahoo, AOL or Gmail account), we will be happy to add you to the distribution list, and send you the eBook, along with others, in coming weeks.

Like this blog? Forward to your nearest engineering or software exec!
Jeff Dalton is a Certified SCAMPI Lead Appraiser, Certified CMMI Instructor, author, and consultant with years of real-world experience with the CMMI in all types of organizations. Jeff has taught thousands of students in CMMI trainings and has received an aggregate satisfaction score of 4.97 out of 5 from his students.

Visit www.broadswordsolutions.com for more information about running a successful CMMI program.

We need the CMMI because we need one standard process, right?

Hey, CMMI Appraiser, I am an engineering executive in Virginia serving healthcare organizations. We use Agile methods for most of our internal projects, but we now need to become more disciplined. A Virginia CMMI consultant recently told us that our organization needs one standard process. This sounds correct, but while we evaluate our options, what is your opinion? ~ Douglas F.

Hey, Douglas,

I hear that all the time.  Consultants love talking about the CMMI as being all about everybody doing the same thing every time. According to them, the CMMI is repeatable and predictable, and that means that everybody has to have the same behavior. One size fits all.

It sounds correct. But is it?

Unfortunately, the CMMI says nothing of the kind.  What needs to be repeatable are the OUTCOMES.  Many projects are unique and require there own "way of doing things" (otherwise known as the PROCESS) - even though they will have repeatable outcomes (high quality software, efficient production, happy customers).

This is not to downplay the CMMI’s ability to help you bring more discipline to your Agile methods. It is a terrific tool for that purpose. As a matter of fact, it is THE TOOL for that.  However, it does not say you need a single standard process. The CMMI says you should have a SET of standard processes from which each project can derive their unique process.  A “set” means more than one. And “more than one” means you need some flexibility and agility to decide how you are going to do something.

So, to be successful in adopting the CMMI in your Agile environment, you’ll need to understand what the CMMI says and doesn’t say. You also need to know that, in the CMMI, flexibility and agility outweigh rigid compliance.  See my Agile Process Manifesto....

For example, let’s say your development team has a new project. Do they want to use Planning Poker to estimate? Do they want to use Wideband Delphi or Cocomo? Or do they want to use something else? The answer depends on what they are building and the constraints they are dealing with.

"Constraint-based process."  I like it.  Look for more of that in the future!

The ability to provide flexibility is the most important thing you can do when it comes to improvement and deploying process. Rigid compliance hurts you, it doesn’t help you.

The great thing about the CMMI is that it is extremely agile. It gives you the ability – in fact, it mandates – that you have choices and agility. It's right in the book ("Integrated Project Management").

I’ll say it again. A set means more than one, and more than one means there is room for flexibility and agility.

Now, there are some excellent consultants who specialize in the CMMI out there. But there are also many (too many) who can barely spell “CMMI.”

The adoption of CMMI carries with it many complex emotions. Some software and engineering executives feel strongly that they want to improve organizational performance. Sometimes they covet "CMMI certification" – and sometimes they want both.

Whatever your goals, Douglas, it can be a challenge understand how to interpret the true meaning of the CMMI at first. Turning to a qualified CMMI consultant can help alleviate the pain, remove roadblocks, and set you on the right path. But beware! A person promising to relieve you of the baggage by promising “one standard process” may not have the skills and experience you need - so tread carefully.

Anyone can claim to be a CMMI consultant. But only a small amount of people in the world have the requisite skills, training, experience, and certifications that qualify them for the task - and an even smaller group of them have a proven track record.  Think about it - it's not just "CMMI."  It's organizational change, communications, successfully giving (and getting ) advice.  It's complicated!

Good CMMI consultants have a deep holistic understanding of the entire model, how it works, and useful ideas on how to use it to improve your company. They will recommend an approach that bridges agility and discipline, in a manner that fits the way your organization is already doing things.

Watch out for CMMI consultants tell tell you one size fits all.  Next time, just say, "NO!"

Like this blog? Forward to your nearest engineering or software exec!

Jeff Dalton is a Certified SCAMPI Lead Appraiser, Certified CMMI Instructor, author, and consultant with years of real-world experience with the CMMI in all types of organizations. Jeff has taught thousands of students in CMMI trainings and has received an aggregate satisfaction score of 4.97 out of 5 from his students.

Visit www.broadswordsolutions.com for more information about running a successful CMMI program.

Adopting the CMMI - how hard is it?

Welcome back to Ask the CMMI Appraiser for today’s installment of CMMI User Stories.
If you are just joining us, we are in the middle of sharing our new study of what CMMI users really think about the CMMI. (Here’s the post that launched this discussion: “What do companies THINK about using the CMMI?”) Our decision to release this information was prompted by the SEI’s recent announcement about the CMMI, which raised a call for more detailed information about how people are experiencing the tool.

On with our analysis. Here’s the fifth question we asked CMMI users:

Was your experience with the CMMI harder than you expected? 

Why ask this question? Despite the fact that a plurality of respondents said that they had received an ROI after adopting the CMMI (see the CMMI Appraiser's most recent post), many also reported that adoption of the CMMI can be quite complex and difficult. We wanted to understand if that difficulty exceeded the expectations of the organizations that adopt the CMMI.

It was interesting to find that 39% of respondents said the experience was harder than expected, while almost two-thirds said they were prepared for the challenge, and did not find adopting the CMMI harder than expected.

What conclusions can we draw? It appears that the SEI and its Partners have done a good job setting expectations, for the most part. The greater percentage of respondents felt that the adoption was difficult, but no more difficult than anticipated.

By the end of the year, the CMMI constellations (as well as other "mature technologies" that are no longer considered "research" projects) will be moved to the new business unit within Carnegie Mellon University (yet to be named). One of the challenges facing this new business unit will be to continue setting expectations among CMMI Users that it is not easy to adopt the Model, but well worth the effort.

For more CMMI User Story analysis, check back soon right here on Ask the CMMI Appraiser.

We’ve also made the information available in an eBook. If you would like to receive the complete set of user stories all at once, simply subscribe at the top of this page to receive eBooks. Provided that you are using a valid company email address (not a Yahoo, AOL or Gmail account), we will be happy to add you to the distribution list, and send you the eBook, along with others, in coming weeks.

Like this blog? Forward to your nearest engineering or software exec!
Jeff Dalton is a Certified SCAMPI Lead Appraiser, Certified CMMI Instructor, author, and consultant with years of real-world experience with the CMMI in all types of organizations. Jeff has taught thousands of students in CMMI trainings and has received an aggregate satisfaction score of 4.97 out of 5 from his students.

Visit www.broadswordsolutions.com for more information about running a successful CMMI program.

Do CMMI Users actually get an ROI?

Welcome back to Ask the CMMI Appraiser for today’s installment of CMMI User Stories.

In an industry full of inquisitive problem-solvers, it was no surprise that the SEI’s announcement about the CMMI triggered so many requests for more information about how people are using the CMMI. After all, folks like you and me need data to help us evaluate where we are in our ongoing quest to become -- and remain -- great companies.

As you know if you are a regular visitor at Ask the CMMI Appraiser, over the past several days, we’ve been sharing excerpts from our brand new study of what CMMI users from 50 companies really think about the CMMI.

Today, we present the fourth question we asked CMMI users:

Did you experience an ROI?

As described in our last Ask the CMMI Appraiser post, adopting the CMMI requires time and effort. And while 90% of CMMI Users said it was worth it, the CMMI can be quite costly. How do CMMI Users feel about their return on Investment?

We learned from many CMMI Users that it can be difficult to ascertain an aggregate ROI, due to the wide degree of variability in products and services delivered by the adopters of CMMI. Even so, we wanted to understand the market’s perception of CMMI, and whether sponsors thought they had received an ROI (internally, we referred to this metric as the “cost factor”).

Over 40% said yes, they had seen an ROI; 40% said they had no idea; 15% said no and 3% said "no comment."

Among those who tracked ROI, most were positive. One respondent told us: “it was one of our most profitable divisions.”

What can we glean from this information? It’s interesting that the same percentage responded that they had received an ROI as those who did not know. This appears to reinforce what CMMI Appraisers and CMMI sponsors alike have suspected for years: that a direct correlation between cost and value of CMMI is difficult to establish. It’s much easier to draw a connection between companies that intend to use the CMMI to radically improve performance, and those that do not.

It’s also interesting to reflect on those who said “no comment.” Were they really in the “I don’t know” category, but unwilling to admit it?  Or were they concerned that they may appear to have sipped the kool-aid?

For more CMMI User Story analysis, check back soon right here on Ask the CMMI Appraiser.

We’ve also made the information available in an eBook. If you would like to receive the complete set of user stories all at once, simply subscribe at the top of this page to receive eBooks. Provided that you are using a valid company email address (not a Yahoo, AOL or Gmail account), we will be happy to add you to the distribution list, and send you the eBook, along with others, in coming weeks.

Like this blog? Forward to your nearest engineering or software exec!

Jeff Dalton is a Certified SCAMPI Lead Appraiser, Certified CMMI Instructor, author, and consultant with years of real-world experience with the CMMI in all types of organizations. Jeff has taught thousands of students in CMMI trainings and has received an aggregate satisfaction score of 4.97 out of 5 from his students.

Visit www.broadswordsolutions.com for more information about running a successful CMMI program.

Continuously improve? What the $%^&! does THAT mean?

Hey, CMMI Appraiser – we are an electronics parts manufacturer in Virginia that recently merged with an organization that follows the CMMI. Even though they acquired us because of our Agile approach, my new manager doesn’t understand Scrum at all. He keeps asking me to push my team to “continuously improve” and “get better” at what they do, and “do more with less.” What the heck does that mean? We’re just trying to build great products, same as always. ~ Dan L.

Hey, Dan, so, your best isn’t good enough, you don’t know why,  and you don’t know what your manager's expectations are for what improvement looks like.  Hasn't he trained you to read his mind?  That would fall under GP2.5 by the way . . . .

I remember earlier in my career when I was the CIO of a company and I heard the same thing.  "Get better!  Make sure your team is continually improving!"  Like you, I said, What does that mean? Come in earlier? Work harder? Be more serious? (They actually said that to me one time. I responded by being LESS serious).

So you are not alone with this problem, my friend. The cool thing is, since then, I’ve done a lot of work with companies that have embraced the CMMI and Scrum in an effort to get better, and I've learned to help demystify things a bit.

In my opinion, the clearest way to think about Scrum and CMMI is in terms of process improvement, not running teams or projects. That keeps us focused on the only project we really care about, the one called, “Making our company Great.” Scrum can help companies adopt things like CMMI, ISO and continuous improvement programs in general.

Now, I don’t claim to be a mind-reader, but that could be exactly what your manager wants: Taking Agile and making it better by applying the lessons of the CMMI.  And conversely, taking the CMMI and making it better with Agile.  If you're getting a recursion headache now, don't worry.  It'll subside.

If so, he's onto something, although maybe not articulating it very well.  This is an idea I’m really passionate about. I believe all organizations can benefit from rethinking the CMMI and Scrum and how they work. It boils down to three simple concepts:

First, the CMMI is a behavioral model, not a process. It was intended to help make things better, a guide to continuously improve. It describes how great companies perform.

Second, as such, the CMMI doesn’t tell you HOW to get better. In fact, the CMMI doesn’t tell you how to do anything. It says, “Here’s what great companies have told us that they do.”  Your job is to apply these lessons to what you are doing in your context.  Tricky?  Yes, but well worth it.

Finally, the truth about CMMI and Scrum is that they are both the "same thing."  They are tools to help solve business problems. They help us improve  requirements churn and volatility, they help us meet schedule and budget, and they help us perform the work that we do every day.  They're not overhead - they're "underhead."

So relax, Dan.  You’ve got everything you need.  Leverage Scrum to get the best of the CMMI. And leverage the CMMI to get the best of Scrum.

If that’s what your manager is thinking, then I see a happy, productive company in your future.

Like this blog? Forward to your nearest engineering or software exec!

Jeff Dalton is a Certified SCAMPI Lead Appraiser, Certified CMMI Instructor, author, and consultant with years of real-world experience with the CMMI in all types of organizations. Jeff has taught thousands of students in CMMI trainings and has received an aggregate satisfaction score of 4.97 out of 5 from his students.

Visit www.broadswordsolutions.com for more information about running a successful CMMI program.

Adopting the CMMI ... was it worth it?

Welcome back to Ask the CMMI Appraiser for today’s installment of CMMI User Stories.
As you know by now, in response to the SEI’s announcement about the CMMI, we decided to provide more information about the experiences CMMI users are having.  Below is an excerpt from our brand new study of what CMMI users from 50 companies really think about the CMMI.

The study consisted of calling CMMI users and asking them about their experiences with the CMMI, good and bad. Here’s our third question:

Was adopting the CMMI worth doing?

Why ask this question? Because, in the early years of the CMMI, the US Department of Defense (DoD) and other early adopters of CMMI, like Lockheed Martin, Rockwell Collins, Boeing and Air Force, were known for conducting top-down, document-heavy, “command and control" software projects. This reputation carried over as these same organizations embraced CMMI. As a result, many smaller, nimbler businesses thought that adopting the CMMI was not a "good fit" for them and was not worth it.

Since then, perception of the CMMI model has changed. The CMMI has evolved beyond the "oppressive interpretation."  A more flexible, agile approach to the model is becoming the new standard.

Despite this change, the CMMI is still time-consuming and resource-intensive to adopt (but well worth it).  Change like this can be disruptive to the organization.  And so we wanted to understand if sponsors thought the overall effort was worth it (internally, we referred to this metric as the “hassle factor.”)

90% said the CMMI effort was worth doing.

10% said the CMMI effort was not worth doing.

What can we take away from this response? Most companies did not experience the CMMI as being oppressive. It may have required a concerted effort to adopt the CMMI, but the vast majority of CMMI users said that adopting it was worthwhile.

Be sure to check back regularly as we share results of the CMMI User Story study right here on Ask the CMMI Appraiser.

We’ve also made the information available in an eBook. If you would like to receive the complete set of user stories all at once, simply subscribe at the top of this page to receive eBooks. Provided that you are using a valid company email address (not a Yahoo, AOL or Gmail account), we will be happy to add you to the distribution list, and send you the eBook, along with others, in coming weeks.

Like this blog? Forward to your nearest engineering or software exec!
Jeff Dalton is a Certified SCAMPI Lead Appraiser, Certified CMMI Instructor, author, and consultant with years of real-world experience with the CMMI in all types of organizations. Jeff has taught thousands of students in CMMI trainings and has received an aggregate satisfaction score of 4.97 out of 5 from his students.

Visit www.broadswordsolutions.com for more information about running a successful CMMI program.

After adopting the CMMI, did you experience a decrease in defects?

Welcome back to Ask the CMMI Appraiser for today’s installment of CMMI User Stories.

As you are probably aware by now (especially if you read my post, “What do companies THINK about using the CMMI?”), the SEI’s announcement about the CMMI created tremendous demand for more information about the experiences CMMI users are having. In response, we are sharing our brand new study of what CMMI users from 50 companies really think about the CMMI.

Here’s the second question we asked CMMI users about their experiences with the CMMI:

Did you experience a decrease in defects?

Why this question? Because sometimes problems occur again and again and that leads software engineering organizations to CMMI (or other process model) adoption. For example, when we first start working with companies, we see that they struggle with late projects, over budget projects, unhappy customers and unhappy employees. These problems are usually coupled with excessive defects in the software they are developing.

The CMMI provides guidelines for deploying processes that improve product quality and reduce defects. We wanted to understand whether organizations actually experienced these benefits through their adoption of the CMMI.

Fully half of respondents said that defects had decreased, sometimes dramatically. Fewer than half said, no, defects had not decreased. 12% had no idea (no idea?  Don't get me started on THAT one!),

What can we take away from this response?  In our conversations with those who said, “No, we did not experience a decrease in defects,” we were surprised - but perhaps they were ALREADY doing the very things that the CMMI guides us to do (a common occurrence).  It makes sense though - great companies often do great things without adopting a process model - that's why they're great companies, and that's kind of the point!

Be sure to check back soon here on Ask the CMMI Appraiser for more insight from our CMMI User Stories.

We’ve also made the information available in an eBook. If you would like to receive the complete set of user stories all at once, including the narratives from each interview, simply subscribe at the top of this page to receive eBooks. Provided that you are using a valid company email address (not a Yahoo, AOL or Gmail account), we will be happy to add you to the distribution list, and send you the eBook, along with others, in coming weeks.

Like this blog? Forward to your nearest engineering or software exec!

Jeff Dalton is a Certified SCAMPI Lead Appraiser, Certified CMMI Instructor, author, and consultant with years of real-world experience with the CMMI in all types of organizations. Jeff has taught thousands of students in CMMI trainings and has received an aggregate satisfaction score of 4.97 out of 5 from his students.

Visit www.broadswordsolutions.com for more information about running a successful CMMI program.

 

After adopting the CMMI, did you experience an increase in quality or performance?

Welcome back to Ask the CMMI Appraiser for today’s installment of CMMI User Stories.

As described in my last post (What Do Companies THINK about the CMMI?), in response to the demand for information created by the SEI’s announcement about the CMMI (in a nutshell, the CMMI is being graduated from "research" status to “mature” status), we are sharing our study of what CMMI users really think about the CMMI.

The study consisted of calling CMMI users and asking them about their experiences with the CMMI, good and bad. Here’s our first question:

Did you experience an increase in quality or performance?

Why do we start with this question? Because different companies have different expectations of the CMMI. Some seek CMMI “levels” for the marketing advantages, or to satisfy customer requirements. Others strive for continuous process and performance improvement; they see the CMMI as the tool they use for guidance.

In both cases, the strength of the CMMI is that it provides organizations with guidelines for increasing performance and improving the quality of products and services. We wanted to know whether organizations actually achieved improvements in performance and quality.

81% of respondents said they improved in these areas.

 

What can we take away from this response? Of those who responded “I don’t know,” many told us that they weren’t tracking quality or performance increases. Many of those who said, “No,” indicated that they were already operating at a high level and the CMMI helped them standardize their excellence in performance and quality.

Most of CMMI Users experienced an increase in quality and performance if they were looking for it.

Be sure to check back regularly as we share results of the CMMI User Story study right here on Ask the CMMI Appraiser.

We’ve also made the information available in an eBook. If you would like to receive the complete set of user stories all at once, simply subscribe at the top of this page to receive eBooks. Provided that you are using a valid company email address (not a Yahoo, AOL or Gmail account), we will be happy to add you to the distribution list, and send you the eBook, along with others, in coming weeks.

Like this blog? Forward to your nearest engineering or software exec!

Jeff Dalton is a Certified SCAMPI Lead Appraiser, Certified CMMI Instructor, author, and consultant with years of real-world experience with the CMMI in all types of organizations. Jeff has taught thousands of students in CMMI trainings and has received an aggregate satisfaction score of 4.97 out of 5 from his students.

Visit www.broadswordsolutions.com for more information about running a successful CMMI program.

What do companies THINK about the CMMI?

Hey, CMMI Appraiser, now that the CMMI is moving to a new business unit within CMU, wouldn’t it be great if we knew what people really thought about using the CMMI? ~ a fellow CMMI Appraiser 

Hey, fellow CMMI Appraiser,

Yes, that’s the million dollar question, isn’t it? A lot of people are keenly interested in the perception of the CMMI in the market today. Especially now that the SEI has announced that the CMMI constellations (as well as other "mature technologies") will no longer be considered "research" projects, and will be moved to a new business unit within Carnegie Mellon University (CMU).

As I’ve said, I interpret this as a good … no, a great thing. The Department of Defense, long a sponsor of the SEI, was very involved (along with CMU) in the decision. The new business unit will be unencumbered by the constraints of the DoD, and will take the CMMI in new, more flexible directions, and to new markets.

And we can’t say no one saw this coming. For the past few years, at SEPG NA conferences, several of my fellow Lead Appraisers and I have been talking about the need for change in the market. We saw CMMI adoption increasing for large and small companies alike. We thought our clients were reaping tremendous rewards. But we had no idea what other CMMI users, those we didn't know, actually thought about using the tool.

In the months leading up to the SEI’s CMMI announcement, I decided to commission a User Story study. My company, Broadsword, selected 50 companies randomly from the SEI’s Published Appraisal Results (PARs). The companies had achieved CMMI Maturity Level 2 or 3 between 2007 and 2010.

The reason we chose this timeframe was to give the CMMI users some period of time with their adoption, allowing for more objectivity.

Next, we retained two call center representatives, who picked up the phone and started calling.

Over several days, we spoke with CEOs, VPs and Quality Assurance Managers of North American companies, large and small. Companies were in the aerospace, defense, finance, transportation, energy and manufacturing industries, and they were all using CMMI-DEV.

We listened to the stories they told about using the CMMI. We captured the data, and analyzed the results.

As far as we know, this is the first study of its kind that builds its case on anecdotal data from actual end users. We gathered stories about what’s working and what’s not working with the CMMI, directly from the users themselves.

I hope the timeliness of this information will prove valuable to you, whatever your role with the CMMI. It was finalized literally days before the SEI announcement.

And so I invite you to check back regularly as we share results of the CMMI User Story study right here on Ask the CMMI Appraiser.

We’ve also made the information available in an eBook. If you would like to receive the complete set of user stories all at once, simply subscribe to reveive eBooks at the top of this page.  Provided that you are using a valid company email address (not a yahoo, aol or gmail account), we will be happy to add you to the distibution list, and send you the eBook, along with others, in coming weeks.

Like this blog? Forward to your nearest engineering or software exec!

Jeff Dalton is a Certified SCAMPI Lead Appraiser, Certified CMMI Instructor, author, and consultant with years of real-world experience with the CMMI in all types of organizations. Jeff has taught thousands of students in CMMI trainings and has received an aggregate satisfaction score of 4.97 out of 5 from his students.

Visit www.broadswordsolutions.com for more information about running a successful CMMI program.