Our people think an Appraisal is an audit and won't say anything!

It seems like in an "audit", the best strategy is to say as little as possible, where in a CMMI appraisal, I want them to talk a lot! We really want to learn from the appraisal and get an honest assessment more than anything.

Any ideas on that, and have you been involved in appraisals of organizations who had this stuffy hang-up? 

A CMMI Appraisal is NOT an audit!  So many people believe this that I've started to speak on this subject at conferences.

If you look back to the original intent of the authors at the SEI, the CMMI was simply intended as a set of guidelines that describe how a "great company" behaves.  That's all!  It was never intended as an audit methodology (as if we need another one of those).

The problem dates back to the days when the DOD insisted that every supplier achieve a "level" of CMMI.  Some enterprising "consultants" then devised an evil scheme to reverse-engineer the CMMI model and convinced customers to create all of the documents in the "typical work products" section of the book, and then supported their own dastardly plan by "auditing" the presence of the work products (hey, PPQA is not too happy about this arrangement!).  

They publicly flogged those who "failed" and smirked and snickered their way through presentations and articles about how the company "failed" their special test (if they would only buy their "perfect project in bag" system . . . ). 

The SEI didn't do themselves any favors by churning out inexperienced Lead Appraisers at an alarming rate and teaching them to focus on evidence over common sense and behavior.

Thankfully that is starting to change.  The SEI has taken steps and so has the market. 

I tend to think that the CMMI is about behavior, not about evidence.  It's about creating a system to make your company better . . . and better . . . and better.  People always ask me at my CMMI Training classes: "Hey Jeff, when will this be over?" and I say "NEVER! Isn't that great!  We get to re-define our work environment until we make it right.  How cool is that?"

Some people say there are already great, and that they don't need no stinkin' model to make them better.  Just ask them "how do you know? We could be even greater!"

I gauge the success of my appraisals by how much I get people to talk to me.  They all file into the room, tight-lipped and nervous, and expect me to start in on "the script" from the other side of the table.  I usually sit next to them, put my pad away, and just pull on the rip cord.  "Tell me a story" I say.  Once they start talking, there's no stopping them!


If you like what you've read and you're looking for a Lead Appraiser, give us a call!

Jeff Dalton is a Certified SCAMPI Lead Appraiser, Certified CMMI Instructor, Candidate SCAMPI Appraiser Observer, author, and consultant with years of real-world experience with the CMMI in all types of organizations.  Jeff has taught thousands of students and has received an aggregate satisfaction score of 4.97 out of 5 from his students.

Learn more about CMMI maturity levels at www.broadswordsolutions.com

How do we compare to other companies that adopt the CMMI?

Appraiser, Is there an online resource where we can compare our CMMI maturity level to other companies?  ~E.I.

E.I., Here’s something you might find interesting.  Anyone curious about comparing their CMMI maturity levels to other companies can reference the following chart:

At a glance, the chart reveals three interesting pieces of information:

1. The vast majority of companies are at CMMI Maturity Level 2 and Level 3 (“Managed” and “Defined”).
2. A smaller amount of companies are at CMMI Maturity Level 4 and 5 (“Quantitatively Managed” and “Optimizing”). 
3. A fairly significant amount of companies did not get a CMMI Maturity Level rating (“Not Given”).  "Not Given" could be the result of a company not caring about what "level" they get, or it could be because they DO care, but just don't make it!

When I displayed these results at a recent CMMI training, several people were curious about that category, “Not Given.”  They wanted to know the reasons for a company not to achieve a CMMI Maturity Level.  I explained that one reason is that their appraisal was not successful – but that’s not the only reason.  Sometimes, companies choose to not be rated.  That’s OK.  But it does affect the size of the “Not Given” category.  You should not assume, however, that this indicates a high failure rate.

To me, the most interesting aspect of this graph is that so many companies were at CMMI Maturity Levels 2 and 3.  In fact, out of the last 1287 appraisals reported in North America, just under half were CMMI Maturity Level 2, and just under half were CMMI Maturity Level 3. 

So how does your organization compare?  Leave a comment below and let us know!

Jeff Dalton is a Certified SCAMPI Lead Appraiser, Certified CMMI Instructor, Candidate SCAMPI Appraiser Observer, author, and consultant with years of real-world experience with the CMMI in all types of organizations.  Jeff has taught thousands of students and has received an aggregate satisfaction score of 4.97 out of 5 from his students.

Learn more about CMMI maturity levels at www.broadswordsolutions.com

We'd like to adopt CMMI but my boss says "no way, it's for huge companies!" Is he right?

Dear Jeff – I’ve heard from my boss that CMMI is only for huge companies.  We’re no Lockheed Martin or Boeing, but we’re growing and want to do things the right way.  Do we have a shot at adopting CMMI?  ~Mitchell P.

Mitchell,

You’re not alone in assuming that adopting CMMI is only for big companies like Lockheed-Martin and Boeing.  In fact, this was true, back in the old days, when companies with 2,000-3,000 or more employees were typically the ones adopting CMMI.  But if you look at recent data released by the SEI, contained within their March 2011 Maturity Profile Report (available here) and the analysis of over 5,000 organizations, you see that by far the largest category of companies adopting CMMI are under 100 employees.

And the cost is well worth it.  For the price of about one FTE you get dramatically improved performance, increased throughput, and higher quality.  Heck,  I'd trade that for five quality engineers!

What this tells you, Mitchell, is that you’re on the right track.  Smaller companies like yours are starting to dominate the market, in terms of adopting CMMI and in the new business you're starting to take on.  Sure, we still see a healthy representation by the large companies.  But the fact remains that the majority of organizations being appraised today are under 100 people.

Going forward, I would say that this looks like the trajectory we are on.  There are several reasons.  For one, a lot of the bigger companies are already using the CMMI, so it only makes sense that growth in the sector would come from smaller companies.  Another factor is that, as engineers leave those bigger companies to start their own small businesses, they bring along the methods and models that helped them be successful in the past.   Competing small businesses start getting their rear-ends kicked in sales, and they become interested in adopting CMMI as well.

It’s all part of a widespread movement by companies large and small in North America to put themselves on the path to greatness and compete in the global market.

Jeff Dalton is a Certified SCAMPI Lead Appraiser, Certified CMMI Instructor, Candidate SCAMPI Appraiser Observer, author, and consultant with years of real-world experience with the CMMI in all types of organizations.  Jeff has taught thousands of students and has received an aggregate satisfaction score of 4.97 out of 5 from his students.

Learn more about adopting CMMI at www.broadswordsolutions.com

We want to start with the CMMI and we need a training program - Guest Blogger!

While I'm on vacation this month we're lucky enough to have a guest blogger! Shawn Rapjack is a CMMI and Process Improvement professional with Science Applications International Corporation (SAIC).  Shawn was on several of the appraisal teams I have led at SAIC over the years and he "knows the ropes" better than most.

Take it awaaaaaaaaay!

-------------------------------------

I’m establishing a training program in my organization.  We want to pursue CMMI!  What do I need to consider?

·       Strategic training needs.  Strategic needs are long term goals (2-4 years off) that your organization identifies.  The strategy aligns with your organization’s business goals. For example, you will address whether engineers need to be trained for future software releases in anticipation of contract wins.

·       Planning!  When employees, and SCAMPI Lead Appraisers, rate organizations - training is often a major negative.  It is the first item to fall to the wayside when budgets are considered and results in downbeat employee morale.   One way to mitigate this is to incorporate training line items (classes) in your budget estimates and document training resources (capability, teaching staff, workspace) in your project plans.  You will need to derive cost and schedule estimates and track these.

·       Types of Training.  Process training educates your employees about the awesomeness of CMMI [and it is awesome! Ed].  Employees need to be indoctrinated and educated about any tailored processes that have been adopted (How does that new workflow process go again?).  Technical training includes technical, engineering-oriented training (in software packages, perhaps).  Organizational training includes corporate-mandated training (security and ethics).

Remember, there are CMMI practices to ensure that staff is adequately trained for their roles – Do process managers have requisite CMMI training?  Have testers been educated in the right software packages?

·       Who is responsible? Decide who is responsible for your training and at what level.  Is your corporation responsible for CMMI process training and then individual departments responsible for technical training?

What’s in a Training Plan?

We spoke about your strategic training needs; the Training Plan discusses the tactical details of how that strategic vision will be delivered.  Organizational goals will be highlighted, budgets will be provided or referenced, and responsibilities will be documented. The means of training will be discussed (in-house, classroom, teleconference, mentoring). Methods to analyze training effectiveness will be considered.

What are in Training Matrices?

Training matrices supplement your Training Plan, providing tactical specifics of who needs and who has taken training. Specifically, they address:

·       Training courses by role.  What training do Requirements Managers need? 

·       Which employees fill what role? (Corroborated with organizational charts).

·       Whether training is required, optional, suggested or not required.

·       Who has taken requisite training and when?  This can be recorded through corporate database tools.

Remember, if you're planning a CMMI Appraisal, you'll want to address this right away!

Shawn Rapjack is an SEPG lead and process professional with SAIC.  He is guest blogging while Jeff is on vacation. 

www.broadswordsolutions.com

How to Prepare for an Appraisal - Guest Blogger!

Dear Readers,

While I'm on vacation this month we're lucky enough to have a guest blogger! Shawn Rapjack is a CMMI and Process Improvement professional with Science Applications International Corporation (SAIC).  Shawn was on several of the appraisal teams I have led at SAIC over the years and he "knows the ropes" better than most.

Take it awaaaaaaaaay!

-------------------------------------

It’s almost time for the appraisal!  Auuuuughghg!  What do I do?

Preparation

Before your initial appraisal, you need to get these activities under your belt:

 - Organization of artifacts.  


You have collected your artifacts – now, they need to be neatly organized for appraisal review.  A variety of mechanisms can be used – spreadsheets, neatly organized document repositories, or websites. CMMI practices should be associated with your project’s implementation notes and applicable artifacts. You don’t want the appraisers hunting around trying to find artifacts – this is very frustrating.  Having artifacts neatly presented is indicative of your project!  How can you manage a multi-million dollar project if you can’t even organize your artifacts?

- Selection of Lead Appraiser and team members


There is a lot to consider when you select team members, but don't underestimate how much it takes to coordinate who will comprise the appraisal team and their availability.  We've worked with Broadsword (Jeff Dalton) on thirteen of our appraisals, as well as some others over they years.

-  If the appraisal team is not from your local office travel arrangements will need to be made.

- Meeting space and food will need to be provided. There's nothing worse than a hungry, grumpy appraisal team!  It is ideal if there is a large conference room with several break-away rooms.  This will allow for appraisal team group discussions and mini-team work.  Snacks are always welcome….

- An appraisal roadmap will be created.   This plan details the schedules, goal and scope of the appraisal. It will be created in coordination with the Lead Appraiser.  The Lead Appraiser will also create a data collection plan, and Appraisal plan, and numerous other plans to support your appraisal.

Appraisal Time!

Its Appraisal week!  You should consider the following activities and lessons learned:

- Create a presentation detailing the major highlights of each project.  The appraisers know nothing about your project – this will provide a good foundation and reference for them. 

- Attend required meetings. Initially, the Lead Appraiser will discuss the appraisal’s goals and schedule.  After the appraisal’s conclusion and depending on its type, findings will be presented.  You will want everyone to attend these!

- Be available for all interviews you are scheduled for.   Interviews are formal and may be with groups or individuals.  Appraisers will usually pose open-ended questions asking you to explain your understanding and implementation of your processes.  Appraisers are furiously taking notes, hoping to have several questions addressed by carefully-worded questions.  They may ask for elaborations.

- Attitude.  Be respectful toward appraisers effort.  Appraisal team members are volunteering (sometimes on top of "real work") to help you out.

- Provide resources the appraisal team asks for.  Appraisers may ask for further clarification of your processes or for additional artifacts.  

- You don't need to know all of the answers!  If you know someone that does, please let the appraisal team know who they are.

- Don't think of the appraisal as CMMI Certification.  There's a reason they don't call it "CMMI Certification."  It's more an opportunity to collaborate with professionals like yourselves to make your work environment better.  So have fun!

Good luck with your appraisal!

Shawn Rapjack is an SEPG lead and process professional with SAIC.  He is guest blogging while Jeff is on vacation.  

www.broadswordsolutions.com