Thoughts on the Certified Scrum Master Class from a CMMI Guy.

Dear Readers,

I just put myself and my whole CMMI consulting team through the "Certified Scrum Master" class. We've been using Scrum for some time in our work with CMMI, but I wanted my team (me included) to take the formal class, taught with great skill by Chris Simms and Jeff McKenna of the Agile Learning Labs.

I've read all the press about this certification not being worth the paper it's written on, that it's a crime to call someone a "Master" after taking one course, and so on. My experience with the class tells me it was well worth the effort. I learned a lot in this class - and most of it wasn't about Scrum at all - it was about getting work done in an effecive manner through team collaboration, and THAT's the good stuff!

We took the course not for the "certification" but for the excellent content and the team dynamics discussions that Chris is so good at. I thought the class was excellent - a good basic grounding in Scrum, and some great hands-on exercises with some philosophy rolled in for good measure.

Chris is a deep thinker who knows how to get a message across better than most.

I'm not sure what all the hoopla about the course name is about. Isn't "Scrum Master" an actual title in Scrum? And doesn't this class teach you the basics so you can begin to act like one? And isn't one of the purposes of Agile to be collaborative, involve everyone, and rise all the boats around you? So, why all this infighting in the Agile community about it?

Is the NAME really the source of all this angst? Talk about WORRYING ABOUT THE WRONG THINGS!

I wouldn't have chosen the name "Scrum Master" myself, but I didn't make the hard effort to create Scrum, so what I think isn't important. I also would not have chosen XP or any of the other "cool" names related to Agile development - that's just me (I'm an old fart that still thinks Computer Science and business are important). But that's not really the point is it? Those ARE the names. If you take a class that teachers you how to be a Scrum Master, than that's what the class should be called. If it was "Scrum Facilitator" (a little more accurate IMHO) would everyone be happier?

The "Certified" part kind of makes me wonder - but again, so what? I know TONS of people that badmouth the course but have never taken it, can't describe Scrum with any accuracy, yet they feel free to spout off their disapproval.  These are the same people that talk about "CMMI Certification."  They're obsessed with the certification part . . . talk about useless overhead!

So I'm not too focused on the exam, but I am geeked to apply some of the things Chris taught us - as a matter of fact I borrowed some of his wisdom the very next day while teaching my own CMMI class!


Oh, and in case you're wondering, I'm convinced now more than ever that Scrum+CMMI = FTW!

Jeff Dalton is a Certified SCAMPI Lead Appraiser, Certified CMMI Instructor, Candidate SCAMPI Appraiser Observer, author, and consultant with years of real-world experience with the CMMI in all types of organizations.  Jeff has taught thousands of students and has received an aggregate satisfaction score of 4.97 out of 5 from his students.

Learn more about CMMI Training at www.broadswordsolutions.com

www.broadswordsolutions.com

Who performs data collection and characterization on a SCAMPI Appraisal?

Dear Appraiser, 

A while back, we attended one of your CMMI trainings that dealt with determining our data collection strategy for a SCAMPI appraisal.  You basically said that it was up to us to collect and begin characterizing our own data.  However, we find this very time consuming and difficult.  Can you do a CMMI training about how we get someone to do the data collection for us?  ~ Tyree N.

Hey, Tyree,

I could, but I won’t. :).  I would never use my CMMI training sessions to teach you how to cheat yourself out of becoming a great, continually improving company.

I’d rather see you master the data collection process and make it your own.

This starts with understanding that a lot has changed since your last CMMI training.  For one thing, the new SCAMPI v1.3 release changes the options related to when the data will be collected.  The time of data collection is determined by what type of appraisal you are conducting, managed discovery, verification or discovery.   

Here is the break-down:

1. Discovery

In the Discovery appraisal, the data is only collected when the Lead Appraiser arrives.  Discovery appraisals are the preferred method of Lead Appraisers who, in my opinion, need more CMMI training on the true purpose of data collection.  Discovery appraisals often seem to end in disaster. 

2. Verification

In the Verification appraisal, you will collect the data, without the help of a Lead Appraiser, before the Lead Appraiser gets there.  Better,  but disaster can lurk for many reasons.

3. Managed discovery

In the Managed discovery appraisal, you will collect the data, with guidance provided by the Lead Appraiser before I get there, and I will help you by facilitating discussions where you can learn new things of lasting value about yourselves as a company, and where you want to be.

Don’t be alarmed if this all sounds new.  I admit, this approach is not typical ... yet.  I've always conducted appraisals using the "managed discovery" approach - even though it was only recently introduced in SCAMPI v1.3.  

Companies are always surprised when I say to them, “OK, for the next month, you guys are going to collect and characterize some of the data.”

 

They say, “We’re going to score our own data?” 

And I say, “Yes, did you think I was going to do all of it for you?”

To be more exact, we're going to do a lot of it together.

The point is, Tyree, you’re supposed participate in it because it's about YOU!  This is about you and your company, and making your company better.  You are the one who needs to learn about it collect and analyze data.  I already know how.  Having me characterize your data alone doesn’t help you at all.

Level-seekers think the only reason to collect data is to get the artifacts to complete the appraisals.

Try to remember why you are doing this, guys.  There’s only one reason.  To make yourself better.

Jeff Dalton is a Certified SCAMPI Lead Appraiser, Certified CMMI Instructor, Candidate SCAMPI Appraiser Observer, author, and consultant with years of real-world experience with the CMMI in all types of organizations.  Jeff has taught thousands of students and has received an aggregate satisfaction score of 4.97 out of 5 from his students.

Learn more about CMMI Training at www.broadswordsolutions.com

What the heck happened to Focus and Non-Focus projects in SCAMPI v1.3?

Jeff, I’m a CMMI Appraiser in New Jersey, a relatively new CMMI Appraiser, actually, so I appreciate the information you have been sharing in this blog about the meaning behind the changes to the SCAMPI model.  I would like to add a question.  When they simplified the terminology by removing the terms ‘focus project’ and ‘non-focus project’ in V1.3, what impact does that have on CMMI Appraisers like me?  ~Plato B.

Wow!  Two questions in a row from fellow Lead Appraisers.  Sure, I'd be happy to answer this one.  The impact of this particular terminology change is significant, and affects all CMMI Appraisers, from new to experienced.  Before explaining how, let’s take a moment to define the terms "focus project" and "non-focus project."

As you know, for each maturity level of the CMMI, there are process areas associated with that level.  To assess these process areas, projects were selected to be focused upon (‘focus projects’).   ‘Non-focus projects,’ or extra projects, were also selected as a way to get extra information to help gather more data and to verify the institutionalization of the process.

For example, let’s say a company had 100 projects.  A subset of 25 projects might have been selected to be appraised: twenty focus projects and five non-focus projects.  Because focus projects had to satisfy 100% of the applicable goals in the CMMI model for that level, and because they had to show evidence for all of the associated practices, we would do a deep dive on all 20 focus projects.

Non-focus projects didn’t require a deep dive that included all of the process areas.  With the five non-focus projects, we would pick and choose what process areas we wanted to look at.  In one, we might have looked at how good they were at planning and estimating.  In another, we might have looked at how strong they were at Verification.  Basically we were selecting process areas throughout the organization, and were looking for examples to verify that, yes, in all of these other projects, the projects were consistently meeting the intent of CMMI.

So what changed when the SEI removed the terms from this version of the SCAMPI method?

The simple answer is the sampling methodology changed.  The sampling strategy is beyond the scope of this post (I have blogged about this subject recently), but the big difference regarding focus and non-focus projects is that they no longer exist, and instead we are now required to create a data collection plan based on an algorithm provided by the SEI.  The plan will take an “inside out” view, requiring us to identify the source of evidence for various process areas by identifying which project we are going to look at for that particular piece of data. It’s more complex than that, but you get the picture.  

The new sampling methodology creates an additional planning effort, making preparation for the appraisal more difficult.   It will also be more difficult for companies to cherry-pick the projects that are going to be appraised, broadening the selection of available projects and increasing the credibility of the appraisal and of the CMMI in general.  All good stuff!

We understand why the SEI did this – in fact, we strongly support it.  The new sampling methodology makes appraisals more consistent and helps deter companies who are interested in gaming the system.  But let’s not kid ourselves about the impact of the change.  By simplifying the terminology, the SEI has radically modified the sampling strategy.  The preparation for the appraisal will require more effort and experience, which will increase the amount of time and effort it takes to conduct an appraisal.

Welcome to the ever-changing life of a CMMI Appraiser, Plato.  No one said the path would be easy!

Jeff Dalton is a Certified SCAMPI Lead Appraiser, Certified CMMI Instructor, Candidate SCAMPI Appraiser Observer, author, and consultant with years of real-world experience with the CMMI in all types of organizations.  Jeff has taught thousands of students and has received an aggregate satisfaction score of 4.97 out of 5 from his students.

Learn more about CMMI Appraisers at www.broadswordsolutions.com

What's with the new "Data Collection Strategy" in SCAMPI v1.3

Dear Appraiser, 

I appreciate your take on the rule changes in the new SCAMPI release [v1.3].  I admit, as a Lead Appraiser, a lot of this has me baffled.  For example, the new requirement that Lead Appraisers have to define a data collection strategy.  I don’t know about anybody else, but I've always thought about how to collect the data.  Are we supposed to do something different? ~ Charles T.

Hey, Charles,

I don't get too many questions from fellow Lead Appraisers (although some should ask more . . . ).

You’re right.  For the best Lead Appraisers, this "new activity" outlined in SCAMPI v. 1.3 isn’t new at all.  If you care about helping companies get better, not just on getting a CMMI level, you'll have a data collection strategy that serves your client in the most useful way.

The difference?  Now there is a formal deliverable associated with it.  And formal deliverables are reviewed, audited, and managed.  The data collection strategy outlines the overall scheme for data collection, declaring more explicitly than ever before how we are going to capture data. 

The creation of this deliverable isn't a mere paper exercise.  It helps us think through the strategy with our clients, and supports the latest (more complex) sampling requirements in SCAMPI v1.3.

If you were not doing this already, this will represent a fairly significant impact on how appraisals are conducted.  The preparation for the appraisal has become more robust, which will increase the amount of skill, time and effort it takes the Lead Appraiser to conduct an appraisal.  It will also become increasingly more difficult for "appraisal mills" to operate on the edges - pimping our 25 or more "successful" appraisals per year.  And this is a good thing!  

Proper appraisal planning takes weeks, if not months.  And that's the way it should be.  Appraisals are not only complex, with many moving parts, but they can impact a business in profound ways.  We don't skimp on this!

The specific changes around data collection strategy include:

• The choice of data collection approach (discovery, managed discovery and/or verification)
• When the data will be collected (e.g. preparation phase or conduct phase)
• What data collection techniques (e.g. demonstrations, presentations, interviews and questionnaires) will be employed for both objective evidence types (artifacts and affirmations)
• A detailed definition of what data will be collected and from what sources or events
• The organization responsible for collecting the data

I've been addressing data collection issues in my last few posts, and will do more in the future.  But for now, a lot of folks are asking why the rules have been made so explicit.  I believe the philosophy driving the change is appropriate and think it's about time the SEI has addressed it.

The SEI does not want Lead Appraisers to perform appraisals that have not been appropriately planned and prepared for.  They want appraisals to be useful and professionals - and I'm right there with them on this.  This might be bad news for professional level-seekers, but it's a good thing for our industry.

Don’t take it personally, Charles.  They weren’t thinking of excellent Lead Appraisers like you when they made this rule.  The best Lead Appraisers have been planning and re-planning all along, because we want to see the client win.  Most of us care more about that than so-called "CMMI Certification."

As General Eisenhower famously said during World War II: “No battle was ever won without a plan, and no battle ever went according to plan.”  Good advice for any appraisal.

Jeff Dalton is a Certified SCAMPI Lead Appraiser, Certified CMMI Instructor, Candidate SCAMPI Appraiser Observer, author, and consultant with years of real-world experience with the CMMI in all types of organizations.  Jeff has taught thousands of students and has received an aggregate satisfaction score of 4.97 out of 5 from his students.

Learn more at www.broadswordsolutions.com

 www.broadswordsolutions.com/about

Come out to the DC SPIN on May 4th! I'm speaking about Blogging!

Hey readers!  Yours truly (Broadsword's CEO and CMMI Lead Appraiser) will be speaking at the Washington D.C. Software Process Improvement Network (SPIN) on Wednesday May 4th, 2011.
The SPIN is sponsored by the Software Engineering Institute (SEI) at Carnegie Mellon University, and Washington's chapter is amongst the most active in the nation.

I'll be speaking about the impact that blogging and social media has on software process improvement and will be speaking about this here blog "Ask The CMMI Appraiser." The will my second appearance at the D.C. SPIN.

For more information about CMMI, CMMI Appraisals, CMMI Training, and consulting go to http://www.broadswordsolutions.com/.

What is the value of a CMMI ML3 Rating?

Dear Appraiser,

I just found you from a web search.  Can you provide some information?

At a high level I am trying to discuss the opportunities that will open up to a company that has a CMMI Level 3 rating. Can you give me any basic numbers?  i.e., a company that has a CMMI L3 can bid on $1B worth of contracts that are only open to ML3 rated companies.

Great question!  But if you're looking for a dollar value, I'm afraid the answer is "it depends."  There are billions in contracts in play where CMMI plays a role - all over the world.

In terms of value, there are so many factors that it's impossible to say "your productivity will increase 50%" or "you'll make 30% more profit."  Although you will see substantial gains, if done right, your company's individual situation is the driving factor in "how much" productivity gain you would see.  Also, the reason WHY you do it will have a major impact.  More on that later.

There have been many attempts over the years to quantify the value of CMMI and most of the efforts have come up short in terms of data quality (IMHO).  I was just in an SEI meeting the other day on this very subject!

If I asked 3 clients to tell me the difference between a "defect," a "fault," and a "flaw" I would get nine different answers.  If they collected "defect" data, it would all mean  different things.  You can imagine the affect aggregating it would have - it would be misleading (that hasn't stopped some from publishing some very authoritative metrics over the years….but that's another story!).  Worse yet, someone might ACTUALLY do something based on the data!  Yikes!

I can tell you that there is a LARGE amount of business in both the federal/state government sector AND the commercial sector that request that you have a CMMI rating to bid and maintain a contract.  And many more that consider a CMMI rating positively in their scoring system for contract award.  This opportunity is many times the annual revenue of even the largest service company – so can best be described as "huge."  For instance, many of the automotive companies require CMMI ML3 to bid on electronics, software, and engineering business.  Many states are starting to get on board as well.

But there are better reasons to adopt the CMMI.

The reason these organizations want you to be CMMI ML3 is because, if you do it correctly, you will experience much better quality, increased productivity, less project delays, you'll be on-time and on-budget more often, and your workforce will be happier and more productive.  This goes for agile, waterfall, V-Model, RUP, or any other types of methods or life-cycle you use.  

You may note that I've stressed "if done right," or "do it correctly" a couple of times.  CMMI can put your company on the path to greatness, but there are many (too many!) examples of companies that did it just to 'get the certificate" and missed the point entirely.  They may get the paper, but they don't get the value.

Your choice!

Your best bet would be to engage with someone who really knows the facts (an SEI Certified SCAMPI Lead Appraiser) and to spend the day with them so you can fully understand what you're getting in to.


oh, I happen to be one of those!  haha.

Good luck!

www.broadswordsolutions.com

I hear that Lead Appraisers have to attended SEPG to keep their certification. Is this really true?

Dear Appraiser,


I heard the the SEI is requiring all Lead Appraiser to attend two SEPG's every three years in order to maintain their LA Certification.  This is incredible!   My company will never pay for all of that! What can be done to change this insane requirement?

The one thing I LOVE about this business is how fast the rumors fly! And they're not just juicy rumors. You know, the kind that happen after-hours down in the mail room.  They're the evil kind and usually someone created them to win some business or to get a free trip to Portland!

Once I met a CMMI Instructor candidate who told his employer that "The SEI requires me to go to every international site to teach this class."  He cashed in his business-class ticket for two coach seats and he and his wife travelled the world!  Whooo hoooo!  Another time I did an appraisal where their LAST Appraiser told them the SEI said "the appraisal team all had to wear matching shirts!"  This is a true story  (and why he was their LAST appraiser!).

As a matter of fact, I make my living being the "second" appraiser.  Think about THAT one.....

Last time I looked, the SEI wasn't very fashion conscious (you can't even buy their shirts anymore), nor were they travel agents.  They are also, sadly for some, not trying to force you to Portland (although I bet they would love for you to come).

The cool thing is that when I hear these rumors I can almost always trace them (I almost said "REQM SP1.4 them....") back to some evil person who is trying to manipulate someone else by using what I call the "SEI Hammer."  This is the same hammer that is used to tell a company that every sub-practice must be fully implemented (so you buy more consulting time), that you need templates for everything (so you buy the consultant's templates) and that every member of your company must take the CMMI class (so you pay instructor fees).

Don't get me wrong, consulting, templates, and training can be a good thing - sometimes a VERY good thing. But nothing is automatic.  And if someone says "The SEI makes you do this" then, you should be asking some serious questions.  95% percent of time it's hogwash (that's nonsense" for all our non-english speakers....).

On the subject of certification renewal, Jeff Welch and the folks at the SEI have done a pretty good job (along with feedback from the Partner Advisory Board and other members of the community) of coming up with a broad range of options for eligibility.  This is all brand new, so some of it could change.

In order to maintain your LA certification there will be a "Point System" where 5 points is the magic number.  You will receive between .05 and 1 point for attending SEI training classes, leading an appraisal, conducting a CMMI class or some other "SEI" activity.  There are maximum point accumulations in each category.  You will also receive .05 points for attending outside training classes, workshops, conferences (bingo!), for participating in a SCAMPI A/B Appraisal, and for being on an SEI working group, the PAB, or other committee.  There will also be points for writing books, articles, or presenting papers.  Heck, maybe I can get a point for writing this blog!  Hmmmmm.....Mr. Welch?

Details of this plan will be available this month from the SEI..... so stay tuned.

So, while I would love it if they would make me come to SEPG every year (since I go anyway) to maintain my certification, it just isn't so!

But I DID hear that the SEI is making me live at the beach year-around . . . . . can I send them the bill?

Our Picks for SEPG 2011!

SEPG 2011 will be here in two weeks!  It's the world's greatest gathering of professionals interested in engineering process improvement and CMMI so you don't want to miss it.  There will be dozens of excellent speakers on many topics, so whatever you choose you'll have fun, learn a lot, and leave Portland smarter about CMMI than before you came.

Here are our "picks" for this year's conference.  My list is merely the topics I am interested in, not a judgement of "goodness."  They are all good!

MONDAY:

The Do's and "Don'ts of Process Improvement with Pat O'Toole.
- Pat is a very experienced SEPG Presenter and always pleases his audience.  He manages to be BOTH funny and informative at the same time (I hear he chews gum at the same time too!).

CMMI v1.3 and Architecture with Mike Konrad ("the wild-man") and Lawrence Jones. These guys always put on a good show - not to be missed!

TUESDAY:

The Agile SCAMPI: Taming the Savage Beast with Jeff Dalton
- OK, you caught me.  Come see me, even if it's for the hats, t-shirts, and other great swag you might win.  Oh, and I'll also be talking about how to make SCAMPI Appraisals bearable too.

Using Organizational Objectives to Guide you Process Improvement Program with Steve Masters
- Steve is a terrific presenter and this is an important topic.  The right objectives are the key to a successful program, and Steve is the guy to deliver it.  Disclaimer: I also think Steve is a really cool guy who knows his stuff.

Planning SCAMPI v1.3 Appraisals with Alexander Stall and Will Hays
- If your going to be conducting a CMMI v1.3 Appraisal, you need to hear what these two very smart guys have to say.  Will literally "wrote the book."  Actually, I think he's written multiple books.

Culture and Process Capability, the Secret Sauce with Hillel Glazer (aka "the reverend") and George Zack.  The #1 enabler of process capability is culture - I don't know George, but Hillel always puts an a good show!  Be prepared for some serious knowledge!

Gala Reception Tuesday night.  Don't miss this, it's a blast!

WEDNESDAY:

Who Cares?  So What? PPQA that Adds Business Value, not Overhead with Heather Oppenheimer and Christie Jensen
- A subject near to my heart, and with a great speaker!  I don't know Christie, but Heather is TOPS!

How to Sell Process Improvement with Molly Levy
- An excellent and important topic for anyone needing to push CMMI "upstairs."

Broadsword Client User Group and Appreciation Dinner Wednesday night.  If you're a client, or have been in one of our many classes, let us know you're going to be in town and we'll send you a personal invitation!

THURSDAY:

CMMI in Social-Media (for the Social Media Challenged!) with Bill Smith ((the world's only "CMMI Entertainer!").

Bill always puts on a great show and you'll learn a few things along the way too.  This is one speaker I never miss!  But watch out for the flying monkeys!

The Good, the Bad, and the Ugly . . . . the Process that is with Sabahat Khan and Rahul Bhalerao.
Real life examples?  Always better than theory.  And Sabahat is a cool guy and one exciting speaker!

Well, those are our picks for SEPG 2011.  See you in Portland!

www.broadswordsolutions.com

What's the biggest changes for CMMI v1.3 if we are going to ML2 or ML3?

Our company was appraised at CMMI Maturity Level Two last year, and next year we plan improvements that should have us performing at ML3.  What are the biggest things we have to concern ourselves with CMMI v1.3?

Great question!  A lot of people have been asking that lately on "Ask The CMMI Appraiser" so let's tackle it now.

By far the biggest changes for CMMI v1.3 have been with the so-called "high maturity" (ML4/ML5) process areas.  *** RANT BEGINS*** I say "so-called" because I don't like that term.  There is nothing about "statistical management" or "statistical techniques" that guarantees that the software being written, or products being built, are of higher quality.  This is a myth.  They COULD be, but the notion that if you apply statistics to problems they will improve was created and perpetuated by people who do not actually do it for a living.  Engineering is system that is about behavior, and no measures, charts, control charts, graphs, or Pareto charts will help you if you don't change behavior!  The same goes for Certified SCAMPI Lead Appraisers, some of whom specialize in so-called "high-maturity."  It doesn't make them better appraisers, consultants, or teachers.  It only proves that they can pass a statistics test.  Big whoop.    *** Rant over****.

But if your company is targeting ML3 behavior, you're in luck!  The biggest change is that some of the concepts that used to be contained in "Integrated Process and Product Development" (IPPD) are now practices ("expected") in IPM and OPD.  This is a good thing.  Anything that focuses on making teams better, on making things less confusing, and on defining team norms, is a good thing.

This will be particularly useful for Scrum teams, who already have some pretty good things going for them, assuming they are not of the Scrum-butt variety!

I was very pleased when the SEI came out with this - a change that makes the most important part of IPPD part of the mainstream. A good change that is most welcomed!

Another interesting change is the elimination of Generic Goals Four and Five. This means that you can no longer take just Project Planning (for instance) to Capability Level Five.  This seems like a rip, until you realize the ML4 was never REALLY Maturity Level 4, but CL4 with a couple of PA's.  That masquerade is over, with any luck.

Finally, although SCAMPI v1.3 is not quite yet released, there are a few key nuggets worth talking about.  We're told that the distinction between "Direct" and "Indirect" evidence is now eliminated, with just "artifact" left in its wake.  There are probably a lot of appraisal teams who have slaved over this cursing the SEI right now, wishing it had happened earlier, but it's better late than never and is a good thing.  The other SCAMPI change is in the sampling method.  No more "minimum of 3 projects allowed" (sorry offshore systems integrators, you're going to have to really be ML3 now).  You're going to actually have to provide data so the Lead Appraiser can make an informed decision.

By the way, if you're one of those companies that tried to hand 3 projects to the Lead Appraiser and hide the rest, shame on you.  I love it when an RFP tells us we only get to look at 3 projects, and we can't ask questions about the rest!  A sure reason to run the other way - as fast as we can!

All of this, and more, will be a good topic of discussion for any Certified CMMI Instructor or Certified SCAMPI Lead Appraiser in the coming months.  You can also tune into one of our free Webinars - the next one will cover this subject - at www.broadswordsolution.com.

Good luck!

www.broadswordsolutions.com

What is a Certified SCAMPI Lead Appraiser and why do I need one?

Dear Appraiser,

My boss said I need to go find a Certified SCAMPI Lead Appraiser to conduct a CMMI Appraisal.  What is a Certified SCAMPI Lead Appraiser, and why do I need one?  And why are they so expensive?

Well, you may not! :)  But if you want to receive a CMMI Maturity Level Rating, so that the great work you and your team have been doing can be verified and recognized, then you will need one.

A Certified SCAMPI Lead Appraiser (or "Lead Appraiser") is someone who is licensed and certified by the Software Engineering Institute (SEI) to lead a CMMI Appraisal (technically called a "SCAMPI Appraisal") at your company and lead a team to a consensus on whether you meet the definition of a Maturity Level "x" company.

The method they follow, the Standard CMMI Appraisal Method for Process Improvement, or SCAMPI, is a regulated methodology that is to be performed with integrity for all companies who desire to be appraised.  Lead Appraisers are schooled, tested, and audited for their expertise in performing these appraisals, and about half of them are also CMMI Instructors.

Why are they so expensive?  Well, there are a number of reasons.

Certified SCAMPI Lead Appraisers TEND to be very experienced - many have not only been practicing engineers/software developers/project managers with advanced degrees, but have also been executives, leaders, authors, and successful consultants for many years.  In short, a Certified SCAMPI Lead Appraiser needs to be "super-experienced" to succeed in their roles.  In addition to that, it is quite costly to go through the SEI's Lead Appraiser school, with many courses, examinations, observations, and other requirements that add to the costs.  Lead Appraisers must maintain a level of professional training and participation to keep their certification.  Finally, there are annual fees associated with being a Lead Appraiser, an SEI Partner, fee for conducting appraisals, and with maintaining currency.

For example, I have been a software developer, engineer, project manager, software architect, line manager, Chief Technology Officer, VP of Consulting, and CIO - all before I was a Lead Appraiser!

So they're not always your typical "consultants."  Many of them can add significant strategic value to your company due to their depth of experience.  In the end, the cost of a Lead Appraiser is tiny fraction of the overall cost of your process improvement program, so it's probably less important than, say, how much internal re-direct is going to be required to put your company on the Path to Greatness!

Good luck.

http://www.broadswordsolutions.com

CMMI and SCAMPI are registered service marks of Carnegie Mellon University

What is the difference between a CMMI Appraisal and a TS9100 Audit?

That is a great question!  Can I use it on my blog?

What is the difference between an appraisal (ala CMMI) and an audit (ala TS)?  The answer to that has several parts, so stay with me :)

Part I: Intent

The intent of the authors of these documents (the CMMI-DEV model and the TS Audit Specification) is different.  CMMI is not a spec, it is a collection of "lessons learned" or "best practices" that are suggested by the authors, who were a collection of companies, consultants, and research scientists.  TS is a strict collection of work products and procedure requirements.  TS was intended as an audit method to ensure that organizations were "doing what they say they are doing" and that they comply, at the deliverable level, to the audit requirements.  The CMMI was intended as a tool for helping companies "see" how a great company behaves, and to use it as a way to incrementally make themselves better.   It assumes that companies get to decide what is best for them.  In a way, CMMI is one-level of abstraction higher than TS in it's requirement to adhere to certain behaviors, with CMMI being "more trusting" of the TS of your professionals.

Part 2: Collaboration

The SEI specifically states that since CMMI is something that is collaborative in nature (with the participants being management, practitioners, and the Lead Appraiser) it does not have an "audit" component, but is appraised for further opportunities for improvement by a joint team led by the Lead appraiser, whose job it is to ensure that the appraisal method is followed with full integrity.  To support Part I (above), it is the Lead Appraiser/Appraisal Team's job to simply identify strengths and weaknesses, not to "pass" an organization.  OEMs and the DOD have mis-applied the intent so that the line between "audit" and "appraisal" has been blurred.  But that doesn't mean they are the same thing.

Part 3: Behaviors vs. Documents

The TS audit seeks documents that 1) say what you are going to do and 2) show that you did them.  The CMMI seeks evidence that people understand the behaviors that are desired by management, and that they are using those behaviors, and continuously improving them.  Along the way they collaborate with their lead appraiser for advice and assistance in interpreting the model itself.  The documents are merely one form, among several, of demonstrating what behaviors have been used and what the results are.

Part 4: Interpretation of Evidence (this is probably the most important difference)

For a TS audit, evidence is paramount.  Do the documents exist?  An Auditor is following a checklist . For a CMMI appraisal, the documents are merely one consequence of process, and it is vital that the Lead Appraiser and Appraisal Team have a great depth of understanding about how the organization works, what their business needs are, and what they are trying to accomplish.  They synthesize this to try to determine if what is being done is useful and meets the needs of the organization, and their findings will reflect that.  It's not pass or fail, but strengths and weaknesses.  The fact that OEMs and the DOD are trying to force a Pass/Fail mentality does not mean that it should be treated that way.

If you follow a process and a plan, and you incrementally make the improvements you need to make, then the appraisal will come when you are ready.  This is one reason why it's hard to "jam" it into a short schedule as you might a TS audit.  You need to convince the appraisal team (and the certified lead appraiser) that it truly is useful and an infrastructure exists for continuous improvement . . . . and that they are all being used by everyone!

www.broadswordsolutions.com

We like our consultant, and want him to be our Lead Appraiser - but isn't this a conflict of interest?

We’ve been working with a consultant who is also a Lead Appraiser.  We’d like to use him for our appraisals since he knows our organization so well.  My boss is challenging the decision saying that an audit performed by our consultant wouldn’t be objective.  What are the rules on this?

As the folks at the SEI are so fond of saying . . . it depends!  Specifically it depends on what you mean by “consulting.”

There is a lot of hyperbole on this subject - with lots of so-called "experts" weighing in.  Here's the answer.

The SEI takes conflict of interest very seriously and the Lead Appraiser’s Professional Code of Conduct requires us to report any real or perceived conflict of interest for every appraisal we conduct.   As part of their QA Process the SEI reserves the right to further investigate any conflicts (or anything else for that matter) to ensure that appropriate behaviors were exhibited.  This makes sense.

So, on the face of that very limited information, it would seem that a consultant might have a conflict of interest.

However, and this is a big HOWEVER (so wake up and pay attention!), not all consulting is the same, and the “type” of consulting you’re asking about matters if we really want to answer the question accurately (and be fair to both you and your potential Lead Appraiser).

Rant begins.....

I like to tell my clients that I don’t do anything.  Yes, I literally tell them “I don’t do anything!”  So why do they keep hiring me?  Why have I had ten banner years in a row with some of the biggest names in the industry?  Simple.  I really don’t do anything. Well, I do advise them on how to be better process engineers (I’m kind of like BASF . . . I don’t make Processes, but I help you make Processes better…).  I help them make complex decisions, tell them where the roadblocks might be, how much things might cost, help them to communicate their objectives, and so forth.  So I guess I do something, just not what you might expect a “process consultant” to do.

Rant continues….

Everyone and his brother claims to be a consultant these days.  But being a consultant means more than being in independent performer.  And being a professional consultant means helping clients succeed by helping them help themselves. (and not leaving too many fingerprints behind…).  If you write processes, perform QA Audits, or code and run test scripts, you’re not a consultant – you’re a contractor.  Sorry.

Many, many contractors believe they’re consultants.  So many make this claim that clients now call contractors “consultants” – but most of the time they’re not.  This further confuses the conflict of interest discussion.

Now back to your question…..

So, if you “consultant” is also your “process guy” – if he writes process descriptions, does the QA, runs your SEPG, or especially “prepares” your teams for appraisals (or anything that looks, feels, or smells like that), then no, it’s probably not appropriate to use him or her as your Lead Appraiser.

But if he or she guides you towards becoming a better process engineer, helps you understand the CMMI better, suggests what training you should take, gives you tips on being a better communicator, how to run your SEPG, facilitates some workshops, and generally helps you down the “Path to Greatness” where YOU decide what your processes and measures should look like, then it may* not be a conflict of interest – but the Lead Appraiser should still document it in the appraisal plan. 

Just like anything else there are gray areas, so have an honest discussion with your Lead Appraiser about it.

One more thought.  Your boss used the work “Audit.”  A SCAMPI Appraisal is NOT an audit, it is an appraisal meant to identify strengths and weaknesses of your organization – which can be uncovered MUCH MORE EASILY if your Lead Appraiser knows a lot about your organization already.  Bringing in a Lead Appraiser “cold” is a recipe for waste and disaster and a huge, unnecessary, self-imposed risk.  Why would you do that?

After all, CMMI isn’t about “getting a level.”  It’s about collaborating with people to make your organization better.  Make it real.

* you may have noticed I've used the word "may" here several times.  This is because every situation is different and has to be evaluated in context.

www.broadswordsolutions.com

What is a "Pre-Appraisal" and what are the rules?

Hey Appraiser!

Can you tell me something about the ‘pre appraisal’ requirements and restrictions?

I want to know the following specifically

• What is the minimum required number of days that the appraiser himself needs to spend with the client company during the preappraisal

• How are man-days different than normal working days when talking about the pre appraisal duration?
• Does bringing more people from the appraiser for the pre appraisal work if the preappraisal days are reduced
•  How many days a client can ask for from the appraiser , for the pre appraisal?

You say "Pre Appraisal" and I say "SCAMPI C!"  Sounds like a song.

There is no formal event in the CMMI world known as a "pre-appraisal" but that term is often used to describe a "gap analysis" or a "SCAMPI C" so I'll assume that's what you're referring to.

To answer your questions specifically:

- There is no minimum required days that the appraiser needs to send with the client company.  In order to do a thorough and professional job the appraiser should spend time talking with a large cross-section of your organization, select project that provide documentation for review, and to review the process documentation.  There should also be time for meeting with your executive sponsor and other key stakeholders

- man-days and "normal working days" should be the same.  Just remember that appraisals often required "pre-work" that is often done offsite, and "post-work" that is often done offsite as well.

- nine-women can't make a baby in one month, so more people from the appraiser's organization may not help.  There are SOME administrative things that can be shared by other people, but the LA still needs to see and speak with everyone, so it doesn't make much difference for a pre-appraisal.

- the LA and the client sponsor should collaborate on the amount of days required.  It all depends on the goals and objectives, the deliverables that have been agreed to, and the scope of the appraisal.  For me, a pre-appraisal is usually 2-3 days, but I have had them last as long as 5 days.

Good luck!

www.broadswordsolutions.com

Can we skip steps from the CMMI for maintenance projects?

Dear Appraiser,

Our company has several projects that have been already developed. Now, our activity regarding those projects is corrective maintenance. We have defined a very complete process, since an issue on the incident is opened to the issue is closed. It passes through the following states: new, assigned, planned, agreed with client, developed, solved and closed.

My doubt is referred to small incidents, in which making the plan takes longer than solving them. We thought on defining a reduced process for incidents that can be solved in less than 8 hours. This process skips some steps from the complete one. Do you think it would be right? To what extend do we have to follow the standard defined processes in situations where its management becomes a bureaucracy?

Great question!

The CMMI is not a "standard process" but it is a set of guidelines for designing a SET of standard processes.  This means that you can have a set of processes for "new development" and a set of processes for "maintenance," and a set of process for . . . . . well, you get the idea.

All the CMMI really is is a set of reminders for us to do what's right for a given situation.  So, if it's new development, perhaps a detailed requirements spec is required, with a lot of oversight and review.  It it's maintenance, maybe it's a work order with very little oversight.  Both could be "CMMI compliant" but each would be unique to the needs of the organization using it.  It's "right-sized."

The "states" you’ve described are like "life cycle phases" on a development project and should be treated as such – just smaller.  The time to NOT follow a process is when it is only a "bureaucracy."  That is a good sign that the process doesn't "fit" for your project.  In that case, do something different!  

I don't think it's a matter of "skipping" as much as it is a matter of doing the "right thing" for a given type of work.  I would recommend you ask the company to provide you with a "lite process" for these types of work, but still one that accounts for all of the required steps.  It's easier than you think – simplify, simplify, simplify – and don't do ANYTHING because the "CMMI says so!"  Do it because it's good for business.

Good luck!

www.broadswordsolutions.com

As Engineers we make decisions all the time, why do we need a DAR Process?

Dear Appraiser,


Our Lead Appraiser is not satisfied with the way we make engineering decisions.  We've tried to tell him that we hire really good people that are really smart, and making good decisions is part of their job description.  What's wrong with this?  Why do we need some heavy process to manage what we already to really well?


Yes, really great people doing great things. . . .why have any process at all?  Boy, you ARE smart!


Many who read "Ask The CMMI Appraiser" have noted that I don't think much of the "advice" you get from some of their so-called consultants, but in this case I'm going to have to call you out.  Good people don't always make good decisions because there are many other factors that affect those decisions - like being overworked, underpaid, and not having the resources to do a quality job.  In other words, good people are human, and humans are fallible.


You're right about not needing a "heavy" process - none of your processes should be "heavy."  They should be useful.  But it's a false choice your presenting.  It's not "heavy or nothing."  Process lives along a continuum, and DAR is the perfect process area to make this point.


DAR is your friend, and it exists to ensure that your buddies and your management don't trash you and your decision after you make it.  It will reduce the chance that your decision won't cost the company millions in unexpected costs (and thereby putting you on 99 weeks of unemployment), and will ensure that whatever you've decided won't be turned around the minute you move up (or move out).


To illustrate the value of DAR in my classes I ask the students to write down the "3-Ds."  Durable,  Defendable, and Deliberate.


A decision is said to be Durable when it stands the test of time, outlasts the person making it, and stays the right decision long after it is made.


A decision is said to be Defendable when it can survive the inevitable sniping be detractors who are offended that they were not asked to make it, those trying to score a point by proving you wrong, or just boneheads in management that think "adding value" means to question every decision a  subordinate makes whether they understand it or not.


Finally, a decision is said to be Deliberate when (and this is ironic . . . ) a well-defined and executed ENGINEERING process is used to make it - thereby supporting the Durable and Defendable parts...


So that's why DAR is your friend, and a good process to use when you need to make any kind of important decision.


Oh, and by the way, smart is not a differentiator . . . most engineers are smart.  It's obviously not enough to be smart.  And using it as an excuse to not be even smarter, well, that's dumb.


www.broadswordsolutions.com