Thursday, December 20, 2007

I have two Appraisal Team questions

1. Exactly what can & cannot be kept following an appraisal in the way of written notes, excel files etc that where generated by the ATMs during the appraisal?

2. Can ATM team members use laptops during the final appraisal in document review, during interviews etc.?

Take only memories, leave only footprints . . . . oh, that's the National Forest!

Any SCAMPI Appraisal generates a large amount of written documentation including interview questions, answers, practice characterizations, findings, and observations. The Appraisal Plan, and subsequent entries in the SEI's SAS system will identify the formal appraisal outputs that, at a minimum, include Final Findings, Appraisal Plan, and the Appraisal Disclosure Statement. Any notes or written documents that identify the source of the information, for example an answer to an interview question, should be destroyed in order to maintain the "non-attribution" policy that exists within the method. NA is a critical component, as a violation cast doubt on the reliability of the responses to the questions. This includes spreadsheets and files that identify which project team information was derived from. I usually recommend destroying everything that is not part of the formal appraisal record.

I also usually ask the Appraisal Team Members to not use laptops during interviews. I have found that this is odistracting to the interviewees, and often makes them more nervous than they already are. I have also seen ATM's surfing the web and playing with email when they should be listening and taking notes.

Call me old-fashioned, but a yellow pad and pen is the best tool for this job.

Is their a set of standard questions for CMMI SCAMPI Appraisals?

Is there an “official” list of questions that appraisers use when they are examining an org’s processes to determine their alignment with a given level of the CMMI model? Or, do the appraisers build their own questions based on process characteristics associated with each of the five levels?

There is no “official” question bank for CMMI SCAMPI Appraisals. Client’s often ask this and reason that such a question bank would help to standardize the appraisal method, however the specific context of each appraisal (organization, industry, product types, culture, et al) make it virtually impossible to make standardization effective. It’s far more informative to design the questions around the organization than to try to get them to answer unfamiliar questions.

Here at Broadsword we have a database of several hundred potential questions worded in many different ways that are mapped to the practices in the CMMI. Some questions map to multiple practices or even multiple Process Areas. We select the appropriate question set after spending some time getting to know our client and then modify the high-context words to better suit the language of their culture. This seems to produce the best results.

Tuesday, December 11, 2007

Are these work products good evidence for CM and TS?

Could you please answer the following questions about CMMI:

1- Are configuration action proposal and configuration improvement proposal direct artifacts if I try to implement the Specific Practice SP1.1 of the Process Area CM (Configuration Management)?

2- Are examples of software process architecture include: object-oriented model, unified model, and waterfall model?

CM.SP1.1 asks us to "Identify Configuration Items" and is part of Generic Goal 1 "Establish Baselines." I'm not sure what a "Configuration Action Proposal" or a "Configuration Improvement Proposal" are, but they sound like "Organizational" work products, and not project-level work products. SP1.1 is looking, literally, for a list of work products that we think are going to be part of our project. Examples include: product specs, requirements, process diagrams, drawings, test cases, test results . . . ."

Software Process Architecture, if I understand your question, defines the process framework that will be used to develop software. OOD, RUP, and Waterfall are all examples of PART of that (Life cycle models) but probably don't include a lot of items that should be part of the process (customer commitment to requirements, PPQA-type audits, peer reviews . . . ). So RUP is a subset of the required architecture, but not the entire process architecture.

Why are GG4 and GG5 not applicable for the Staged Representation and Maturity Levels?

I have a question regarding Generic Goals and Practices in the CMMI. In the Staged Representation GG 1 to 3 are Applicable even though the Maturity Level is 5. But in Continuous Representation GG1 to GG5 are applicable to Capability Levels 1 to 5. Why is it like this?

Fortunately, your not the first person to express confusion about this one. The answer is simpler than you might think.

First, a small correction. GG1 is not required for the Staged Representation. Staged appraisals start with GG2. You are correct that GG4/5 do NOT apply to the Staged Representation.

When you approach the CMMI using the continuous representation (one process area at a time is appraised) you need all of the Generic Goals (GG1-GG5) because you are not being appraised on the interdependent aspects of the other Process Areas. In other words, if you being appraised just on Configuration Management (CM), we want to know if you planning it, assigning resources, etc for for Level 2, if your Institutionalizing a defined process (for Level 3), if your managing it quantitatively (for Level 4) and if your optimizing it continuously (Level 5). If you take the Staged approach (a set of PA's at each appraisals) those things in GG4 and GG5 are ALREADY present as process areas (QPM, OPP, CAR, and OID). Also, in ML3, OPD, OPF, and OT help support what is in GG3.

So, given all this, you PROBABLY won't go to Capability Level 4 in OPP and QPM, or Capability Level 5 in OID and CAR because it would be redundant to GG4 and GG5.

Hope this helps!

Saturday, November 24, 2007

Technology meltdown (again) causes blog outages

Hey Readers!

Well, for the second time this year my traveling laptop refused to boot up. It's not such a bad problem when we're home at the Broadsword office, but it always happens when we're on the road, far away from discs, files, and backups. We love the Sony Vaio's here (light and fast) but after the third meltdown I'm looking for something new. They really don't hold up well on the road (sorry Sony).

I'm typing now on Patty's Apple MacBook and giving some thought to making the switch over to the dark side (or the white side, to be more precise). As a Windows user since the beginning (and DOS before that) I'm loathe to admit that a Mac might meet my needs, but with MS Office, MindJet's MindManager, and some of the other tools I use also available on the Mac, it might just work. I'm getting to the age where playing with drivers and hardware is just downright boring. I just want it to work!

Fortunately, we've been practicing pretty good Configuration Management here at the 'sword, so restoring files shouldn't be a problem, although emails could be a problem unless I go to another Outlook-capable system. I'll be executing a product-selection DAR soon enough on a new machine and software (if it's a Mac). I need to ad "durability" to my scoring criteria.

So, I'll start with a Requirements Development exercise, execute a DAR, and finally SAM using IPM/PP/PMC to help drive the process.

In the end though, the decision will be made by accounting :)

If anyone has successfully transitioned from Windows to Mac in an engineering work environment I'd be interested in hearing from you.

See you all when I'm back up and running!

Wednesday, November 21, 2007

Some crazy holiday blogging!

Hey readers,

Right now we're blazing down the highway (I-80) through Ohio on our way to our annual Thanksgiving event with my four siblings. As usual, I'm thinking about PROCESS and answering a few blog questions.

We're in a massive RV - 32 feet long, the boys are in the back playing "World of Warcraft," the wife is at the wheel, and I'm the Internet junkie, managing a wireless web inside the RV all through a Blackberry 7250. Slow but effective for all three laptops.

Here in the US we're about to celebrate Thanksgiving, and it's an appropriate time to say THANK YOU to all of you readers and posters who have made "Ask The CMMI Appraiser" successful this year. With over 1000 hits a week it's way more than I ever thought it would be and I've met 100's of people since I started a year ago.

For all of you in the US, have a great Thanksgiving. If you're outside the US, here comes an extra day for you to get ahead of us! Don't worry though, we'll be catching up on Friday.

Keep the questions coming!

Tuesday, November 20, 2007

So what is the difference between Agile and Waterfall anyway?

Could you tell us what the difference between Agile and Waterfall?

Sure, Agile guys are super laid back, and waterfall folks are really uptight :) No, that's not true, I know lot's of uptight agile guys too!

There are many differences between Agile methods (XP, SCRUM, Crystal, etc) and traditional "waterfall" methods, and a whole book could (and should) be written about it, but there are a few things that stand out for me (these are not all inclusive by the way).

- In an Agile environment, budget and time are fixed around Releases and Iterations, and scope (within each Release) is negotiable.

- In a waterfall environment, scope is fixed and both budget and schedule are negotiable (no one wants to negotiate these, but they do).

- In an Agile environment, a complete lifecycle is executed incrementally and iteratively against a subset of requirements during each iteration, enabling both the business customer and the development staff to learn along the way. These small sets are delivered to customer in useable chunks.

- In a waterfall environment, the requirements are fixed at the beginning, extensive planning is done, and all changes are managed through a formal change control process. The entire set of requirements is completed, the code is designed, built, and tested, and then delivered.

Finally . . .

- In an Agile environment, they pride themselves on focusing more on development and less on documentation (for better or worse)

- In a waterfall environment, most everything is written down, many times (for better or worse)

Both of the last items are problematic, but that seems to be the behavior.

How do I get smart FAST about PPQA?

I've been assigned the PPQA process area at my company and I don't know much about it. How do I get smart FAST about the practices?

How to get smart fast? OK, first get some #8 AWG/3 wire and and a plasma welder .. . .

I would start by simply reading the PPQA PA making sure to read the informative material (in other words, the whole chapter). Remember that PPQA has multiple purposes - one focuses on "are we using the process?" and "is the process appropriate?" The other is, "what insight can we gain by understanding these two things?" Is the process useable? Is it helping people? How can we make it better?There are numerous books on the CMMI that could help you - one that I like because of its organic approach is Mike West's "Real Process Improvement with the CMMI."

Good luck! And don't hurt yourself!

Monday, November 19, 2007

How many CMMI Experts do we need to achieve CMMI Level Three?

Dear Jeff,

We are suggesting that the IT shop of a company should move from level 1 to level 3. They have all of the documents available to give the benefits of implementing CMMi, however, they did not take into account the resources required to move up the maturity level. My question is: how do we size the number of CMMi experts our client will need to transform its IT activities from level 1 to 3 (say for 800 people). I would appreciate any thoughts that you might have on this.

Wow, you have ALL the documents in place? Doesn't that make them Level Three already? I'm kidding of course.

There are two ways of looking at your situation. My preference is to say "no additional resources are required" to achieve CMMI Maturity Level Three.

How is this possible? It's because the CMMI isn't something you can "do" to a client. It's something only they can do to themselves. It isn't about documents, it's about culture. I have had appraisals where all of the documents were in place, but it was obvious the company was not performing any of the behaviors from the model, and they were not successful at achieving CMMI Level Two or Three.

If an external firm does this to a client, they will leave at the conclusion of the engagement with all of the knowledge of how to develop and sustain the process. Part of Level Three is knowing how to design, deploy, and sustain a process. This might be good for business and billing rates, but it's not the best thing for the client organization.

Here at the 'sword we normally recommend to have one person onsite part time helping to guide the client organization towards managing this themselves - but any more than one tends to muddy the waters and the organization starts to depend on us too much and won't change themselves.

Now, once in awhile we come across a client that wants to hire an outside firm to do this for them. This is not something we do, but if they insist on a recommendation of needed resources, we have learned that to kick off and sustain an effort to move from CMMI Maturity Level One to ML3 takes about 3% of the overall engineering staff, plus another 1.5% to perform "PPQA" responsibilities. Given that you've said your client has 800 practitioners, that would account for about 24 people plus 12 for QA. How many of those are client personnel would depend on their skills and availability.

I would implore you to consider alternate ways to accomplish this goal that involve the client taking on as much of the load as possible. Your chances of success (and theirs) are far greater if you do!

Sunday, November 11, 2007

CMMI Accelerator Products

Hey Readers!

I don't usually self-blog but it's a slow Sunday and we here at the 'sword have some exciting news. For several years we've been focused entirely on consulting and training, and it's been going great. We've been growing fast, and bringing on new talent, so a few months ago we all got together to do some "strategic planning."

When discussing the many clients we have helped this year (thanks to all of you!) we realized that there were some companies out there that may want to "roll their own" when it comes to CMMI. Hey, this is great and is the same thing I did when I ran my first CMMI program (seems like decades ago).

Our sales person, Jill, kept asking what were the things all companies needed to do for CMMI that might be mundane, tedious work, and would our own internal collection of templates, tools, work products, procedures, and processes be of any use if we were to "productize" them in order to save these guys a few thousand hours? What a great idea!

So, we're announcing the release of our brand new "CMMI Acccelerator Boxed Set," an appraisal proven set of work products that will shave about one-thousand hours off of your program if you're just starting up.

For more info about the CMMI Accelerator click over to

Thanks for listening!

Wednesday, November 7, 2007

Can we just ignore Product Integration?

We have made solid progress on a very neat integrated process tool including process models, deliverable templates covering the full lifecycle from the point where one gets the contract to go-live. I am not clear in my mind how to deal with product integration. If I was a large system integrator, I would expect it would be all about taking the big software components and making them work together. In our case, I feel we have no significant product integration, because our method is always integrating the new item with the old as part of our normal lifecycle. Therefore, I would like to say that product integration is not relevant to us, but suspect that is not possible?

Well, let's start at the end first. You're correct, Product Integration cannot be "out of scope" for purposes of a CMMI Appraisal. In other words, if your goal is to achieve CMMI ML3 performance, then PI is a "must have." The only process area that can be out of scope is Supplier Agreement Management (SAM) and ONLY of you truly don't use suppliers.

That said, it's interesting that so many people overlook what they are already doing when it comes to "developing processes." I have not seen your system, but based on your description it sounds as if it manages the sequence, creates the environment, and has established procedures and criteria (or else it would not work!). If it also validates program-program interfaces, and perhaps even creates some documentation, even better. Didn't I just list more than half of the practices within Product Integration?

Sometimes the "stuff" we do everyday is, by definition, the process. You may need to round it out with some process definitions, guidelines for tailoring, training, and policies, but in the end, it sounds like a "PI Process" to me. With all of the bad tool implementations out there it's refreshing to see an example where a tool actually helped.

I recently conducted an appraisal where the client had a similar system to manage the entire lifecycle. Their system also created the executable package, the documentation, and installed it on the client site (they had a dedicated, long-term client). After digging into it for a couple of days we determined that it (with a few documents) did meet the spirit of Product Integration.

These types of innovative solutions are too few and far between in our business. Documents do not make a process - and the mentality that it takes a million documents to succeed at CMMI is just wrong. Congratulations on a great start.

How much of our company has to be ML 2 for us to say the whole company is ML 2?

Can you tell me what percent of your company needs to be recognized as certified Level 2 in order to state that your entire company is Level 2?

Contrary to popular belief, an entire company rarely achieves CMMI ML2 (or any other level for that matter). There is a concept in the SCAMPI Appraisal Method called an "organizational unit" that defines the target organization within a company that is to be appraised. When the appraisal is complete, you may say that your company has reached ML2, but in reality perhaps only a single organization in your company has reached that goal. That information is displayed on the SEI's appraisal results page, although I would argue that most people aren't aware of the concept when they look at.

The definition of the Organization Unit, or OU in Lead Appraiser speak, is something you discuss, and agree upon, with your Lead Appraiser. It needs to be credible (such as an entire location (Dayton, OH), a product line (mainframe payroll systems), a type of technology (all web development), or a management tree (everyone reporting to the Director of Engineering. . .). Inside of your OU, you and the Lead Appraiser will select a group of projects (3 or more) that is a representative sampling of the OU. Again, this is something you and LA agree on in advance.

A poor example is one where the OU is "all projects with the charge code 145387.

When it's all over (and if you're successful) if you're like most companies you will say [company name] is CMMI ML2. Some are more specific, and name the OU, but it my experience, most do not (right or wrong).

When you enter Baltimore-Washington airport there is a huge banner across the road that says "Lockheed-Martin - CMMI Level Five." I can assure you that, while they have achieved ML5 within some organizations, the "entire company" has not.

One exception I can think of is a company whose sole business is writing software in a single location. Perhaps then the company is ML5.

So, there are two ways to slice it. First, the organizational unit is agreed upon, and then a set of representative projects is selected. Make sense?

Thursday, November 1, 2007

How do we avoid bulky processes when using the CMMI?

We are in the process of defining our process artifacts after a CMMI Gap Analysis . Due the lack of experience in CMMI I am finding the definitions we are creating are too bulky and that worries me lot when I think of the implementations . Suppose we define all that is required , how can we best take extracts and publish it in a QMS system for others to practice?

Remember this:


Sometimes, "lack of experience" is exactly the RIGHT thing needed to be successful! I like the way you're thinking - don't stop doing that!

Usable and useful processes are a pre-requisite for any company to be successful, and too many go down the road of "over-interpreting" the CMMI so that you end up with heavy, burdensome processes that are not useful. Remember, your engineers are not there to "follow the CMMI," they are there to build creative solutions to business problems using technology.

One common mistake is trying to implement a process that includes all of the "typical work products" and "subpractices" within each specific practice. This was never the intention of the model. These are only examples, and are not "required" components of the CMMI.

What you need to do is internalize the meaning of the practices and goals in each process area and piece together a puzzle that shares work products and procedures across many process areas - in other words INTEGRATE don't REPLICATE.

Once you've identified all of the processes and work products conduct a CONSOLIDATE exercise where you consider the data in each work product and determine ways to reduce and eliminate documents by combing similar data, the ELIMINATE redundant and useless data.


Wednesday, October 31, 2007

Why doesn't the CMMI address resources or people?

I know little about CMM levels as I am new in this industry. I went through the different levels and have observed that all these levels are related to the product and its quality only.

I want to know why resources or employees are not considered in these levels as the products are developed by these resources.

Ahhh, the smell of fresh blood! Welcome to CMMI and thanks for the post!

It took me years to really understand the model (some would say I have a long way to go!) but trust me when I say that it does address resources and people in many ways - although it's not always as obvious as it could be.

According to the CMMI, a "project" involves the application and coordination of resources (people, time, money, hardware, software, tools, data, et al) towards the achievement of a goal or objective (like the creation of a product). How do you apply these resources? You apply them through the use of processes and procedures right? Those processes are controlled and monitored by these same processes.

There is also a "version" of the CMMI called the People-CMM, or P-CMM that specifically addresses "people" related issues, but it is not yet commonly adopted in engineering or IT organizations. Cheers!

Must we use historical data for estimating?

This is first time I am directly writing to you, otherwise I've just been reading your comments and feedback on the group.

I have two questions on which I'd appreciate your comments, because answers to these will help me in building sound concepts of CMMI v1.2. Let me give you a brief history of our organization. We are striving for ML 2 organization and ready for our Class A Appraisal.

Since we are striving for ML 2, we are in the phase of data collection which later would be referred to as historical data for the estimation of effort, cost and size of the project. Currently we are estimating Size using FP Technique and Effort using PERT technique. But there is no relation between size and effort established in our current OSSP.

It is also not required in the sub practices of SP 1.2 of PP. To satisfy SP 1.2 and SP 1.4 of PP Process, is it enough to show that we are in data gathering/collection phase and no direct relation between size and effort data exists or we have to show the relation and people checking that historical data as well while making estimates for the projects?

My second question is, trainings are planned, conducted and people do their routine work after getting training. Kindly let me know how the adequacy of trainings is determined for GP 2.5. How much details evidence is required for an organization that is going for ML 2.

Is it justifiable if an LA fails an organization just because

1. He is unable to determine the adequacy of trainings
2. Direct relation between size and effort data is not established and people are using their expert judgment but that judgment is not supported by historical data?

Your answers will be really appreciated and helps me and my organization to have better implementation of processes.

You've obviously been reading up on the model. You must be having trouble sleeping!

On the question about using historical data to estimate, and what is required by the model, I would ask this: if it were required by PP, does that mean companies that DON'T have historical data cannot achieve CMMI ML2? Of course not. I don't think that was ever the intent of the model. There are many ways to estimate the project, and using historical data is only one of them. The fact that you are developing a historical dataset is a good thing, and will only strengthen your performance and appraisal ratings in the future.

As to the training, we expect you to provide evidence that the right people were trained in the right process areas. There is no need for everyone to take all the training, or all levels of training. At ML2 there is no requirement for a formal "training capability" as there is in ML3, so 1:1 training, mentoring, presentations, or CBT's would all qualify as long as you kept records of people having attended or executed the training. Keep training sign-in sheets and the training materials and you should meet the requirements of GP2.5.

As for an LA "failing" an organization (technically speaking there is no "failing") because he is "unable to determine the adequacy of the training", I would say that this would be valid if he truly could not evaluate whether you had training or not. He should not be evaluating "goodness," only that you trained people appropriately for their roles.

Same for the next question. There is nothing in the model that requires you to use "historical" data for estimating size. That said, "expert judgement" may or may not be appropriate depending on what you mean by that.

Either way, don't let an appraiser tell you how to run your business or claim that there are prescriptive ways to perform a process in the CMMI - there are not any. It's strictly a definition of "what" not of "how" and the subpractices and work products are only suggestions.

Best of luck to you.

Tuesday, October 30, 2007

Can you validate our Control Limit formula?

I follow your site regularly and find your thinking very refreshing. Thanks for sharing a wealth of knowledge with the community. I have a question bothering me for a long time,

While calculating Control limits in an SPC chart, I have seen several organizations use the following formula:

Lower Control Limit(LCL) , Upper Control Limit (UCL)

LCL = Mean - 3Sigma

UCL = Mean + 3Sigma
CL = Mean

What do you think?

What do I think? I'm thinking this gives me a headache!

I'm not a statistician, but I do write a blog, so of course that makes me an expert at everything! (I love the Internet!). Your formula is similar to the formula taught in the SEI's CMMI Understanding High Maturity Practices class and also in the best text I know on this subject, "Measuring the Software Process" by William A. Florac and Anita Carleton.

You'll need to read the book to gain a thorough understanding of the theory - but I warn you, save it for a night you can't sleep!

Why can't we jump right to CMMI Maturity Level 5?

We've been told that an organisation should not go directly for CMMI level 5. Could you please confirm this and also tell us if any statement is available to substantiate the above?

Also, please clarify why we need to certify at Level 3 first and then target to Level 5.

Wow, we've got some over-achievers out there!

Like many things in the CMMI model, the advice you heard includes the words "should not" and not the words "can not." There is no policy from the SEI that explicitly says "you cannot go directly to ML 5," so, by definition, it is possible to do so.

Should you do it? That's a completely different discussion. I believe it's a mistake to do so because organizations can't change overnight, it takes time to absorb change, and you will often get a much better return on investment if you take a measured, gradual, and iterative approach to process improvement.

HOWEVER (and this is important) the SEI has declared that companies who pop-up as ML 5 for the first time that have never had a formal appraisal in the past will be subject to additional auditing and scrutiny. If they find that your appraiser should not have awarded you ML 5 then they will revoke it. This is not a problem if you really are performing at Level 5 . . . but there are numerous examples of this happening and the appraisal was, in fact, not valid.

Beware of the Lead Appraiser who recommends this type of approach. In my opionion someone that advises you to jump right to ML 5 is either un-ethical or, even worse, ignorant (this is by far the bigger problem). Achieving ML is (and should be) extremely difficult and while its effects can be very positive, it is a lengthy journey.

The answer to your ML3 question is the same. Not required, but highly recommended. ML3 provides you with a foundation for consistent process performance, something required to achieve ML 4 and ML5.

If I were working with your company I would want to know why this is important to you. If it's to save on appraisal costs, then this is not the solution. Appraisal costs are roughly 10% of your overall process improvement effort - hardly worth even worrying about.

Best of luck.

Monday, October 29, 2007

How can CMMI work for small projects?

We are currently CMMI Level 2 for our deliverable products to customers. However, we tend to work on many very small internal projects (1 person, less than one month delivery) to support testing of products. As we gear up for Level 3, the thought of having a gazillion artifacts to cover 18 process areas seems like overkill for these projects. Is there some point where an organization can say a project is too small for CMMI compliance, as long as there is some minimal process in place for this micro project (requirements, test, estimated schedule, ...)?

This is an excellent (and common!) question.

Let’s start with the concept of “tailoring.” Tailoring is introduced in ML3 within the OPD and IPM process areas. The net on tailoring is that each project should adapt the process (or “tailor it”) for their own specific situation.

As CMMI is an “organizational” model, leaving projects out of scope can be problematic. At the same time, regardless of the model you use, do you really want individual engineers working in an ad-hoc way? What if they come up with something that is really cool and can be used by the rest of the company? What if their one-person project has a high risk profile?

So, the answer is not in having them create a “gazillion” artifacts (technical term) but having them tailor the process so it makes sense for their project. How about 1/2 a gazillion?

A couple of other points:

There is nothing in the model that requires us to use a gazillion documents, and I hope you were not guided in that direction by an LA or other CMMI consultant. You owe it to your practitioners to evaluate the impact on them and reduce, consolidate, and lighten the documentation to a reasonable amount of weight. Have you considered alternatives such as digital cameras, white board cameras, cards, databases, et al to reduce the load? Sometimes a single work product can (and should) satisfy multiple practices in the model.

Just to clarify, there are 18 PA’s in ML3, but not all of them require projects to create work products. OPD and OPF, for instance, are organizational, not project focused. So are parts of IPM. Is SAM applicable? That may be another that does not require much documentation. Same applies for parts of MA, VAL, and VER (SG1 in each one).

And of course, many of the GP’s don’t require project artifacts either.

Best of luck to you!

Please help! Our Appraiser doesn't understand us!

Please help!

We are due to have our CMMI level 3 final appraisal soon & have run into a problem. We have had some gap analysis & employed a consultant to help us some time ago. They came up with a really good estimation process that helped our business no end over the past few months. At pre-appraisal the representative from the lead appraisers company tried to tell us that all CMMI estimations had to be done by size. We had already investiated this point with other CMMI consultants who had clearly told us that with CMMI v1.1 this was true, but with v1.2 that this was not the case; we could use size, but other methods (e.g. effort - as we do use) were perfectly acceptable for CMMI level 3, 4 & 5. With this knowledge we did not want to change a system that really worked for us. Knowing this we presented the evidence & discussed with the person from the lead appraisers company until he finally 'admitted' (off the record) that we were correct, but that he could not guarantee that another lead appraiser would be aware of this subtle change.

We took the attitude that if we were meeting the standard & that the method was giving a benefit to our company that it was fine to continue with it. Our lead appraiser is now saying that our ('non size') estimation method will mean that we will not pass CMMI level 3.

I have emailed the SEI who initially said that 'if the practice worked for us & did not adversely effect other areas' then it was acceptable. However they stopped short of saying that our appraisers information was out of date. We have emailed them again, but thay have said that they can not get into a dispute between an individual lead appraiser & a company. So basically:

1. Is it correct that estimation does not have to be based on 'size'
2. If this is the case then given we have a contract with the lead appraiser how can we get the SEI to get the lead appraiser educated correctly, as he & his company are too stubborn to admit that their information is out of date & that they are wrong on this point.

Boy, everybody has an opinion!

It sounds as if your LA is trying to be very a wee bit prescriptive about how you use “size” to conduct estimates. Sometimes SOME appraiser’s don’t know where their advice ends and their appraisal starts. If someone is telling you that anything in the CMMI “has to be done . . .” a certain way, I would view that as a reason to look for alternate appraiser. Who knows what else he is “interpreting” for you? The vast majority of LA's are reasonable, logical individuals - there are a few that are not.

There is a single goal in Project Planning for estimating: SG 1 “Establish Estimates.” This is the REQUIRED component. Now, in order to satisfy this goal, the model EXPECTS (read: doesn’t require) that you’ll determine what the scope is, what work products are going to be created and what their components are, and how much it’s going to cost (in time and money). If you don’t do those things, you must provide evidence of an alternative that gets you to a reliable estimate (and one that has “helped our business no end over the past few months” sounds pretty reliable). What you are describing could be an alternatve to "size" for estimating.

I assume your LA is referring to the second practice, SP1.2 “Establish Estimates of Work Product and Task Attributes.” If he were to read the informative material it goes on to say “Size is the primary input to many models . . . “ (notice it doesn’t say “all models.”). It goes on to say “ . . . can also be based on inputs such as connectivity, complexity, and structure.” This, as I read it, also means there are other methods, yet to be determined (or understood by the SEI), that one can come to an estimate. The options in the list are examples, not the only choices. Alternates are allowed if they satisfy the Goal.

You didn’t provide detail on HOW you were producing estimates, but, even if “size” were required, there are many elements of “size.” In the Agile world, time (manifested in Releases and Iterations) is a “size” attribute, as is number of features. Isn’t effort, if viewed as dollars, hours or amount of time, a calculation of size? I think it could be.

This type of “over-interpretation” of the model is a pet peeve of mine. I know of too many companies who have soured on the entire CMMI experience because there are a few appraiser’s out there with either no imagination, limited or no experience in software engineering, or have such a closed mind that they think you need to do it “their way” for it to be valid. Well I’m here to tell you . . . . there are many ways to establish estimates. This is too bad, because the CMMI can truly be a powerful and liberating model.

Bottom line on the LA is this: the SEI WON'T get in the middle (nor should they). LA's are "authorized" by the SEI, and therefore, we are permitted to deliver these appraisals on their behalf. This method works for the vast majority of appraisals . . .

So, what should you do? If you truly believe he won't/can't understand your business, it's time to face that fact and quickly disengage yourself from him - and find another one who will work WITH you to understand how you run your business.

Best of luck to you!

Saturday, October 6, 2007

What is the appropriate coverage for Quantitative Management?

Our consultants are heavily emphasizing the use of regression, ANOVA, etc. as an interpretation of what the CMMI expects with regards to quantitatively managing processes and projects. I don’t disagree that these are very valuable techniques to use when appropriate and when in line with your business objectives (per the CMMI). However, in an organization of 800+ engineers and hundreds of projects, I propose some sort of stratification to the project pool that characterizes projects with respect to size, priority, relevance, risk, alignment with business goals, etc. as a basis for the application of statistical management. To require all projects to use predictive models in order to satisfy QPM is a rather narrow interpretation from my perspective. This really drives organizations into the “checking the box” mentality, which ironically, is the opposite direction the SEI intends. I believe we will ultimately converge on solid ground because logic and rational thinking usually prevail when mutual understanding is achieved, but the SEI may be swinging the pendulum to far in an effort to strengthen appraisal validity in the high maturity areas. Your thoughts?

Sometimes my posts can be a little long-winded, and with your question being a long and complex one we could do the readers a favor by me just saying "I'm with you," but let me go a bit further.

You're correct in stating that the methods and techniques should be appropriate and tied directly to the goals and objectives of your business - so that's a good indicator that you're on the right track. That said, there are unlimited approaches you could take to capturing, analyzing, and reacting to the data and the methods you've mentioned are but a few. There really is no right answer. Anyone who tells you otherwise just doesn't get it at all.

The bigger, and more important part of your question seems to apply to what I call "coverage." I don't believe there is ANY guidance in the model, nor did the SEI intend for there to be, that instructs us to apply statistical techniques on ALL projects or even ALL process areas. On the contrary, OPP and QPM are ONLY meant to be applied when we have the data to indicate that there is even a "special cause" variation in the process. That's why it's so important to be thoughtful and diligent in selecting metrics when you are at the Level Two stage - it's THAT data that we're talking about that will be our higher level "noise detectors" that will lead us into performing OPP and then, QPM.

Performing the OPP and QPM processes on ALL project or ALL PA's invalidate their reason for being. Should you perform QPM on QPM?

I like to describe Level Two as a chainsaw, Level Three as a broadsword, and Level Four as a scalpel. Level Five is a laser, but usually the students are looking at me funny so I stop at Level Four.

Would you ask a surgeon to cut up firewood with a scalpel? Of course not. But you MAY ask him to perform analysis on a tiny fiber that looked suspicious to you as you were analysing the results of cutting up firewood.

And would he perform that analysis on EVERY log? No, of course not. Only on the one's that the data told him it makes sense.

So, neither coverage in terms of projects, or in terms of PA's is intended to be 100% - or even close to it. The percentage of coverage is based on the data - not on the need to meet the models requirements.

In this case, you should always be saying "the data made me to it!" If you have that to show, the SEI would be plenty happy.

Does the PPQA group have to be the 'Pain in th A$#' group?

Hi Jeff, nice blog!

I have a question about PPQA and PPQA of PPQA (GP2.9): how to implement these practices without creating a burden or overhead to the organization? I mean: how to make this group add value to the company instead of being the “pain in the a#$ group?"

Hey thanks! It's always nice when the readers don't hate the blog! Also, this is the first post I've had to censor for language. Fun stuff!

What a great question this is! You must have heard that PPQA stands for "Process Police and Quality A#$###*&. Well, contrary to popular myth this isn't really what PPQA is for. So let's start with my definition.

PPQA, or Process and Product Quality Assurance (and GP2.9 of any process area) exist to HELP us. How about that! It's like the old fashioned police whose job was "to protect and serve" but somehow has morphed into "profile and pull over." Of course, just like the real police, PPQA is so much more than what is percieved and joked about.

How did this metamorphasis happen? Well, let's go back to the beginning. Why do we even need PPQA? We need it to ensure that what we rolled out as a process is actually working, is helping projects, and is having a positive effect on the business. In order to do that, it must be determined whether or not people are even using the process to begin with. And that's where too many PPQA teams stop! They struggle so much with the compliance piece (mostly because people are not using it) that they're exhausted and never move on the the more important "providing insight" piece.

Providing insight back to the process owners (and SEPG) is the best way for us to learn whether we did a good job rolling out the process.

Whoa! Hold on a minute.

What was that you say? You mean we didn't do a perfect job designing and rolling out the process and now some PPQA puke wants to tell me something I already know! Darn PPQA zealots!

When I hear an executive or engineering manager whining about how people aren't following "my" process, my tingly senses start going off. So I tell them . . "hmm, guess you kind of screwed up on rolling out the process, eh bub?"

You see, if we did a great job designing the process, communicating the process, and educating about the process, people would probably use it. If we over-engineered it, designed it poorly, announced it with an email, and said "go forth and be process focused," then we did a poor job.

And only PPQA can tell us that.

So, start with a re-definition of PPQA's role. It's to provide independant insight back to the process owners as to how well the process is working. This is a direct reflection on the process owners and their ability to roll out processes effectivley. The SEI wisely realized that we drink our own kool-aid, the that can cloud our judgement.

Say that, and the project teams will look at you in a whole different way. Good luck!

Wednesday, September 26, 2007

How will we be evaluated for SAM for a ML2 Appraisal?

Can you tell me what as a company we will be measured on within SAM to achieve CMMI level 2?

The answer to your question has several parts.

First, is SAM applicable to your company?

SAM, or Supplier Agreement Management, is the one Process Area in the CMMI that can be considered "not applicable" by your Lead Appraiser and Appraisal Team. SAM is usually applied if you engage an external provider to perform a service or build a product that becomes part of your company's product or service offering. Some appraisers have said they include SAM if ANY product or service is purchased by your company - I disagree. Should you perform SAM for buying pencils, pizza, and toilet paper? I don't think that was the intent.

Second, the model itself outlines the Specific Goals that are required, and the Specific Practices that are expected to be performed to achieve them them:

SG 1 Establish Supplier Agreements
SP 1.1 Determine Acquisition Type
SP 1.2 Select Suppliers
SP 1.3 Establish Supplier Agreements

SG 2 Satisfy Supplier Agreements
SP 2.1 Execute the Supplier Agreement
SP 2.2 Monitor Selected Supplier Processes
SP 2.3 Evaluate Selected Supplier Work Products
SP 2.4 Accept the Acquired
Remember, the SG's are required. The SP's are only expected.

Finally there are the Generic Practices. If your attempting to achieve CMMI Maturity Level Two you will have satisfy GG2 and all of the GPs GP2.1 - GP2.10.

GP 2.1 Establish an Organizational Policy
GP 2.2 Plan the Process
GP 2.3 Provide Resources
GP 2.4 Assign Responsibility
GP 2.5 Train People
GP 2.6 Manage Configurations
GP 2.7 Identify and Involve Relevant Stakeholders
GP 2.8 Monitor and Control the Process
GP 2.9 Objectively Evaluate Adherence
GP 2.10 Review Status with Higher Level Management

More detail is available in the CMMI model itself.

Good luck!

Tuesday, September 18, 2007

How many process can be in a project?

How many processes can be in a project according to the Integrated Project Management Process Area of CMMI?

A lot. Or a little.

How should we 'Elicit Customer Needs?'

The CMMI asks us to "elicit needs." Can you give some examples of how to perform this practice?

Sometimes customers believe they are giving you their "requirements" when if fact they are presenting you with a list of needs. A requirement is a specific thing (clear, traceable, testable, etc....) and often the "requirements spec" we received has few of these things. RD.SP1.1 starts with "Elicit Needs" and asks us to perform a process to identify the needs from which the requirements are developed.

There are as many ways to "elicit" a customer need as there are companies. Each one I've worked with has their own approach and style, but there are some similar threads out there that I can offer as examples.

- Facilitated sessions with the customers and engineering/requirements folks where the needs are mapped out and debated using a simple white board approach. Make sure to bring your camera! And yes, photos can be placed under configuration control (and can "count" towards evidence, if managed and used appropriately).

- Brain storming sessions that are structured using a mind-mapping" approach and a tool like MindJet's MindManager (a personal favorite). Assign different "branches" of the map to appropriate stakeholders in breakout sessions and then have an integration party at the end! It helps to have someone adept at the tool "play it" in real-time as people are debating.- Graphical approaches using white boards, swim lanes, or graphical tools like iRise (a great tool for this by the way). Different colored pens and someone who can draw helps here - the high-end consultancies use real artists for this (as opposed to those of us who are not!)

- Prioritization workshops where the customer relates to the engineering staff what is most important to them. What can they live without? This subtractive approach is good for reducing scope.

- A detailed requirements spec sent from customer to supplier in a bidding process. This is very common in the manufacturing world, and is almost always incomplete. But, it's the way of the world for thousands of engineers. The thing to remember here is, your work is not done just because they send you a 400 page document - it may not have a single "need" in it!

- In an agile setting, prototypes, sample screens and reports, or "Spikes," an early working model of the most important features requested, so the customer can agree on what they want. This method is iterative of course, because the customer rarely knows how to articulate what they really want.All of these, and others, can (and should) be iterative. In other words, it's more than one session, or spike, and is a back and forth negotiation with the client.

Interpreted literally, the CMMI tells us to "elicit needs" (as if you can just go off and do some elicitin') but remember that the actual effort could be days, weeks, or months in duration, and could happen repeatedly throughout the lifecycle of an overall program - resulting in many releases of the system.

Have fun!

Wednesday, September 12, 2007

Which version of CMMI should we be using?

I'm confused about the different versions of the CMMI. Which one should we be using right now?

I understand your confusion. In version 1.1, there were multiple "disciplines" within the v1.1 framework. Software, Systems Engineering, and Supplier Sourcing were all available. When v1.2 was released in August of 2006, you could choose v1.1 and an accompanying discipline OR you could choose CMMI - DEV v1.2. As of August 2007, CMMI-DEV v1.2 is the only version of the CMMI that is active and supported. The concept of disciplines has been eliminated, and the "DEV" refers to the only "constellation" that is available ("Development"). Other "constellations" are planned for the future (Acquisition, Services) but have not been released.

So the answer is simple - there is only one choice right now, CMMI v1.2-DEV.

Thursday, September 6, 2007

What is the best tool for implementing CMMI?

We need to implement CMMI. What is the best tool to do this?

Excuse me while I take a tool to my head. . . oh, sorry, I thought I was in the wood shop!

By far the best tool to use for CMMI is the one between your ears.

Applying logic and sound business judgment in selecting your approach, who should be involved, and how success is defined will be the single most important “tool” in your arsenal.

I worked with a client once who spent millions on a “tool” to “do CMMI” but they gave a blank stare when asked about their business goals and objectives.

That said, you'll likely end up with more than one new toy to play with if you decide to go agead and buy something.

Tuesday, September 4, 2007

I don't want to bore you but, how do we start with CMMI?

I don't want to be boring so I'll just get to the point. We are going for CMMI Level 2. We don't have any processes or documents for the different process areas. My question is "how do we start with CMMI?"

zzzzzzzzzzzzzzzzzzz. Just Kidding!

The great thing about process improvement is that, at any given time, somewhere in the world, there is someone just starting out. There is no such thing as a boring question (well maybe "my boss says to get to level 5 in six months, is this possible?" is a boring question)!

Where to start? Well the CMMI reference itself is a great place to start out. If you're going to try to organize yourself around Level Two then read, in section 2, the seven process areas that are contained in Level 2 (REQM, PP, PMC, PPQA, MA, CM, and SAM).

Remember that the goals are required, the practices are expected, and everything else is information.

Next, do an HONEST self-evaluation of each practice based on a set of typical projects in your company. The sub-practices and informative text in each practice will help you to understand the context of each practice, but remember that these are only examples.

Once you know where you are as it relates to the model, put together a plan to close the gaps, and at the end of each phase have an external CMMI Appraiser stop in and conduct an informal evaluation and help you refine your plan to achieve.

There are also several great books by my friend and colleague James Persse and another by Michael West that will help. Also visit our resources page at for some downloads and visit Hillel's "Brutally Honest Totally Hip CMMI FAQ" at

Finally, there are a wealth of documents on the SEI's site at

Good luck!

How long do we have to wait to be re-appraised?

He long is the waiting period to have a re-appraisal?

If you're asking how long a successful appraisal is "good for" the answer is three years. If you've had an unsuccessful appraisal then the answer is less clear. The SEI does not provide any formal guidelines for this, as every organization is different.

It really comes down to the depth of the weaknesses identified during your appraisal, and whether or not you can credibly make the corrections, build up some maturity, and show results on a reasonable amount of projects.

In some companies this could be six months - in others it could be six years. It just depends on your situation.

The bottom line is that there is no formal requirement from the SEI on this but you will need to ensure that your Lead Appraiser is comfortable with your progress from the last appraisal before you conduct a second one.

What if we don't perform DAR on a project? Should we still fill out the forms?

Process areas like DAR etc... may not be used every time in all project circumstances, although informally some of it may happen. Does it mean we use the standard DAR template to formalize these exercises ?

Hmmmm, should you fill out a form only for the purpose of "satisfying" CMMI even though it may not be needed? My good friend and colleague Hillel might say this type of behavior "enraged him" in response to this non-value added activity. I'll stop short of being enraged and settle on perplexed. Why add the overhead?

I would never ask you to fill out a form just for the CMMI. There are projects that just don't perform any DAR practices, nor do they have a need to. If that is the case then DAR evidence just wouldn't be available for that project. Of course, it's not a free pass to ignore DAR either. Someone with appropriate authority should want to know WHY DAR wasn't needed, and a process for tailoring would need to be demonstrated, but that's another subject.

Bottom line? Don't do things "just for the CMMI." Do things that make sense for you and your business - and have an appropriate process for tailoring.

Tuesday, August 7, 2007

What is a Software Factory?

Can you describe what a "software factory" is?

The “Software Factory” is something that is common with larger, multi-departmental organizations where IT is serving a broad array of constituents. It may exist in any type of environment, but this is the most common scenario I’ve seen.

Factories are often focused around the development of foundational technologies – objects, web services, or modules that serve multiple applications and can be reused by the various constituent groups. One very interesting model segments the IT development organization into two segments; a “factory” that develops and certifies these foundational modules, and “customer facing” developers that use the components, as well as additional html/java/php etc. to assemble end-user applications.

In this type of model, the factory has little or no customer contact, but serves the other segments of the IT organization. There are also examples of the Software Factory within the big consulting firms – the earliest of which was Ernst & Young’s “Advanced Development Centers” that used a variety of acceleration techniques to serve clients directly. Accenture has followed up with a similar concept. I’m not sure these a true “factories” as they develop custom software that are almost all one-offs, but they use that expression nevertheless.

In my view, the value of a factory is their ability to focus on churning out a set of proven, well developed components that benefit the rest of the organization. You’re experience may be different, but that’s what I’ve seen.

Some REQM Practices seem complex if done properly. Can you provide some practicle examples?

Can you explain following two REQM practices with some practical examples?

- Identify inconsistencies between the project plans and work products and the requirements

- Establish and maintain an organizational policy for planning and performing the requirements management process

Sure! Practical examples is what this 'blog is all about.

I'll start with the last first. Your question appears to be about REQM.GP2.1 "Establish an Organizational Policy." Let's start with a question: "what's an organizational policy?" According to the CMMI, an OP is: "A guiding principle typically established by senior management that is adopted by an organization to influence and determine decisions." To put it more simply, it's management's expectations for performing a process.

My advice to you here is to keep it simple and not use your policy to describe the process. You will probably be going through an exercise to define a Requirements Management process, so there is no need to be redundant within the policy. A simple documented policy that sets management's expectations is sufficient.

"All employees are required to read, understand, and follow the REQM process as defined in the process guide."

Of course, there needs to be a process guide for this to be valid, and a valid REQM process as well to point to.

The second part of the practice refers to "maintaining" the policy. This implies a cyclical review of all policies, and updating or changing it as appropriate (keeping it all under CM of course!)

You first question refers to REQM SP1.5 Identify Inconsistencies Between Project Work and Requirements. What could this possibly mean, and whose job is it?

To begin with, we can think about this practice as part of PMC because it is clearly a project management activity. It involves keeping up to date on all of the latest requirements changes, and ensuring that the tasks being performed by the project team reflects the latest approved requirements set.

Imagine a large scale project with dozens of developers who are each assigned a set of requirements or features to implement. They may not be involved in all of the meetings and approvals for new or changed requirements. How does the PM ensure that, as requirements change, the developers are working on the latest and greatest requirements set? How do we avoid having them work on the old requirements, or when the requirements have changed to something different?

The place to manage this, IMHO, is in the traceability matrix. This tool can quickly identify who is working on what code and gives the PM a way to rapidly determine what work needs to be halted, modified, or added.

Thursday, July 19, 2007

Could you imagine a CMMI-compatible ERP?

Can you imagine a CMMI-compatible ERPs? It would be, indeed, a kind of “Big Brother” system.

I think you answered the question already. I could imagine it, but I sure wouldn't recommend it!

If I understand your question, your asking if a system that globally controls practitioner's work so that they were "CMMI-Compatible" would be a good idea.

There are several products on the market the purport to deliver CMMI compliance if only the developers would just fill out the forms and use the system. These products try to tightly control the environment so that developers CAN'T go any other route. Believe it or not, Microsoft claims to have such a system (don't get me started . . .). Compuware has one also.

I guess I would have to say that anyone who advocates this type of approach may understand workflow and military efficiency, but probably doesn't appreciate the creative process used to develop software. As I've said many times, software development is not linear and cannot follow a linear process succesfully. The "big brother" approach defeats the purpose of improving process performance and probably won't result in accelerated development, reduced defects, and better retention of employees- the three biggest reasons I believe the CMMI is valuable.

The CMMI is not about filling out forms - it's about increasing productivity through the elimination of non-value added work, improving project visibility, and using fact-based data to make sound business decisions. This is something that I appreciate more and more the older and more experienced I get.

What is the difference between RD and REQM?

What is the difference between REQM and Requirements Development? What are some of the the leading tools for capturing requirements?

Requirements Management, or "REQM," is all about maintaining the set of requirements that you have, and the process of accepting new ones. In the CMMI world, that includes understanding them, committing to them, managing changes to them, maintaining the appropriate degree of traceability, and understanding how they relate to the actual work being performed by the team.

Requirement Development, or "RD," is about the transformation of customer needs into requirements that can then evolve into a design and/or code. This includes eliciting the customer needs (JAD sessions, interviews, et al), transforming those needs into requirements, evolving them into product requirements, allocating the requirements across releases, teams, developers, or modules, validating them, and ensuring that they fit within the customer constraints and assumptions.

I am not aware of one single tool that does all of this, but parts can be supported by Borland's CaliberRM, IBM's ReqPro, or Doors. My preference is CaliberRM, but they all pretty much do the same thing. They are also all parts of a larger "development framework" that these companies market.

Tuesday, July 17, 2007

Does a company that is certified in SigSigma need to get certified in CMMI?

Why do top IT companies with six sigma certification need CMMI certification???? how will it help them? In case they refuse this certification will it affect its market reputation?

No company "needs' either one - but they are both helpful for benchmarking and measuring performance.

Six Sigma and the CMMI are completely different (although often confused) animals. Companies seeking Sig Sigma focus on defects and corrective action, with the hope of being able to correct the cause of the defect and eliminate it in future releases. The focus is largely on metrics.

CMMI is a process model intended to guide organizations through a cycle that improves productivity and performance, eliminates waste, reduces defects, and gives management much better visibility into project performance. The CMMI's focus is both process and metrics.

There is no real conflict between the two except when you realize that in order to predict defects and take true corrective action you must be performing and CMMI Levels 4/5. Many companies attempt Six Sigma programs long before they are mature enough, often leading to failure and wasted effort. Without the standard processes implied by the CMMI, it would be pretty tough to identify the process cause of a defect.

That said, neither has a “certification” per say. A company “achieves” CMMI Level 2/3/4/5 and “achieves” 3/4/5/6 Sigma.

In general, I think the CMMI Appraisal is a much stronger vehicle for evaluating the capabilities of a company, as it is a “best practices” model that used external appraisers to conduct the assessment.

CMMI is verifiable (by the SEI), whereas Six Sigma really isn’t. Any company could claim to have achieved SixSigma, with CMMI you are "awarded" a "rating" by a Lead Appraiser, and you can verify it on the SEI's web site.

As far as market position, I think it depends on the industry. In defense and manufacturing both SixSigma and CMMI are common, but no one is requiring companies to adopt SixSigma that I know of. The same is not true for CMMI. Many OEM's and defense agencies (as well as electonics, health care, and insurance) are requiring that their suppliers achieve CMMI Level Two as a minimum.

If you're in one of those categories, or you want to sell to one of them, the CMMI might be your best route.

Wednesday, July 11, 2007

Should SAM be in our appraisal scope?

Do organizations that develop software applications over SAP or Oracle Applications and that deliver no products or products components to their customers require the SAM Process Area to be in appraisal scope?

As you may know, SAM is the only PA allowed to be “out of scope” for a CMMI Appraisal. However, it's not automatic. The exclusion of SAM is something that will need to be understood, negotiated, and agreed to between the appraisal sponsor and the Lead Appraiser.

In general, I am comfortable excluding SAM if the appraised organization does not purchase key products or services from a supplier that ends up being integrated into your product. In other words, if you hire a company to write a device driver for your handheld inventory device that you use or market, then SAM would be applicable. On the other hand, if you purchase an off-the-shelf “c” library that offers a spell check routine, and you got it from CDW for 29.95, I would not apply SAM. SAM could also apply to internal suppliers, like an operations department, if a service-level agreement exists and they do indeed need to integrated into your product.

Another test I would apply is, is it possible to apply the SAM practices to a particular vendor? For instance, SAP probably wouldn’t comply with your request to “monitor selected supplier processes.”

Should we receive a "Partially Implemented" if we don't use historical data for estimates?

The majority of my appraisal team wanted to rate PP SP 1.4 “Partially Implemented” because they felt we were missing one critical element: how we took actuals from the prior year and translated those into plans for the current year (e.g., with a multiplier, due to complexity differences and/or team experience differences). I felt that the absence of that piece of written data was not sufficient to prevent us from attaining the larger Goal. What do you think?”

In general, I’m reluctant to require any prescriptive method as the only way to satisfy a practice. In other words, a project can very easily estimate properly without specifically taking actual estimates from the prior year and using them with a multiplier as you have described. I agree with your assessment, the absence of a specific piece of written data should not stop you from achieving a goal as long as you are achieving it in some other way.

The suggested work products in the book are just that, suggestions. There's no need to take them literally.

Sunday, June 10, 2007

What can be evidence for "Estimating Rationale" in Project Planning?

What can be documented evidence for "estimation rationale?"

Hey, I asked a Sr. Engineer for an estimate and he told me. Isn't that rational?

It may be rational, but it isn't evidence of estimating "rationale." That darn "e!" changes everything. I hate that!!

Estimation rationale is a way to explain how you and your team came up with an estimate. Did you use any specific estimating process? If so, what were its outputs? Did you count reports, screens, features, functions points, or lines of code? If so, what was the output of that exercise? If you’re in an “agile” world, did you estimate a number of releases and iterations and allocate the high-priority features you planned on delivering during each one?

I always ask myself “why does the model want to know this?” when I’m stuck trying to figure out why a practice was put into the CMMI. In this case, they seem to have been looking for evidence that an engineer didn’t just “guess” at what the estimate would be – but went through some process to validate it. This is process is the“rational.”

So, how did you validate your last estimate? The output of that work is your evidence. Or . . . did you just guess?

Tuesday, May 22, 2007

Can CMMI apply to a Document Production company?

Hi Jeff,
My name is Renuka from Bangalore India, working as a Quality Executive. I have been in the domain for more than 2+ years and have actively participated in CMMI L3 implementation mainly for the Process area SAM.
Current I'm with a company whose main business is Technical communication for Major development companies in bangalore. We have a tailored QMS in place for the document development lifecyle model. Is it possible that a Document development company could be appraised for CMMI? The QA team here also is an internal development team, which supports for software development eg: online applications internally (who would like be be a revenue generating team instead of only support function).
We have tailored the SDLC to DDLC & use the QMS for the process compliance.

Interesting application of the model! Does your company produce a product? Does it take requirements in, manage them, have a plan, measure performance, design and build, and verify? If so, then I see no reason why you couldn't successfully use the CMMI to benchmark your performance.

Two hints I would give you though. First, spend some time with your selected Lead Appraiser to make sure he/she REALLY understands your business. If you select one with only a systems background he/she MAY not be able to logically make the transition to apply the model to your business. Secondly, take a look at the Continuous Representation of the model. You may discover that it is more appropriate for your situation. Just a thought!

How do we measure CMMI Compliance?

Our Company is growing organization having 500+ software developers and test engineers in the embedded and ASIC division. I am head of SEPG department and implement CMMi level 3. Currently, we are doing self appraisal.
I am looking for standard compliance matrix for CMMi level 2 and 3 process areas for my organization. If you do not mind, then kindly please send me the template for measuring CMMi Compliance Index. Also, please share the document / templates for AgileCMMi for all CMMi level 3 process areas.

Congratulations on performing your self appraisal. Let me ask you this, what are you using to "self-appraise?" This may be the only tool you need. There is no "standard compliance matrix" unless you consider the practices in the model to be one. The way to measure CMMI Compliance is to engage with an authorized SEI SCAMPI Lead Appraiser (either in your area or one that will travel to you) to conduct an objective SCAMPI Appraisal. SCAMPI is the method we use to evaluate CMMI performance and it does not include an standard forms or matrices - although it is a robust methodology.

As a SCAMPI Lead Appraiser, it's my job to evaluate process performance across your organization against the required components of the model (the Goals) and the expected components (the Practices). For a self assessment you could do the same by opening the book and going through it practice by practice Have fun!

Is there a difference between Risks and Risk Sources?

Hello Jeff,
Thank you for your blog which is very instructive and useful for all of us.
I’m part of a quality team as a trainee in an IT company in France, and working on a risk management tool.
After having studied risk sources and risks categories in both CMMI - RSKM and SEI risk taxonomy, I found that most of the time, a given risk may also be a source of risk.
Lack of Human resources may be a risk, but also a source for the risk “Product not delivered on time”;
The element “Uncertain requirements” is present in both RSKM risks sources and risk taxonomy categories.
My question is: "is there a distinction between risks and sources possible, and is it useful to build a risk management tool or plan?"

First of all, let me congratulate you on your obvious level of knowledge about Risks, as well as your knowledge of the CMMI model. Excellent for a trainee! I'll be looking forward to YOUR Blog in the future!

Second, let's step back for a second and try to view the CMMI as a set of guidelines, and the examples in the model as a set of suggestions. In doing that, we see that identifying risk sources is a neat way to ensure that we capture most of the important risks, by providing us with "memory joggers" to help us identify the important ones. You have done so with great clarity.

As to a distinction between risks and sources, you're right! Some risks and sources are related, and some risks can become sources themselves. Should you treat these things differently? I would say "no" and here's why. It's very difficult to get people to adopt a process of any kind, primarily due to the culture change. Add to that the complexity of tailoring guidelines (which force the Project Manager to interpret the process and make decisions about what is appropriate) and it becomes, in some cases, nearly impossible.

As I always am conscious to keep my "Agile" hat on (I believe "Agile" is a philosophy as well as a family of methods) I am always looking for simplicity and an opportunity to streamline as much as possible. In the spirit of this philosophy, I would argue that it would be better for the organization as a whole to deploy a simple set of "risk sources" and not muddy the waters with the maddening and circular discussion that would inevitably take place once you introduce the "source or risk?" conversation. Believe me, I've been tortured through many a similar discussion.
That's what I think anyway. Others may (and probably will!) disagree.

Monday, May 14, 2007

Dear Jeff

I’d like to ask what CANNOT be tailored out from a CMMI perspective. For example, if a tailoring request will result in not meeting an SG or GG in one of the projects, should the request be approved?
I understand that we should determine which parts of our processes must be done, but, from an appraisal viewpoint, what would be considered improper tailoring?

Great question. I think you’ve answered part of it already.
If the tailoring results in not meeting and SG or a GG then it shouldn’t be approved (assuming CMMI compliance is one of your goals of course). But the story doesn’t end there. Remember that the Goals are required, but the practices (SPs and GPs) OR AN ALTERNATIVE are expected. By expected the CMMI means that they need to be satisfied in order to satisfy the goals. I’d be hard pressed to come up with an example of a goal that didn’t require the SPs to actually be satisfied (although there are alternatives – especially in the Agile world).
For instance, it would be pretty tough to “Manage Requirements” (REQM.SG1) without understanding them, obtaining commitment, managing traceability, etc… You get the picture.

One way to handle this is to identify all of the “must have” work products and processes, and then the “should have” work products and processes. Like the practices, the “should” really are “must” unless the project has a good reason for tailoring it out – and has some alternative for it.