Monday, May 26, 2008

If we fail a SCAMPI Appraisal when can we try again?

If we were to fail our SCAMPI A Appraisal, when can we try to be appraised again? Who decides, the Lead Appraiser?

Great question! Even though there is technically no "failing" an appraisal, I know what you mean. If there are weaknesses discovered, you can indeed not achieve CMMI MLx.

If there are gaps in the Specific or Generic Goals, it is up to your organization to fix them if you wish to be successful with an appraisal. If it's something simple like, GP2.1 Establish and Maintain a Policy for performing the (x) process, then it might be possible to re-schedule an appraisal quickly. If it's something more complex like GP2.8 Monitor and Control the Process, then the cycle time it would take you to perform this process and demonstrate it through objective evidence could be months, if not years.

The Lead Appraiser is a relevant stakeholder in the decision to re-appraise, but he is not, as our President claims to be, the "Decider." It's really up to your sponsor and your organization.

If you truly believe you are ready to be re-appraised, based on the evidence, then you should go ahead and re-schedule it. If your LA does not agree, that is one good data point in the decision making process, but it is far from the only one.

If your LA is the only one saying you can't do it, and you strongly believe you have the evidence to prove adherance to the model, then you might consider a new LA. That said, they are experts so take this type of decision with care.

There is no "rule" about a waiting period after an unsuccessful appraisal - it's up to you to decide. Of course, failing a second time can be very painful.

Can we go to ML5 if we're an Agile organization?

Our company uses Agile methods and we want to go all the way to ML5. Is that possible? How many companies have done this?

The methodologies you use, assuming you actually use them appropriately, have little effect on your ability to achieve ML5.

Many have said that the CMMI is "waterfall" or that it can't apply to "agile." This just isn't true. I worked with a company recently that told me that since they were Agile they couldn't achieve CMMI ML5 because they used "agile metrics" and the model required "waterfall metrics" like CPI and SPI.

Really? Where does it say that? As a matter of fact, collecting CPI and SPI, or any other measure around time or dollars at the project level is just dumb in an agile world that has fixed budgets and fixed time-boxes.

I believe that agile is better-suited for ML5 than more traditional methods - if only because we can re-factor to focus on the high-priority items - a basic tenant of ML5 and Agile.

It's tough to know how many companies have done this - the SEI doesn't tabulate methodologies used during a SCAMPI, but I work with one ML4 company that is agile, and I can assure you that it works well.

There is a new SEI Technical Report on this subject which I was lucky enough to co-author. It is being release this month - look for it on the SEI's web site.

Monday, May 5, 2008

We have a lot of tools. Should these be managed as Process Assets or as software engineering tools?

My organization is CMMI ML2 and preparing for the ML3 appraisal. We have SEVERAL tools (DOORS, Change/Synergy, PVCS, SLIM, and a host of custom-built apps) we use to enable and enforce our software engineering process.

Should these tools be considered process assets and part of our PAL or resources for performing the processes? If so, is there an advantage to them being managed by our process board, or being managed as normal software development? Also, would improvements/changes to these products be considered process improvements (executed under our OPD/OPF implementation)?


Only kidding, it's a great question!

While the CMMI itself does not directly speak to this question, it does give us examples of things that should be in the PAL, or things that should be "managed" in general. That list does include tools such as compilers, databases, requirements management tools and the like. So the theoretical answer to your query is, yes, they are process assets that should be in your PAL, and you're PAL is managed by your process group.


I'm not one to be too concerned with the theoretical - especially if it's only an example list created by a bunch of smart folks at a university. From a more practical perspective, that is still in-line with the spirit of the CMMI, it would be quite difficult, if not impossible, for your process group (SEPG perhaps?) to "manage" them when most of these tools need genuine IT support, require budgets, licensing, purchasing, and maintenance. The SEPG, or other process groups, are ill-suited to perform all of THESE processes, as well as having their hands full managing the more "typical" process assets they already manage.

One approach would be to use our friend SAM to manage those who manage your tools, and let them provide them as a service with service level agreements.

On your other question, improvements made to these tools, or new tools to support processes, could, and probably would, be considered "improvements" executed under both OPF/OPD and within the context of GP3.2 of ALL of the ML2 and ML3 PAs.

How can we successfully scale up and outsource work to skilled development houses?

We want to outsource work to skilled development houses but incur a minimum upfront cost in doing so. Do you have any advice, resources or links that would help me investigate how to make this possible?

I can’t offer any honest advice on outsourcing – mostly because I’m against the concept in general, but also because I’m no expert on the subject.

But I actually had a client once who had similar goals. Here's what I told them (and what ended up working).

Step back and look at your company from a process perspective. Where is the real value of the company? Where is your Intellectual Property? What do you do better than anyone?

I would recommend that you identify these high-value core processes within your own company, and try to resist the temptation to outsource them - even if they are engineering related.

In general, at least here in the US, outsourcing has been a financial success, but a business failure. Why is this? Because we (and I assume most others) are terrible at eliciting end-user needs, converting them to product requirements, and then translating them so the average software engineer can understand them. I say “average” because, as you scale, that is what you end up with. The quality of service you may be used to quickly degrades as organizations scale up, up become’s unmanageable as outsourcing is mixed in.

The reason it has been a financial success is, as one CIO I know put it, if “I’m gonna get crap, it might as well be cheap.” This is a dismal assessment of our industry for sure, but it’s not that far from the truth.

One way to combat this problem, and prepare for successful growth, is to use something like the CMMI to focus in on the high-value processes you will be maintaining, and also focusing as much effort on the process interfaces. Ensure that your outsourcer is able to interface with you at a similar maturity (or capability) level. For instance, if you master the art of Requirement’s elicitation and development, be sure your outsourcer’s Verification / Validation processes can respond in kind. There are a lot of disaster story’s on that subject.

If it were me I would retain processes related to Requirement Development, Requirement Management, Verification, Validation, Product Integration, Process Quality (PPQA), Project Management, and Supplier Management. Let the outsourcer take the processes associated with design and development, Configuration Management, and engineering Measurement. You should, in either case, own the architecture of the software, for this is where the real crown jewels are located.

Just mastering the process of translating business needs into technology instructions (which will become much more difficult when engineer's are no longer onsite) will pay for the cost of your up-scaled development organization.

Good luck!