Tuesday, September 30, 2008

Isn't QPM just really great project management?


We're ML3 and we believe we have world-class project management capabilities.  We do all of the things in PP, PMC, RSKM, SAM, and IPM really well and we must be performing "at the next level."  Doesn't that just make us ML4?

Hmmmm.  uh .. .hem.  Let's take a look.

You're right in that being ML4 requires you to have a solid foundation with proven performance from the ML2 and ML3 process areas - so you get points for that.  Without that you're not even in the game.

But QPM is different than "advanced project management."  It's not necessarily "better" project management (although I would argue that it would make managing projects "better") because "better" is subjective.  But it is different.

It's different in that it depends upon the use of statistical techniques, as well as the presence of process-performance baselines and models, to be useful.

SG1: Manage the Project Quantitatively

The word "quantitatively" is a special word in the CMMI that infers the use of statistical methods.  The practices supporting this goal all depend on the use of these methods to succeed.

SP1.1 Establish the project's objectives

These objectives are for quality and for process performance.  They need to be based on what can actually be accomplished, and that information is only available if you're performing OPP successfully. In the case of a "mandate" from management (a quality "specification") you ALSO will need OPP to identify the risks associated with the mandate, and the corrective actions that will need to be taken to achieve it.

SP1.2 Compose the defined process

Using the data from OPP as your guide (baselines and models), and within the context of the objectives from SP1.1, select the appropriate sub-processes from the set of standard processes (OPD) that will enable you to achieve your objective.  This is a little like a more granular, data-focused way of performing IPM SP1.1 and SP1.2.

SP1.3 Select sub-processes that will be statistically managed

Which sub-processes will you need to manage / monitor in order to understand if you are going to achieve your objectives?  Again, OPP can help here.

SP1.4 Manage Project Performance

Using the aforementioned monitoring, manage the performance of the project, taking corrective action as needed.  Why do I have data points outside of my control limits ("assignable causes")? Time to find out (you can use CAR for this).

SG2: Statistically manage sub-process performance

Use statistical methods to understand variation and take corrective action when necessary. Some of these practices will probably be performaned along with SG1 (above).

SP2.1: Select measures and analytic techniques

What measures am I going to use to steer the ship, and what techniques (e.g.; process performance charts, histograms, XmR charts, etc) am I going to use to understand variation?

SP2.2 Use statistical methods to understand variation

Use the techniques and methods to identfy assignable causes of variation (e.g.; outside of control limits).  

SP2.3 Monitor performance of selected subprocesses

Understand how the various sub-processes you have selected are performing, so you can understand progress towards achieving the objectives set in SG1.

SP2.4  Record statistical management data

We need to update the baselines created by the OPP capability with actual process-performance data - and this is where we do that.  This way the next poor schmuck that comes along will learn from MY screwups.

Whew!  Well, there is a lot here, and it's way different than PP/PMC.  Is it worth doing?  

Well, let me ask you this.  Do you want to identify and eliminate defects earlier?  Do you want the next project to be better than the current one, and the next one after that even better?  Do you want to spend less time on drudgery and more time on engineering?


I'm not sure I understand OPP - what's it all about?

Hey Jeff,

We're a ML3 company and we'd like to move up to ML4.  There is a lot of argument about the meaning of OPP.  Could you shed some light on this?

Absolutely!  Almost all organizations struggle with the four "High Maturity" Process Areas, and Organizational Process Performance, or OPP, is the one they struggle with the most.

It's impossible to claim you are performing at "high maturity" without OPP.  It's a foundational process area that provides an infrastructure that allows you to use the other HM process areas (as well as perform better with the ML2/3 PAs).  It exists to establish and maintain (sound familiar?) the basic statistical data you need to continuously improve your projects and your organization.

SG1: Establish Performance Baselines and Models

The practices that lead to achieving this goal are practices that support the selection of processes to statistically monitor, establishing organizational objectives, establishing measures to use, creating baselines of process performance, and creating process performance"models" to help predict the outcome of a set of processes being performed (and to assist projects in the selection of sub-processes from the organizations standard set of processes - QPM)

So, let's say that our goal is for all projects to have a customer satisfaction rating of 10.  Can we achieve it?  What does history tell us?  Is that realistic?  What were the causes when project's did NOT achieve this?  And what might we do with our projects to achieve this goal specification?

These are the questions that OPP is designed to support.

SP1.1 Select Processes

Exactly which sub-processes (from the standard set of processes) will we be including in our analysis effort?  You often see process elements related to Peer Reviews, Defect discovery, engagement (TeamScore) and the like.

SP1.2 Establish Process Performance Measures

Which process attributes are we going to measure?  If you were to select engagement, you might measure actual participation vs. planned participation.  For peer reviews you might measure preparation time for each peer review.  These are measures of some attributes of the sub-process.

SP1.3 Establish quality and process performance objectives

These objectives come from various sources.  Some come from the business ("we need to increase sales by 5%) or from the engineering process itself (reduce defects by 12%).  If we have enough data to establish an analysis of the process performance variation, and we can calculate the natural bounds of process performance, then this will have to be our objective for the process - as it's all the process is capable of delivering ("Voice of the Process")!  In other words, the process is telling you"STOP! You can't reach this objective!"

SP1.4 Establish Process Performance Baselines

These baselines are the data that represent actual process performance (and it's variation).  We typically will plot these data using some type of process performance (control) chart or histogram.  We are trying to establish what the sub-process is capable of - these natural bounds are called the "voice of the process" and they establish that, if the process remains the same, here is what will likely occur (within limits).  

So, using engagement as an example, we can measure customer engagement's effect on customer satisfaction by measuring their engagement at different points in the process, and comparing that to historical customer satisfaction results.

SP1.5 Establish Process Performance Models

Models are used to estimate the value of a process performance measure (the "result") if a given set of sub-processes is performed.  

There are many techniques available for this - simulation for instance - but in the end this is a "what if" analysis using the historical data to estimate an outcome with a high-degree of probability.  This is useful for projects as they attempt to predict the outcome, as opposed to just guessing.

So what?

You've probably noticed that there is nothing in OPP that actually says "fix the problems."  
That's OK - there are other process areas for this (CAR and OID).  

OPP is a "capability."  It gives us the data we need to make better decisions about the use of the process - and it tells us exactly what we're capable of.

Whether we like it or not!

Sunday, September 28, 2008

Lot's of people are moaning about the SEI's new interpetation of High Maturity - what does it all mean?

There's a lot of "noise" being made about the SEI's "High Road" view on level 4 and 5, although the model itself hasn't changed much for these 4 PA's.  My company, a past Level 5 achiever is ready for a CMMI ML5 renewal and there's lots of moaning and groaning and questions about this "High Road".What's really the new emphasis (or activities or requirements) that the SEI is imposing at Levels 4 and 5?

This is an AWESOME question.  Thank you!  There certainly has been quite a bit of conversation about this in the past two years, and while the practices have not changed all that much, the informative material has, and that material forms the basis for understanding the meaning of the practices.  So, let's explore it.
The short answer is, too many organizations (along with their Lead Appraisers) were achieving ML4/5 without understanding the basic requirement to monitor and control selected sub-processes using statistical techniques, and using that as a foundation to improve the process and organizational performance.  This includes the introduction of new ideas and innovations - which at ML5 should be, at least in part, based on statistical data about process performance.
Here is my "high-road" interperpation of the PAs.  Keep in mind that there is a lot of detail in these and I am speaking at the 30,000 foot level.
OPP - the foundation of all HM practices is Organizational Process Performance.  It is meant to establish a monitoring and prediction capability using statistical techniques.  Here we select which sub-processes we want to include in our baselines, we set objectives that are within the natural process limits based on historical data ("voice of the process") and we model "what ifs" based on that voice (and other objectives).  It is expected that you will use a variety of techniques (for example process performance charts, histograms, Pareto analysis, et al) to understand process performance.  Just metrics isn't enough.  You need to be identifying "assignable causes" or "special cause" of variation using these techniques.
Think of this as the "single-source of truth" when it comes to process performance.
QPM - Here we will task projects with using the data from OPP to set, and hopefully achieve, the objectives for the project.  While OPP is focused on the meta-data (organizational), projects focus on the "micro-data," meaning what is going on at the phase or iteration level of their project. They compose their process based on the set of standard process using the baseline data from OPP to help them selected the sub-processes and set objectives for "what is possible" on their project (again "voice of the process").  They determine what THEY will statistically monitor, using similar techniques, like process performance charts to identify variation, and they monitor process performance, taking corrective action if this data appears to tell them they are not going to meet their process and quality objectives.  Actual performance data is then used to update the baseline process performance data that OPP is monitoring and tracking.
Think of this as the projects benefiting from past projects (using OPP), and then passing that benefit fo future projects (QPM to OPP).
CAR - Causal Analysis and Resolution is a (mostly) project-based PA that analyzes statistical data (provided by both OPP and QPM (through OPP)) to identify defects or problems (process and product) and examines possible causes using techniques such as mind-mapping, fish-bone diagramming, and the "five-whys."  Proposals for correction are made, these proposals are implemented, the "correction" is evaluated, and the data is recorded so that OPP can use it by updating their baselines with actual performance data.
Think of this as a way to know which problems you should be attacking in which sequence, and then ensuring that you actually solved it.  

OID - Organizational Innovation and Deployment is an organizational level PA that is similar to OPF in some ways, in that it strives to understand process needs, meet organizational objectives, and implement action plans, but it's far more deliberate AND is based on statisticdal data. Proposals are collected, and the feasibility of those proposals is analyzed, using data and modeling provided by way of OPP and CAR.  The deployment of new processes is also more cautious than OPF, in OID pilots are used, effects of the change are evaluated, and deployment is managed all within the context of meeting the objectives of the business. 
Think of OID as a more cautious, deliberate, and focused way to improve the process, with a higher degree of certainty that the improvements will not just be "different" but will enable us to better achieve process and quality objectives.
You can think of High Maturity as a set of techniques to bring the "Voice of the Process" (the actual performance results, within limits, of a stable process being performed) and the "Voice of the Customer," the specification that is set as a goal for us to achieve.
Hope this helps!

Can I lead a CMMI effort without being a certified assessorr

I've been assigned the leadership of our CMMI-based process improvement effort.  Do I need to become a certified Lead Assessor to do this?

Congratulations (I think?)!  This is a great opportunity for you to have a real impact on the direction of your company (or it could be a spectacular failure, which wouldn't be so good)!

There is no certification required to lead this type of effort, so have at it!  If you choose to conduct a formal SCAMPI appraisal you will need to engage a SCAMPI Lead Appraiser.

However . . . .

This type of effort would be more successful if you had at least the basic "Introduction to CMMI" training class, and if possible, the "Intermediate Concepts of CMMI" course as well.

The CMMI is a complex set of interdependent practices, and even though I've taught the "Intro" class over 40 times, I still learn new things every time.

"The Book" won't tell you anything about how to successfully lead a project like this though  - it will merely give you a view into what other successful companies have done with their process (i.e.; "best practices").

Good luck!

Tuesday, September 16, 2008

Hey Jeff! Some of the process areas seem redundant. Why do we have Requirements Management AND Requirements Development?

I have few questions about understating the difference between the following Process Areas of CMMI.
1. Difference between Requirement Management and Requirement Development.
2. Difference between Measurement and Analysis and Quantitative Process Management
3. Difference between Organization Process Focus and Organization Innovation and Deployment
Could you please provide me some insight on these process areas to clearly explain the difference. I would be grateful if you can provide me some examples.

A casual reading of the CMMI model specification may leave you with the impression that they are redundant, but a bit of research will reveal that each process area has been carefully designed for a specific reason.

Requirements Management provides us guidance for accepting new (initial or changed) requirements.  We're asked to ensure they "pass the test" for acceptance, that everyone understands what accepting that requirement means to the project team, that traceability is updated (or created), and the coordination with the plan takes place.  It is a Project Management activity (usually).

So for instance, a requirement MUST be testable, traceable, provided by the right person, and accepted by the right person.  That's the first practice in Requirements Management.

Requirements Development, on the other hand, involves asking and understanding the needs of the customer, developing a requirements specification, drilling down to product and/or technical requirements, and validating the agreed upon requirements. RD works WITH REQM, but they are different.

Measurement and Analysis provided guidance for creating an infrastructure for organizational measurement, and for the collection, storage, and analysis of those metrics.  It is foundational for ML2 (and the rest of CMMI).  Metrics need to tie to the goals and objectives of the organization - indeed - they need to support them.  You might say they need to be traceable to the goals.

Quantitative Project Management (you wrote "process" but I think you meant "project") is a set of practices that guide project managers towards the use of statistical models (generated in OPP) to select and manage their process (by "composing the defined process") in order to meet quantitative ("statistical") goals and objectives.  The statistical implication is always present in ML4 or ML5 process areas.  It is not about creating or using measures, it's about monitoring the project, and then feeding the data back into the baseline.

Organizational Process Focus is a set of practices that guide you through the management of your process improvement project.  It includes setting of needs, assessments, planning, and executing the process project plan. 

Organizational Innovation  and Deployment) may seem a bit like OPF, but in OID improvements are selected using the data generated from OPP ("quantitative vs qualitative") and QPM. Innovations are therefore much more targeted and granular.  Also, piloting of the proposed process improvement is expected.

Take a detailed read through these process areas and you'll see what I mean.  Good luck!

Monday, September 15, 2008

Can a company go right to Maturity Level Three or Four?

Our company just performed a self-assessment and we believe we can jump right to Maturity Level Three or Four.  Are we allowed to do that?

First of all, congratulations on completing a self-assessment.  That's the first step to improving you company, your products, and the satisfaction of your customers.

There is no specific rule against performing a ML3 appraisal "out of the box" without first performing a ML2, although the SEI cautions you that you "should" not skip levels.  That said, I've performed a number of appraisals for organizations that started at ML3.  So it's possible.

It may not be advisable though.  In order to deploy a useful and sustainable process that actually helps your company (a novel idea!) it's helpful to think in terms of releases and iterations.  People learn incrementally, as do organizations, and your chances of success are dramatically increased if you deploy smaller components of the process over time.

ML2 is designed to cover the basics of running a project-based organization.  It may seem like it's easy, but it can be difficult, and the results can be dramatic.  It would be healthy for your organization to first achieve CMMI ML2, run it for a year or two, and then address ML3.

But, that said, I've seen companies push it through, although the results have not all been successful.  If you're going to do it, consider conducting a less formal SCAMPI-C or SCAMPI-B appraisal prior to your SCAMPI-A to ensure that you really ARE performing at ML3 before you conduct an appraisal that goes into the public record.  Good luck!


Saturday, September 13, 2008

Join us for our October 'Introduction to CMMI' class

Fellow blog'sters,

Fall is a great time to be in the upper Midwest in Michigan and attend one of our fun and informative "Introduction to CMMI" classes!  Not only will we have some great debate and conversation about process improvement, but we'll also be revealing the "Secrets of CMMI Appraisals" and how to deploy process improvement using "AgileCMMI."  Visit www.broadswordsolutions.com for more information!

Is the CMMI just for Software?

I'm so happy to have found your blog.  I have a question.  My company believes that CMMI is only related to software development and our information only web site (authoring of announcements) should not be included.  Is this true?  If not, why?

The genesis of the CMMI was indeed in software, but since its introduction in 2001 it's been pretty clear to most adopters that it is a process model for any kind of product delivery, although engineering is the most common usage.

The key here is to not assign categories of work but to ask "what is it that I have to do to deliver this product?"  
I have several clients in the "content management" business whose work revolves around placing information on the web for others to use.  They asked me the same questions - so here's what I asked them: Do they have to plan for this?  Do they have to validate that the data is correct?  That the server works?  Is there budget associated with their work and can they exceed it?  Do they need to be concerned about overwriting the data or the currency of the data? 

Just these few things encompass the CMMI Process Areas of Project Planning, Validation, Configuration Management, Product Integration, and Project Monitoring and Control (and there are certainly other things they do right?).

So, on the face of it, it appears that the CMMI is quite versatile and could apply to this scenario.  Good luck!

What is the difference between the different CMMI Levels?

Please can you tell what the main differences between each CMMI level are?

Welcome to the CMMI!  The great thing about this business is that new people are always coming into it! Sometimes we get so wrapped up in debating esoteric points that we forget that new people are wondering what the heck we're talking about!

The different CMMI levels describe the process performance behavior of an organization starting with a chaotic state and progressing upward.  Without getting in to all of the details of Maturity Levels and Capability Levels (all things you will learn when you take any "Introduction to CMMI" class), let me try to paraphrase it and net it out for you.

A Level One organization is pretty chaotic and reactive, responding to defects and issues as they arise.  They typically experience a high level of defects discovered late in the engineering cycle, miss delivery dates, and underestimate the project effort.  Sound familiar?

A Level Two organization demonstrates some basic process management at the project level - specifically around the project management (PP, PMC), measurement (MA), Configuration Management (CM), QA (PPQA), Requirement Management (REQM), and Supplier management (SAM) areas.  There will  be policies, plans, some training, some monitoring, and some review by management (among other things) of the process, albeit at the project level and perhaps not consistently across the organization.

A Level Three organization has invested in a process infrastructure, has a set of standard processes from which projects can choose, uses an engineering life cycle, continuously improves the process, and manages risk.  There are eleven additional process areas in Maturity Level Three (including all of the engineering processes) and the whole thing is managed like an overall process program.

A Level Four organization has a statistical understanding of the way their process performs.  They are able to understand what occurs when a particular sub process is employed, and they are able to predict what WILL happen (based on data) if a sub process will be used.

A Level Five company uses that statistical data to continuously improve the process and to bring predictability to the introduction of innovations and improvements to the organization.

A good Introduction to CMMI instructor should be able to elaborate on all of this.  Good luck!