Dear Appraiser,
I’m not sure that the CMMI consultant we’ve been working with really understands what our data collection approach needs to be - especially with the new release of SCAMPI v1.3. He says the appraiser just shows up and looks for documents and he refuses to prepare us in any way. Can you shed some light on what’s changed? ~ Arnold C.
Arnold, thanks for your note. It’s not uncommon to hear that a "consultant" is confused!
Data collection for appraisals is an area of specialization, and a Lead Appraiser can really help you out and save you a lot of misery. That’s partly why I’ve decided to address several of the rule changes in the new release of in SCAMPI in my latest blog posts. Since you have a specific need, today we’ll look at the various data collection strategies. Data collection is a complicated subject, and well beyond anything that will fit in one blog post. But stay tuned!
Today we'll focus on the three different types of data collection strategies identified by SCAMPI and available to Lead Appraisers. The options (or "modes") are:
- Discovery
- Verification
- Managed discovery
The v1.3 terminology changes associated with each option represent a fairly significant impact on how appraisals are conducted. Let’s look at them separately:
- Discovery
Think of discovery appraisals like the discovery phase in legal proceedings. The Lead Appraiser does not participate in any preliminary collection or analysis, but simply shows up for your appraisal onsite (the last part of the appraisal), and data discovery begins at that point (and you'd better hope you're ready!) Needless to say, Discovery appraisals, while very popular with level-seekers (and Lead Appraisers that want to appear omnipotent) they very often do not lead to useful results. More often they lead to an extremely stressful appraisal with late nights and lots of heated debate and bruised egos. In the early days of CMMI, this is how most appraisals were conducted. Not my first choice.
- Verification
At the opposite end of the spectrum are verification appraisals. Verification appraisals are about getting the team to identify all the work product/artifacts in advance. When the Lead Appraiser shows up, he or she verifies that you have selected the correct data (or not). In my opinion, this model has always been superior to discovery because it required you, the appraised organization, to learn about yourselves over time, in an iterative and incremental way, leading to greater introspection and great value. The verification approach gave you a lot of opportunities to solve problems, which meets the spirit of the CMMI much more closely than the Discovery appraisal, which for all intents and purposes is perceived as an audit.
"Back in the day" prior to SCAMPI v1.3, Discovery and Verification appraisals were our only choices. But neither really felt right to me. I felt we needed something more. Since we were allowed to tailor I starting performing "Verification with a twist" appraisals, and turned them into my particular speciality. It didn't meet the definition of Verification Appraisal exactly . . . .until ,like magic, a third option appeared which closely matched what we already were doing (I love those change requests!)
- Managed Discovery
Managed discovery is a new category in the model. It’s a hybrid of discovery and verification.
Actually, managed discovery is just a terminology change for the method I was already using to conduct Verification Appraisals. It’s how many of the best Lead Appraisers/CMMI consultants have always done appraisals. We play an active role in helping you prepare, review and identify artifacts in the weeks and months leading up to the appraisal. We help you plan, and we help you manage the data collection process because we care about your success. Remember, we're not auditors!
Here’s an example. When Broadsword conducts an appraisal, I meet with the appraisal team at least 60-90 days before the appraisal to define how the data will be collected. I help them form their teams, and I collaborate with them to develop explicit assignments for them to analyze specific projects or process areas. Then I send them out to collect and analyze the the data.
I meet with them once per week (or more if they need it . . . and some do!) to review what they’ve come up with. I answer their questions, I give them suggestions for improving the process they are using. I talk with them about software development, project management, and engineering, check what they’ve found and lead a discussion on what it all means in the context of the CMMI model. Through the whole process of collecting data I manage the collection and learning process. Then, when I arrive for the onsite to perform the Discover portion, we only have to interviews and some additional verification to perform. The artifacts have already been reviewed!
Let’s face it. The purpose of data collection is not to audit the process or require the company to show us 8,000 documents. The purpose of data collection – and all actions associated with the SCAMPI -- is to have reflection, discovery and introspective discussions about the work products over time. To determine if they are useful to the company and contribute to their success. You can’t do that in audit mode.
If your CMMI consultant can’t get that, it might be time for another terminology change … as in former-CMMI consultant.
Jeff Dalton is a Certified SCAMPI Lead Appraiser, Certified CMMI Instructor, Candidate SCAMPI Appraiser Observer, author, and consultant with years of real-world experience with the CMMI in all types of organizations. Jeff has taught thousands of students and has received an aggregate satisfaction score of 4.97 out of 5 from his students.
Learn more about the right kind of CMMI consultant at www.broadswordsolutions.com
No comments:
Post a Comment