Our LA says reviewing test results satisfies VER SP2.2. What do you think?

Just wanted to know your clarification on the following

Process Area: Verification (VER)	Specific Practice: SP2.3 Analyse Peer Review Data	Reviews and Internal Testing (Component, Unit, Assembly, Performance) - The input provided by the Lead Appraiser(LA) was that analysis done in any of the above testing phases by the project can be considered for VER SP2.3
Process Area: Validation (VAL)	Specific Practice: SP2.2 Analyse the results of the validation activities	The following are being used for addressing VAL SP2.2: As per the inputs from the LA, evidence of reviews done by customer for design or any inputs provided for testing.

Nice chart!

VER SG 2 is specific to PEER REVIEWS, so testing evidence is not typically what we look at.  If your LA is saying test results, and review of those results, is valid Direct Evidence for VER SG2 then you are getting incorrect information that may not stand up to scrutiny.

VER SG1 and SG3 can indeed be evidenced IN PART by test results and reviews, but not completely.

Peer reviews are structured reviews, walkthroughs, Fagen Inspections or any other formal review of a specific work product or set of work products.  They need to be identified through practices in SG1.  They also must be planned for.

VER SP2.2 “Analyze Peer Review Data” includes not only analysis of the results of the peer review, but an analysis of the preparation, participation, and data associated with the conduct of the peer review itself.

Some of VAL can be evidenced by customer review outputs, but SP2.2 is specific to performing analysis, so just meeting minutes or a sign off falls short of the intention of the model’s authors.  It includes analyzing results against expected results, categorizing or measuring (i.e.; analyzing) the results, and recording information.  Customer inputs for testing does not satisfy this practice.  Again, the information you are getting would not stand up to scrutiny.

www.broadswordsolutions.com

How to not pick a Lead Appraiser!

Dear Readers, 

I often get questions about how to go about choosing a Lead Appraiser.  LAs should be selected after a careful discussion of their experience, knowledge, culture, philosophy, and willingness to spend the energy to learn about your business before he/she waves his magic wand and proclaims you MLx (of course, it's more complex than that!).

By way of example, here is the worst possible thing you can do.  I had an inquiry the other day that went like this:

 Riiinnnnng! (telephone)

Me: “hello?”

Caller:  “hi, we’re shopping for an LA to give us a maturity level.”

Me: “Wow.  hmmm.  Great.  Well, uhh, ok...  tell me, what are some the reasons you are interested in achieving a CMMI maturity level?”

Caller: “We don’t need to talk about all that. We have a company from India that says they can take us from ML1 to ML3 in three months for $15,000.  Can you beat that?”

Me and my telephone: “click”

I’m serious.

www.broadswordsolutions.com

I was selected to be an Appraisal Team Member but I don't show up on the SEI's site. What gives?

My company is now working on the CMMI-3 appraisal. I attended the "Intro to CMMI" training and also successfully registered as an ATM on the SEI website. Two weeks later, I'm still not showing up in the SEI's system.
I was told by the CMMI consultant to check "My Appraisals", but unfortunately, till now nothing can be found (http://sas.sei.cmu.edu/AppSys/).  I think the Lead Appraiser has NOT selected me from the pool.

What concerns me is any adverse impact on my company's CMMI-3 appraisal if still nothing can be found in "My Appraisals" by the start (even the end) of SCAMPI A appraisal ?

Could you please kindly tell me the criteria for an ATM to participate on a CMMI-3 appraisal?



In order for you to participate and be added to an appraisal a few things have to happen.

     - You need to successfully complete the Intro to CMMI course

     - You have to demonstrate the pre-requisite experience as outlined in the SCAMPI MDD for engineers, project managers, or managers

    - You need to fill out, sign, and submit the SEI’s FERPA form (if you are in the USA)  

    - You need to register on the SEI’s ATM Pool registration page (sas.sei.cmu.edu).  If you were able to do this then the first two items were satisfied.

    - Your Lead Appraiser must create the Appraisal Record in SAS

    - Your Lead Appraiser must add you to the appraisal as an ATM

    - You need to complete the SCAMPI Team Training given by your LA

 This all should be done prior to starting the appraisal, but it MUST be done by the end of the it. 

From your message it sounds as if your LA has either not created the appraisal record, or if they have, has not added you the appraisal team.

www.broadswordsolutions.com

Should we be freezing our artifacts six months before our SCAMPI Appraisal?

The Contractor at my Government office who is responsible for building a new, complex IT system will be conducting a SCAMPI-B this fall in anticipation of an eventual SCAMPI-A the following year. 

The Contractor wants to "freeze" various process documents 6 months prior to the SCAMPI-B so that all process output artifacts are in sync with the process description artifacts used in the appraisal. 

Freezing "living" documents for 6 months seems a bit extreme to me, given that some of the documents are deficient in how thoroughly or accurately they describe existing process. Does this seem like a reasonable request? (Note: I'm fine with not changing the actual processes/procedures for 6 months. It's the idea that we won't try to improve the deficient description of those existing processes that concerns me. Particularly since the Contractor continues to bring on new staff, who need to rely on these documents).

I'm visualizing what "freezing the artifacts" might look like - sounds like an old batman episode!

Freezing artifacts is often one of the things we see from organizations that are trying hard to “pass” an appraisal, but perhaps don’t have the processes embedded in their culture (which is kind of the point, isn’t it?). 

It seems as if your contractor is overly risk-averse.  A SCAMPI-B is in itself a risk mitigation event, and to work so hard to reduce risk for this event seems counter-productive.  It’s supposed to uncover problems.

In your case it seems like the risk mitigation strategy they are deploying is in conflict with the very essence of continuous process improvement!  It’s one thing to freeze process – it’s another thing to freeze the outputs.

I usually recommend freezing everything starting with the SCAMPI A Readiness Review (about 60 days prior to the onsite of a SCAMPI A).  Anything earlier than that isn’t productive, in my opionion.

www.broadswordsolutions.com