Wednesday, October 31, 2007

Must we use historical data for estimating?

This is first time I am directly writing to you, otherwise I've just been reading your comments and feedback on the group.

I have two questions on which I'd appreciate your comments, because answers to these will help me in building sound concepts of CMMI v1.2. Let me give you a brief history of our organization. We are striving for ML 2 organization and ready for our Class A Appraisal.

Since we are striving for ML 2, we are in the phase of data collection which later would be referred to as historical data for the estimation of effort, cost and size of the project. Currently we are estimating Size using FP Technique and Effort using PERT technique. But there is no relation between size and effort established in our current OSSP.

It is also not required in the sub practices of SP 1.2 of PP. To satisfy SP 1.2 and SP 1.4 of PP Process, is it enough to show that we are in data gathering/collection phase and no direct relation between size and effort data exists or we have to show the relation and people checking that historical data as well while making estimates for the projects?

My second question is, trainings are planned, conducted and people do their routine work after getting training. Kindly let me know how the adequacy of trainings is determined for GP 2.5. How much details evidence is required for an organization that is going for ML 2.

Is it justifiable if an LA fails an organization just because

1. He is unable to determine the adequacy of trainings
2. Direct relation between size and effort data is not established and people are using their expert judgment but that judgment is not supported by historical data?

Your answers will be really appreciated and helps me and my organization to have better implementation of processes.

You've obviously been reading up on the model. You must be having trouble sleeping!

On the question about using historical data to estimate, and what is required by the model, I would ask this: if it were required by PP, does that mean companies that DON'T have historical data cannot achieve CMMI ML2? Of course not. I don't think that was ever the intent of the model. There are many ways to estimate the project, and using historical data is only one of them. The fact that you are developing a historical dataset is a good thing, and will only strengthen your performance and appraisal ratings in the future.

As to the training, we expect you to provide evidence that the right people were trained in the right process areas. There is no need for everyone to take all the training, or all levels of training. At ML2 there is no requirement for a formal "training capability" as there is in ML3, so 1:1 training, mentoring, presentations, or CBT's would all qualify as long as you kept records of people having attended or executed the training. Keep training sign-in sheets and the training materials and you should meet the requirements of GP2.5.

As for an LA "failing" an organization (technically speaking there is no "failing") because he is "unable to determine the adequacy of the training", I would say that this would be valid if he truly could not evaluate whether you had training or not. He should not be evaluating "goodness," only that you trained people appropriately for their roles.

Same for the next question. There is nothing in the model that requires you to use "historical" data for estimating size. That said, "expert judgement" may or may not be appropriate depending on what you mean by that.

Either way, don't let an appraiser tell you how to run your business or claim that there are prescriptive ways to perform a process in the CMMI - there are not any. It's strictly a definition of "what" not of "how" and the subpractices and work products are only suggestions.

Best of luck to you.


Anonymous said...

Please note that I am only an ATM (not a Lead Appraiser), but I have just been through a level 3 appraisal where estimation caused a REAL problem. My comment/advice is that estimation is an area which appears 'easy', but it has several different ways of doing it well & giving a real benefit to your projects. BUT some lead appraisers are very blinkered/lazy & fail understand the true meaning of the model, & simply have 1 or 2 interpretations that they know, & try to force you to do these. From personal experience I strongly suggest that you work closely with your lead appraiser & whatever you agree GET IT IN WRITING before the appraisal starts. If you have an appraiser who inists that estimates HAVE to been done by size, then change him/her straight away. This is the 1 point of the model I have really investigated with the help of the SEI & can truly speak with confidence. Size works really well for some projects (including some in my company), but is NOT the best solution for all.
Hope this helps.

Anonymous said...

Good dispatch and this mail helped me alot in my college assignement. Thanks you on your information.