Just the FAQs – How can we REALLY know how things are going?

[Dear Readers, for the past several months, our good friend Pat O’Toole, CMMI expert and seasoned consultant, has been collaborating with us on a monthly series of CMMI-related posts, "Just the FAQs." Our goal with these posts is to provide answers to the most frequently asked questions about the CMMI, SCAMPI, engineering strategy and software process improvement. This month, Jeff shares his secret for assessing the capabilities of product development teams. Take it away, Jeff! ~ the CMMI Appraiser]

Hey, Jeff – We want to assess the capabilities of our product development teams. I’m reluctant to schedule a SCAMPI appraisal because, I understand, they are no fun at all. How can we really know how things are going? ~ Bradley S.

JEFF: Wait a second. I love conducting appraisals. No – I really do!

I know what you’re saying, Bradley. Appraisals are lengthy, tedious, soul-sucking events that drive the joy and passion out of the otherwise happy-go-lucky appraisal team members who have been “volun-told” to spend weeks locked in a room ….with me! Seriously, what could be more fun!

But what I love most about appraisals is how eloquently and predictably developers, project managers, line managers, CIOs, and CEOs tell me how awesome they are. Planning, designing, traceability (OF COURSE!), peer reviews, collecting lessons learned (OBVIOUSLY!), and more are all described in excruciating and colorful detail. And they’re always so great! Heck – they’ve “self-assessed” at ML5, and if I only understood them I would agree. It’s a hoot!

After listening to all of the colorful descriptions of their awesome behavior, I train my gaze to the other side of the room, and in the furthest corner, in the darkest reaches of that corner, in the most hidden part of that corner, a tester is waving her hand across her neck while mouthing the words “NOT SO MUCH.” It’s always the best part! My personal record for the length of time it takes to see this is 37 seconds, but who’s measuring?

See, if you ever REALLY want to know what is going on in your organization, ask a tester. As consumers of distorted vision, weak requirements, poor design, and sloppy code, they are asked to behave like one of Harry’s Potter’s Dementors, the fantasy non-beings that takes in chaos and misery, leaving the developers and project managers happy and burden-free. Tough job!

While the most common reaction to busy testers is MORE testers and tools, building great technology products is about more than just testing code. In fact, it’s hardly about code at all. Yet most books, articles, and conference speeches about improving software quality focus on testing tools and automation. While these are all good and necessary discussions, we have thus far fallen short of reaching the goal line: consistently building high quality products that delight our customers.

Isn’t it about time we add something new to the discussion?

As a Lead Appraiser and Agile coach, I am often asked to assess the capabilities of product development teams, and I can usually ascertain strengths and weakness within fifteen minutes – if I start with the test team.

Great software is an ecosystem borne from a glimmer in someone’s mind, matures into needs, grows into requirements, transforms into designs, manifests itself as code, all the while being validated and verified, until it finally emerges to delight and satisfy our customers.

Instead of starting the discussion with expensive tools (who doesn’t like a new toy?), take your first step toward building better products by using the tool between your ears to answer the following questions that every tester asks:

1. What is the Product Vision? Unclear product vision manifests itself as chaos during testing because developers and analysts take it upon themselves to interpret – or even create – product vision. Product vision exists at multiple levels: company, product, team, and individual, and should be clearly defined and written down at all of them. You’ll need it later when you’re pressured to cut corners after everyone forgets why they’re building the product! Guidance and tips for developing a comprehensive product vision live in Requirements Development SP1.1 and SP1.2, Technical Solution SP2.1, Measurement and Analysis SP1.1, and elsewhere. Put them to work to create an integrated view of your organizational and product goals, objectives, and outcomes so that everyone clearly understands them.
2. Are the Requirements any good? Most industry studies peg misunderstanding of requirements as the primary reason we experience product defects and unhappy customers. Many testers know intuitively that this is a serious problem that costs our industry billions of dollars – and they know it because they are beaten up day-in and day-out by the downstream effects of weak requirements. The CMMI’s REQM SP1.1 provides’ excellent guidance for identifying the most insidious defect – lack of clarity – and correcting this process defect will results in a dramatic improvement in customer satisfaction and product quality. Adopting Test Driven Development, a staple in the agile community, is a solid implementation of practices in Requirements Development Specific Goal 3, and this can be your best tool to avoid the frazzled hairstyles that most testers are sporting after the “night-before-launch” test party. 
3. Is the Code any good? The CMMI gives scant attention to code quality, but some guidance is available in Technical Solutions SP3.1 and a thorough examination should be included in any software development appraisal effort. Clean code doesn’t happen on its own, but is the result of well-established behaviors, processes, and coding standards: naming conventions, formatting standards, variable passing conventions, complexity guidelines, and of course, code reviews. I learned this lesson many years ago when I received a curt note from the head of the testing team titled “thank you for letting me do your unit testing for you.” If a tester tells you they are capturing defects that fall into the aforementioned list, it’s a strong indicator that clean code is not on the top of anyone’s list.
4. Are we always improving? I tell my classes that “those who are always improving, win.” There is ample guidance in the CMMI about continual improvements, in fact the ENTIRE CMMI model is about this subject, but that doesn’t stop numerous Maturity Level Three companies from ignoring the advice in Generic Practice 3.2, or the dynamic duo of Integrated Project Management and Organizational Process Focus. Testers will immediately tell you whether they are seeing the same types of defects over and over (and over) again. Listen to them.

Appraisals can be tedious and time consuming, and everyone is looking for ways to optimize and accelerate the process (while reducing cost). So, with that in mind, I have a humble suggestion – start by asking a tester. You’ll be glad you did, and they might enjoy being in the spotlight for once!

© Copyright 2015: Broadsword Solutions and Process Assessment, Consulting & Training

“Just the FAQs” is written/edited by Jeff Dalton and Pat O’Toole. Please contact the authors at jeff@broadsword.com and pact.otoole@att.net to suggest enhancements to their answers, or to provide an alternative response to the question posed. New questions are also welcomed!

Like this blog? Forward to your nearest engineering or software exec! 

Jeff Dalton is a Certified SCAMPI Lead Appraiser, Certified CMMI Instructor, author, and consultant with years of real-world experience with the CMMI in all types of organizations. Jeff has taught thousands of students in CMMI trainings and has received an aggregate satisfaction score of 4.97 out of 5 from his students.

Visit www.broadswordsolutions.com for more information about running a successful CMMI program