For more than the last 10 years I’ve been thinking a lot about CMMI. Many of these thoughts have been ruminating on the ideas of how to incorporate CMMI in ways that add value, demonstrate effectiveness, and don’t disrupt the operation. I’ve even opined much in this blog (in too many ways) on the need to know what your processes are before you can use CMMI to improve them, and that for many operations, CMMI isn’t even appropriate.
A few recent discussions and experiences put a particularly fine point on the extent to which CMMI is really the ‘cart before the horse’ when applied within an operation that has yet to clearly discern its process, and here’s why:
Most operations I’ve encountered are not ready to use CMMI because they are unclear on exactly how they make money.
Obviously, I’m not talking about the accounting process of billing out invoices and depositing checks. And, I’m not even talking about the voodoo around figuring out how to ensure that internal costs (salaries, equipment, etc.) are less than what they charge clients for the work they do.
So, I must be talking about something more subtle. I wish I were. And, this is what’s both frightening and sad. I’m merely talking about the relationship between capacity and demand. For that matter, I’m not even worried much about demand, which is another matter. I’m mostly talking about capacity.
What is capacity?
According to some, capacity is a measure of volume of work. Throughput, for instance. According to others, it’s the wherewithal to do the work. Either way, too many operations don’t know what their capacity is.
What does it actually take to get work done? And, along with that, can it be reasonably expected of the operation to reliably and predictably continue to run how it runs (not knowing exactly how it runs) and to have any right to greater-than-zero confidence that they will continue to run as it does?
For one thing, many operations run outside of reasonable tolerances. In particular, people put in many many hours of unpaid over-time [in the US this is common for salaried employees]. This is an "out-of-tolerance" condition. It is unreasonable to expect an operation’s greatest source of working knowledge to continue to work nights and weekends. Furthermore, it is risky to do so. One client said he couldn’t be away from the office for 5 minutes before product would stop shipping. Eventually he will get married or his wife will have a child, or, heaven-forbid, he may take vacation!
What’s worse is the extra time he and his team put in is entirely unaccounted-for. His employer merely estimates, contracts, and bills enough to cover his and the team’s salaries, not what it actually takes to get work done.
Another reason true capacity is obscured is because more work going into the operation than there’s product (or services) coming out. The most common cause of this is the misperception that work started = work completed, but this is an incomplete equation. But a better equation to work with is work worked-on = work completed. The key mistakes is the assumption that started = in-work. That’s true for maybe 50% of the actual started work (often less).
This next reason for the lack of insight into true capacity (or capability, really) is that so many operations don’t account (either in their estimates — which sort-of makes sense — or in their capture of time-spent — which is unforgivable) for the time to correct defects, time to perform rework, time for paying-down technical debt, or time and effort to tracking-down the causes of defects and rework to avoid defects, rework, and technical debt in the first place!
I’ll end with one of the toughest, most sensitive observations of the last few months. Some of my best clients (from the perspective of having their act together) have strong confidence in functional competence and, admittedly, weaker confidence in their programmatic credibility. And, to put it plainly, by "programmatic" I mean their ability to have the same confidence in the rationale for their estimates and plans as they have in their ability to produce what their clients want.
In these operations, I’ve found fundamental disconnects between how work is estimated and why clients should trust the technical competence of the operation. In other words, they build trust with their prospects and clients on their ability to do the work and build the products, but in order to get the work in the first place they have to use a lot of hand-waving and breath-holding when it comes to their estimates.
A more-or-less summary way to describe all of this is as follows:
Most operations have Built a way of working that they’ve managed to Capitalize. Over time, they’ve found that their Build*Capitalize approach is tough to Sustain. Try that they might, whether it’s CMMI or something else, they’re looking to "fix" the equation on the "Sustain" side. The problem is that CMMI *does* operate on the Sustain side, but the problems with the operation aren’t in the "Sustain", it’s that their approach to Capitalizing on what they Built is no longer Sustainable. What needs to change is on the Build side. Occasionally, there’s a need to revisit the Capitalize component, but most often it’s the Build that needs refactoring.
Hence, applying CMMI to an operation whose "build" is broken is putting the cart before the horse. While it’s possible to build CMMI practices into the operation’s way of working, this is an activity of the "build" side of the equation, the sort I noted above contrasting from applying CMMI to the sustain side. If CMMI is to be truly about improving the processes of the operation in a "sustaining" sort of way, and not defining them, the operation must understand what’s going on, and that means it must know its capacity. Because unless it knows its capacity, it doesn’t really know what’s going on.
