Picking a Lead Appraiser: "Dammit, Jim! I'm a doctor not a bricklayer."

In this quote, CAPT Kirk wants Dr. Bones McCoy to do something he feels he's not-qualified to do because he doesn't know how to treat the species.

  

I'm using it to explain that organizations looking for a lead appraiser to work with them towards an appraisal and/or to perform an appraisal ought to think of what we do as they would think of a doctor, not a laborer or vendor. 

Do you really want the lowest price doctor?

For that matter, is the highest price doctor necessarily the best in town?

When reaching out and interviewing for a lead appraiser or CMMI consultant, you:

• Want the person who is the right person for the job.

• Want someone who is qualified (definitely not under-, but preferably not over- either).

• Not the lowest bid.

Seriously, whoever you hire for this effort has in their power the ability to make or break your future.  They literally have the health and well-being of your organization in their hands.  They can put you in the dump just as easily as they can take you to the next level.

They should see themselves that way as well. 

Unfortunately I've got too many sad stories of appraisers/consultants who definitely see that they can make or break you, but they don't feel like they personally own the responsibility for what happens to you when they're done. 

If it costs too much?  So what?  
If you get no value?  Not their problem.  
Didn't see any benefit?  Didn't learn anything?  Things take longer and cost more and you're not seeing internal efficiencies improve?
YOU must be doing something wrong, not them.

In an AgileCMMI approach, your CMMI consultant and/or lead appraiser would see themselves as and act like a coach, and would put lean processes and business value ahead of anything else.  And, an AgileCMMI approach would know that when the processes work, they add value; when they add value people like them and use them; when people like and use them, the next “level” is a big no-brainer-nothing.  You get it in your sleep.

Let me know if you want help finding the right lead appraiser or consultant.

So you're really interested in CMMI for the rating only….

What can you do?

This entry addresses an important tip in those cases where you need to demonstrate a CMMI rating even though you’d otherwise have no intrinsic business reason compelling you to do so.

But first, I admonish organizations doing the compulsory CMMI ratings requirements in the first place.  So, if you’re a company being externally compelled to get a rating (which is different from being told “you need to improve!”), you might want to send a link to this finger-wagging to whoever needs to hear it.

However, as such a company being externally compelled to use CMMI (just to get a rating), this tip will make it MUCH easier and more beneficial.

Oh, one more thing…. I don’t mention this in the clip: If you’re such a company, don’t look to hire the cheapest, fastest appraiser/appraisal you can find.  Doing that will only make the cost and pain worse.  Both, short-term and long-term.

NEXT WEEK: Everything you thought you knew about CMMI is wrong.

NUTs, GUTs, Hours and Days. They're all AUTs and should be treated as such!

NUTs:: Nebulous Units of Time

GUTs:: General Units of Time

AUTs:: Arbitrary Units of Time

So much emphasis is placed on time and estimates in planning development work. In reality, even hours, days, and weeks are nothing more than arbitrary measures.

To be clear, time as we typically account for it, is an important component of planning and estimating. But, as are many other aspects of development (and life in general?), are open to several interpretations, and, when taken literally, can result in undesired consequences.

Let's face it, in the world of development, estimates have lost their original dictionary meaning. I, for one (and I doubt I'm alone), believe this to be unfortunate. Estimates were never meant to be locked-in forever. They were meant to be a guide to making the next set of plans, then moved forward to make the set of plans after that, and so forth. But when estimates started to be used as the basis for long-term budgets, they lost their original definition and took on the expectation of being accurate.

(You can see how this lead to the ideal that to improve estimates, requirements needed to be more, more, more of everything about them.)

Everyone knows that estimates are frequently largely fiction. That's true of even the most capable and mature organizations. (In fairness, that comment applies when experienced organizations are trying something completely new. In those cases, the new aspects of the effort have low confidence in estimates and the common aspects of the effort would have higher confidence in estimates.)

However, even when not attempting to estimate "the whole thing", when projects are only trying to estimate a single user story, organizations get very skittish about making any commitments to estimates. This is often a symptom of placing a high premium on the perceived value of the time. In other words, a "day" = a "day" and a "week" = a "week", just as we'd count time between now and our next vacation.

As a result of this reluctance to ascribe estimates due, in part, to the automatic psychology attributed to the significance and meaning of the number, and in part due to the concern of being held to them, the notion of "Story Points" (look here and here) came about. Story points offers a less concrete, more relativistic, and seemingly more natural way of estimating.

Story points allows estimation based on the relative perceived effort required to complete one user story as compared to other stories. This story takes longer than that story, and so on. This eliminates the natural tendency to put undue emphasis on the number and provides a means of filling a time-box with a number that can be later compared to how much work was actually completed in that time-box.

This approach easily lends itself to tracking story points per iteration, which is one way to measure "velocity". With the velocity value, the total story points of the project, and the length of the iterations, one can project how many iterations, and thus, how much time, before a project is likely to be completed.

There's one (maybe more, but only one that we'll look at here) challenge with story points: as beneficial as they are at eliminating the many "psychological" connections to time that are associated with using "time", they're also not very natural to humans. Even those experienced with using story points have learned that the estimates aren't consistent, story point estimates from iteration to iteration aren't stable or predictable, and, when required to plan for new, or large, or long-term, or complex situations, points from previous projects aren't much help and don't satisfy many organization's needs for budgeting.

Nonetheless, I still like the idea of story points as a teaching tool and here's why. It helps introduce the idea that estimates, whether in points or hours, or days, or whatever, are supposed to be meant for tracking progress, not setting expectations that are to be written in stone.

This is especially true when taking a time-boxed (and hopefully incremental and iterative) approach to development. The purpose of the estimate is to see how much can get done in the available allotted time and then throughout that time to check to see how much progress is being made, make adjustments to the expectations, and then at the end of that box of time to see how much was actually done.

In other words, the estimate of a task's effort should be used as a measure of productivity, not as a measure of accuracy.

Throughout and at the end of the time-box, it is valuable to take note of the differences between estimated and achieved productivity so that future estimates can be somewhat more accurate, but only so that productivity can be more predictable. Clearly, as productivity predictions become more accurate, then budgeting becomes more accurate.

When estimates are used for measures of productivity, it doesn't matter how much time, in physical clock terms, is being ascribed to the tasks. The number becomes as arbitrary as the notion of time itself. Time is merely the "distance between events". We're conditioned to be familiar with time as being discrete and concrete. So, when we use "time" to describe estimates, we're drawn to compete against the clock to achieve the task within the allotted time. An alternative is to complete as much of the task as we can within the time-box without killing ourselves, then looking back at how much "time" it took to get as far as we got and to use that lesson to better understand our productivity.

Another way to look at it is that "time" allocated to a task isn't really physical "clock" time. It's just a way of guessing a rough idea of how much work can get done in the boxed time. As it is, it's highly unusual for physical clock time to align itself nicely with how much work is actually done in that span of time. But, when it is understood that the estimate of time is nothing more than a guess used to fill-up a time-box-full of tasks, time as applied to estimates might just as well be an arbitrary number.

The only way to get a sense of "estimated time" to align with "clock time" is, well, over time and experience. This time and experience on the project level can take a while to attain. One way to short-cut the wait is with SEI's Personal Software Process (PSP), which works surprisingly well in development of other work, not just software. At the team level, there's the TSP, which builds on the PSP so that personal productivity can be used collectively with the overall productivity of the team.

However, this post isn't a commercial for TSP/PSP. The point to this post is that time associated with the clock comes more naturally to humans, and that using time as a measure of productivity makes estimates more valuable. When estimating a task, worry less about estimating correctly. When working on the task, worry less about getting it done within the estimate. Worry more about checking your progress against the estimate and making adjustments accordingly throughout and at the end of the iteration.

Keep in mind, whether used for productivity or budgeting, organizations who expect the estimate to stay precise throughout the project are deceiving themselves. Organizations who use the 1st estimate as the basis for the entire project are even more deluded. Organizations who are being given the freedom and flexibility to pursue incremental and iterative development often also have the luxury of not being held to providing imaginary long-term budgets and thus using estimates iteration by iteration is acceptable to their leadership.

Used as a measure of progress and productivity, estimates have much more power and add much more value than when used for long-term budgeting. Within a few iterations, estimation, budgeting, productivity and predictability will converge so that clock time and estimated time will be more meaningful to everyone involved.

For what it's worth: there nothing about the above that runs contrary to CMMI. Not.One.Thing.