Cut, Paste, and Why Requirements are Tricky

I've been using PDAs for about 10 years now.  So, while I wasn't the among the fist Apple Newton users,  and I didn't buy the first version of the PalmPilot®, I did follow shortly thereafter with another Palm device and was a Palm fan until it came time for me to move to a "crackberry" because I could no longer wait for Palm to come out with an email-over-air device.

Rest assured, when Palm came out with its Treo 650, it couldn't have been a moment too soon, as I lusted for it upon its announcement and needed only the smallest excuse to plunk down about 500US$ for it at the time.  That excuse came in the form of hardware failure on my RIM device.  (It was getting up in age -- not to mention that I bought it 2nd-hand.)

Well... my life with the Treo was bitter-sweet.  Software and synchronization/integration issues brought my productivity to a grinding, frustrating, halt.  By then I was keeping tabs on the latest Windows Mobile® devices purveyed by my wireless provider and saw that a device was due with *everything*.  A good phone, Email, text, browsing, broadband-over-wireless, WiFi, and simultaneous phone and Internet access all built-in.  I practically counted the hours until it was finally released after being tantalized mercilessly by the press releases and ads.

Life was good.  I really had no complaints.  Until a few days ago.  It died.  Well... mostly died.  The touch screen stopped responding, to be specific.  Despite a built-in keyboard, the device relies on its touch screen.  And, in the middle of a business trip, the touch screen stopped working.  Back at the hotel, and a while at my laptop, I learned that this seems to be common for that device.  In fact, I found myself considering myself to be lucky it lasted the roughly 20 months it did.

I put together my list of requirements, prioritized them, and a bit of research later I'd settled on my next device.  Being due an upgrade anyway, I decided to stop in at a local store to get one on my way to the airport for my flights home.  I decided to go with a device from another manufacturer that didn't have a touch screen since the previous day's experience taught me that relying on a particular feature for functionality was risky.  I was even willing to sacrifice the WiFi requirement to gain a form-factor benefit, so I had a bit of technical/performance trading-off to do.

A few hours of messing around with the new device and I was both pleased and perplexed.  Everything seemed to be what I had read about it.  Oddly enough, I couldn't figure out how to cut or copy and paste.  Furthermore, I could only find one alarm.  Which, for anyone power-using their PDA knows, alarms are critical.  So, back in my home, I downloaded the complete user's guide (since all they put in the box these days with new phones is the "quick start" guide).

Reading through the guide I learned that, indeed, the device should have Copy/Cut & Paste, as well as two alarms (which is still lame, but 'fixable' with 3rd party software).  So I called the manufacturer -- a world-wide company renowned for being early adopters of what's known in the industry today as "6-Sigma" -- and spoke to a customer technical representative. 

The conversation went something like this:

Me: I can't seem to cut and paste, but the user guide says I should.  I mean, it's not that it's not working, it's not even in the menu.

Rep: No, my technical notes [a.k.a. "release notes"] don't say anything about being able to cut and paste unless you're in [application].

Me: Yes, I can do it in [application] but I should also be able to do it elsewhere.

Rep: Why do you think so?

Me: Because it's in the user guide and it's a basic function.  Why wouldn't you be able to cut and paste?

Rep: What user guide are you looking at?

After guiding the representative to the exact user guide I was looking at, she apologized and said that the guide is wrong, but the phone doesn't actually have that functionality.  So I moved on the why the phone doesn't have more than one alarm.

Me: OK, so can you tell me where I might find the functionality to set more than one alarm?

Rep: Sure, just go to [......]

Me: Yes, I'm there, just like in the user guide.

Rep: [Sounding surprised] Oh, that's strange, I've got a phone like yours right here and you're right, it only has one alarm.

Me: Would that be another user guide defect?

Rep: Yes, it seems so, I'm very sorry.

Me: "Sorry" doesn't change the fact that your defect rate is way too high here.  What happened to 6-sigma over there?

Rep: I'm very sorry.  I don't have any other information for you.

. . .

I was rather quite disgusted and was halfway out the door to my local store to return the device by the time the phone was back on hook.

After all, I'd become accustomed to copy/cut and paste on a PDA for over 10 years.  What sort of PDA doesn't have cut and paste?  How is that a business device as it's intended to be sold and used?  No copy/cut and paste from email, text messages, or the web?  Furthermore, it seemed that the manufacturer *did*, in fact, expect that basic function to be shipped.  So what happened?

Clearly, there were errors in the user guide, in the representative's information, and inconsistencies with the device.  In CMMI, we'd find these defects avoided in strong practices in traceability, work product verification against requirements, technical data packages, releases/baselines, and configuration audits, and validation, among others.

Yet, I realized that none of my research included checking whether I could cut and paste, or even how many alarms the devices have.  In fact, they weren't among my requirements at all.  They were just assumed to be "basic" functions!

So, in the store I look at the device I ranked #2 in my previous research.  Lo-and-behold!  It has the exact same lack of functionality as the earlier device despite being from different manufacturers!  What does that indicate? 

It's a supplier issue!  Both manufacturers must get the software from the same supplier!  Hence, the one company's expectation of functionality to the point of including it in their user guide and release notes only to later take shipment of the supplied wares without all expected features and functions!  What I then put together (in about a nanosecond) was that someone (sometimes someone tied with marketing?) promised to ship the product to the consumers but the product wasn't ready!  In fact, they expected certain features and functions but for some other reason the supplier of the software couldn't keep to the pre-defined delivery schedule!

(In any case, I ended up with my #3 choice, the one with the touch-screen I was trying to avoid.  I'm quite pleased with the improvements they've made since the previous version.  They've also improved on the form-factor.  And, as a bonus, I've already got plenty of accessories for it.  So I can save a little money there.)

Were there defects in the first attempted purchase?  Were these defects excusable?  Those aren't really the point to this post, are they?

The point is this: Requirements are tricky.  For lack of complete requirements a company has earned themselves a tarnished reputation, and I spent several hours being frustrated and having to take unscheduled time to deal with the error.  However, at least from my perspective, I could recover easily.  The iteration time and the correction were both fast.  In other words, I failed early and adjusted.

Were my requirements unreasonable?  Probably not.  But the episode highlights the challenge with requirements and the benefits to being flexible, iterative, and able to correct quickly.  Customers or developers who believe anyone will have all the requirements right and complete -- including all the assumed and derived ones -- are fooling themselves.  Witness the supposedly 6-Sigma manufacturer who "froze" the requirements based on what they thought they'd get and now they've got a black eye.  Whereas for myself, I iterated, and got the right product after all.

As seen elsewhere...

A recent thread over on the extremeprogramming Yahoo! group delved into whether or not CMMI sucks. One sub-thread was orbiting on the topic of Generic Practices.

As some folks know, the Generic Practices are what lead to the "institutionalization" of process improvement. In discussing this concept, the following lengthy (in text) but concise (relative to studying CMMI) explanation was given as a way to understand "institutionalization" by understanding the "Capability Levels" in CMMI.

The full post is here. The relevant text follows with small edits indicated by []s:

=-=-=-=-=-=
"Institutionalization", besides being a ridiculously long word, refers to the depth to which you have knowledge of your process. Institutionalization also often implies the extent to which your processes are ingrained into your organization, but really, when you look at what institutionalization involves it's more about how well you know your processes, not how widespread any given process may be throughout your organization.

At the lowest level at which anyone gets any 'credit', "institutionalization" is hardly the term appropriate for the state of the process. This is "level 1" where the process gets done, but by no means is it something you'd say any forethought, planning, or commitment was put forth into getting the process done.

The next level (level 2) is where we start to see some concerted effort towards treating the process as though it's something we cared about.

We see that the process is something someone in charge wants to be done (a.k.a. "policy"), we see that we know what tasks are involved in executing the process (a.k.a. "plan"), we see that resources have been allocated towards doing these tasks and that the tasks have been assigned as someone's responsibility.

If training for the project's execution of the process is needed, that's done at this level as well. We'd also expect that we'd see the outputs of the process as something we cared about so we'd control the outputs so that we could appropriately version, find, and update those outputs over time.

Given how much we've already invested in this process, it makes sense then to involve those folks who hold a stake in the outcome and to monitor the process' progress and activities, making changes to the plans, scheduling, or resources as needed to keep the process rolling.

We'd also want to keep tabs on whether the process is meeting the objectives of why we wanted the process done in the first place. And, finally, we'd review all of these process-oriented activities with people who can make decisions about the cost/ benefit/ value/ funding/ resources associated with the process fairly regularly over the life of the project.

These activities comprise what CMMI calls a "managed" process. An organization needs to know what process it's going to follow and what makes up that process if it's going to manage it. Thus comes the notion that the process is "institutionalized" as a "managed" process. We know enough about the process to manage it.

Beyond this level are 3, 4, and 5. Sometimes it's easier to understand "why" level 3 by looking at levels 4 & 5 first. At level 5 you know enough about your process that you can optimize it by eliminating the "noise" in the process.

A noisy engine can often be quieted by simply tuning it. Adjusting fuel, air, timing. But there's nothing outside the engine that's causing it to be noisy, it's just the engine itself. A noisy engine usually means inefficiency. The noise is just a symptom of that inefficiency. The same is true for processes. But in processes, true noise elimination is something that can realistically only be done mathematically. So, at level 5, the noise is found and reduced using models and statistics. Noise usually isn't spread all over the process, it's usually limited to some particular subset of the process. Usually, it's just some sub-process of the process on which statistics are applied.

Before you can get to this point, however, you must first be able to eliminate (or at least control) external factors that unnecessarily influence your process. This isn't "noise" because noise comes from the process, just like in an engine. And, just like in an engine, this is more like a rattle or a knocking sound, or even blunt-force damage. Something is either very broken or badly impacted by something related to, but not in control of, the engine. [In other words, the engine/process in not fully in control.] But, unless we know what the engine is expected to look like and operate we don't really know where to look to eliminate the issue. We need (with engines) the engine's shop manual which includes its diagrams and models. With processes, it's the same.
[]
We need to be able to model them before we can determine what's supposed to be there and what's not. [I.e., we need to know what an "in control" process looks like and what it's capable of doing.] The engine shop manual has performance specifications, and so do the processes at level 4. Capability Level 4 produces the models and performance expectations for each process as well as for the project itself. Without these we can't get to level 5 because, while there's certainly noise in the system at level 4, there are also too many other special causes of variation [let alone whether or not the process is in control] that must be eliminated before we can start to optimize in level 5.

Together, levels 4 & 5 are very much parallel to what many people know today as "Six Sigma".

So, now there's level 3. What's in there? If levels 4 & 5 are about getting to the point where we know so much about our processes that we can use statistics to eliminate process variation and noise, then capability level 3 must be where we eliminate chronic waste. How do we discern the chronic waste from the "necessary" activities? Well, we must first define the process so that we can then improve it.

There's no point in trying to improve a process that's not defined, and, there's no point in trying to define a process that's not even managed, and no point in trying to manage a process that no one does, wants, or needs.

This is what the generic practices of CMMI do. They create an infrastructure to better understand the process toward the ability to optimize it. Starting with doing the process, then managing it, then defining and improving it, then getting into statistics to model and predict performance which ultimately opens the door to optimization.

Believe it or not, organizations at (true) levels 4 & 5 are highly agile. They can pretty much roll with anything that's thrown at them. True level 4 & 5 organizations are NOT straight-jacketed by their processes, they're actually freed by them. If anyone is in (or has been in) a so-called "level" 4 or 5 organization and felt stifled, I'd wager the organization was not really "in it" for the improvements.