Reintroducing "engineering" to software.

I've been noticing an interesting "convergence" going on in this corner of the universe:  "engineering" is being re-introduced to the idea of software.  It's fascinating how no sooner do I have the idea to write this blog (while entirely not connected to the Internet), then I re-connected to get an article that's entirely speaking of the same root issue.

Of course, I'm not saying that engineering had completely left the software universe, though, a strong argument can be made that for the last decade and then some, software has allowed engineering to escape from the premises.  In particular, architecture, analysis, systems thinking, design, hardware and other integration issues, and planned, deliberate, methodical testing have largely been allowed to merely "emerge" from the work that was completed.

Again, I'm not advocating for Big ____ Up-Front as a solution, what I'm pointing out is that many people who embrace agile methods incorporate engineering practices into how they organize and perform their work, but enough don't that it raises issues with agile scalability.

And here's where I *am* expecting to annoy some people...

Programming and Development are NOT the same.

Development is an engineering function.  Developers ought to be using engineering practices in what they do.  Just look at the word "development".  The connotation is that something is "grown" or "evolved".  The denotation of 'development' in the technical sense is that it is done deliberately, not by happenstance.

This idea is where I believe software, in general, not limited to agile practices have short-changed themselves.  Too often, activities that amount to nothing more than programming are called development when no actual engineering is happening.  In other words, programming is allowed to take place without any (or at best without enough) engineering, and therefore what's really happening is the building of something without any/enough forethought about the thing itself that is to be built.  Instead, what happens too often is all the focus is on "staying busy" (albeit on ostensibly priority work), but what is worked on is absent sufficient technical rhyme or reason.

Is this true of all agile development?  NO.  But, it is what happens in many organizations when they don't have sufficient technical leadership.  For what it's worth, many development projects don't need much engineering, and product development is sufficiently described in tasks defined by few people.  So the jump from development to programming is small and fast. 

However, there are projects (or tasks) of sufficient technical complexity that skipping the engineering and handling such projects/tasks as programming alone is where I believe a space is created for the unfair reputation for agile and its scalability, as well as some of the anti-process bias among agile proponents.  When I read (and sometimes contribute) to agile and non-agile software groups, I'm often struck by the same thought: where's this person coming from?  This is basic engineering!

But that's the matter, isn't it?  Programming isn't development without engineering and too many programmers aren't engineers (not should they be) but are being told to "develop" without given the time or resources or something to do the engineering.  And so what they're really being told to do is "program", not "develop".  Someone, somewhere doesn't see and/or understand that what many projects need are to be engineered.

I think what this points to is a persistent phenomenon plaguing software: it's not being taken seriously as an engineering discipline.  Sometimes by leadership in organizations where software is being worked on, sometimes by programmers and sometimes by customers.  I'm sure there's plenty of blame to spread around, and spreading the blame is both a waste of time and not the point at all.

Programming is to software as assembly is to construction.  Not everyone swinging the hammer needs to be the civil engineer nor the architect, and not everyone with a nail gun can be the foreman (and no, I'm not likening the skill set of programmers to those of construction workers, and no, I'm not saying construction workers aren't smart....geez).  There has to have been engineering taking place before software can be actually developed, and as evinced by the kinds of challenges I encounter regularly, enough software shops are going about their work absent acknowledgement or awareness or consideration for the engineering that has taken place or has yet to take place (or should have taken place but didn't[!]).

Process stuff generally finds its roots in engineering.  Especially process stuff as found in CMMI for Development.  Excepting processes that are over-engineered, are themselves lacking in engineering, or are odious even alone in a room, I'm beginning to piece together that resistance to processes in general, and CMMI in particular, is actually from a lack of engineering discipline in the software practice and not from anything intrinsic to process as a topic.

It's no wonder CMMI is so hard to use by so many, it assumes people are not only experts in process improvement, it also assumes everyone using it is an engineer.  Some people are nail-gun swingers, worried about getting enough done that day to avoid having to work on the weekend.  Meanwhile, someone else already worried about in what order to build piece the trusses together and someone before that worried about the right number of trusses and their thickness and someone before that worried about its shape.

It's becoming fairly clear that anyone fooling themselves into believing that agile advocates not doing an architecture at all, or a design at all, or other engineering activities at all are doing themselves a disservice.  In fact, I'd go so far as to say that once an architecture has been settled upon and once a design becomes clear, that agile practices can happen more freely and effectively.  More so, I'd assert that the future of agile "scalability" depends on these.

What I and a colleague are setting out to do over the next few months is help agile scale by re-introducing engineering to software, and while we can't fix the software universe, we hope to help agile out by giving it some engineering practices that software (as a whole) lacks -- not everywhere, just in too many corners -- but that we believe agile can really take and run with.

Let me know if you want to play with us.

Another Lesson from my Not-Yet-6-Year-Old

 

My not yet 6 year old knows the difference between "design" and "build".

The other day he was discussing train track construction with my wife and one might think what I overheard is a result of me teaching him my trade.  But it wasn't.  I have no idea where he got this from, but I know it wasn't from me.  Nonetheless, I'm glad he actually knows the difference. 

"Design", he says, "is just when you're doing it on paper."

"Building is when you're actually making it."

"First I do a design so that I know what I want it to look like.  Then I build it."

I could have cried.  Instead, I just said, "I need to blog that!"

So later when I went down to look at the results I asked, "Is this how you designed it?"

He said, "no, I had to work around all this mess.  I didn't feel like putting all these toys away, so I built around them all."

The lesson here is for all of us.

Just because we know the real thing will likely end up differently when we go to build, doesn't de-value what we gain from the exercise of taking the time to design first.

Your "design" might be a test or a feature, but it came first and then reality caused adjustments.

That's how engineering works.  Engineering that doesn't iterate, increment, test and adjust is engineering that will fail.

You can't improve what you don't have.

In a recent meeting at a client, a very intelligent conversation took place among seasoned process professionals about their own process improvement efforts. This conversation helped crystallize a thought in a way that's so simple, merely stating it comes across as being so obvious as to leave one wondering why I'd mention it.

So many implementations of CMMI become so NON agile and so bureaucratic simply because when setting out to use CMMI, the organization doesn't have processes/ procedures/ standards of their own, and endeavor (whether knowingly or not) to use CMMI as the definition of their processes rather than as the model to improve their processes.

This same misapplication of CMMI can be blamed for so many organizations (and individuals) perceiving CMMI as being a process method or development standard. Certainly, this is what CMMI becomes when processes are defined by it, rather than improved by it.

It's this simple: organizations need to have processes before using CMMI to improve them! Of course, if an organization doesn't have their own processes, it's a great opportunity to create really great ones when they build the improvement activities (a la CMMI) into their processes while they're designing/ constructing those processes.

This is what we end up doing with most of our clients, only we're very lucky. Most of our clients don't need us to discover their processes while we're at it, they just have us coach them as to how to re-factor their processes with CMMI as an ingredient.