Monday, March 20, 2006

Change is Hard

We’re going through an interesting phase in the project on which I’m currently working. The project itself is the survivor, or resurrection, of a project that, hard as people tried, could not be made to work. We’re now trying to come back around and make it work, and I’m learning many new things about people and projects from it.

There are plenty of reasons the project failed, almost none of them inherently technical. Business management tried to do it “on the cheap” by outsourcing everything, outsourcing far too much really. Then there were the usual problems of loose and sloppy requirements that were capable of infinite redefinition, tool selections that were too complex and so hurt more than helping. The real point, though, is that the project was effectively a failure, one in which we’re going back and restarting because the product is important and the market, amazingly enough, hasn’t been taken by another vendor.

Project management is doing the right things (and no, I’m not project management, I’m more the project gadfly, trying to keep the process honest.) They’ve decided that the right place to start is with a blank directory, and all previous decisions are available for rethinking. They’ve concluded to take a more incremental, iterative, and agile approach. They’re introducing rigorous unit testing, and rethinking every major decision, starting with the operating system and hardware platform for the project. (I’ll grant that being acquired by Sun has something to do with changing the platform, too.) They've recognized that the process they used was inherently too "heavy", took too much unproductive effort and generated too little value.

What’s interesting to watch, though, is this: no one thinks the old project was a success. None the less, the notion of actually changing the way things are done is almost unthinkable. Every time a question comes up, the instant reaction is to try to add more process, with more documentation and inherently more ways of spreading responsibility, and generally also to revert to the way things were done before.

Even though the way it was done before is the way the project failed last time.

It’s an interesting lesson. I’m inherently neophilic: if I hear of something new, I want to try it. I particularly hate repeating something that didn’t work the first time; I think “it didn’t work” is sufficient reason not to do it again.

A long time ago, when I was just a child, I read a book by a guy — Jack Douglas — who had been a writer for Jack Paar. Even as a precocious 10 year old, I thought the book was mostly hilarious, but Douglas also told a story that was so sad it makes me uncomfortable to think about today. Douglas had a fire, and while getting everyone out, including his dogs and cats, one of the dogs — a dachshund — got loose. They found him later, dead, under the bed in the master bedroom. In his fear, he had run back into the burning house, because he felt safe hiding under the bed.

How often do we find ourselves hiding under the familiar bed, and not seeing the real danger?

14 comments:

vnjagvet said...

STY:

Interesting story. It sort of reminded me of Edison's quote, "I have not failed, I have just found 10,000 ways that don't work".

Having found the ways that didn't work, he did not repeat them.

MeaninglessHotAir said...

Bravo! Best article I've read on here in quite a while.

MeaninglessHotAir said...

They've recognized that the process they used was inherently too "heavy", took too much unproductive effort and generated too little value.

I had a meeting with a bunch of folks from a certain tech company from the vicinity of Puget Sound tonight and discovered something that astonished me: they don't have "process". Their small development teams are required to ship, ship fast, and ship accurately. That's the whole process. How they accomplish that is their business. Amazing. It's controlled chaos really, similar to open source development, oddly enough, except with a deadline. And if they don't get it right the first time, they ship again.

I think "process" is way overrated.

gumshoe said...

MHA had some good comments.

this quote seems to fit
with what he said:

"Management by objectives works if you first think through your objectives. Ninety percent of the time you haven't."
- Peter Drucker
__________________________
here's my 2c

"Then there were the usual problems of loose and sloppy requirements that were capable of infinite redefinition, tool selections that were too complex and so hurt more than helping."

Q:who writes good requirements?

Q:why can't they fit on one side of one sheet of paper?

"Even though the way it was done before is the way the project failed last time."

Q: has everyone voiced their opinions on why
the previous project failed?

Q: why aren't those observations
an integral part of the new requirements?
(ie whay are you having to point this out?)

you mentioned an iterative approach:

"They’ve concluded to take a more incremental, iterative, and agile approach."

many times people don't know what they want until thay know what they clearly *do not* want.

you've had your first iteration
(the "failed" project)...

i'd mine it for what worked.

when people overinvest
in the first try,they're often reluctant to tear it down
and start over.

Unknown said...

seneca:

That is interesting. It does say something about human nature and our need for routine, sameness, stability, take your pick.

Barry Dauphin said...

If we tie this to the post above, sometimes the group has one chance to get it "right". Fitzgerald & Co. might not get another bite of the apple. They could think through what didn't work this time and fix it, but that will be for another case besides Libby (unless the judge finds a way to give them a Mulligan).

Eric said...

Hmmm..do you work where I work, Seneca? Sounds like it.

Most of the coding I do would be utterly trivial if not for the fact that the business users keep changing their minds on what they want.

Charlie Martin said...

gumshoe, I agree.

Mining the old project for what worked is word for word the metaphor I use.

On the other points, to some extent the reason I have to say this is that it's my job. I was originally hired as a consultant to survey the code and tell them how much trouble they were in (ans: lots.) Now I'v been told my role is to stand up and be a technical "thought leader" --- at least so far, my boss has actually been very supportive of my troublemaking.

But I wonder if I'm not learning a bigger lesson, too.

Rick Ballard said...

Why would a new team drawn from a pool of people trained in the same manner as the old team come up with a different solution process? If 'weight' was the problem with the initial solution, the implication is that material sourcing and design safety factors were probable culprits. "Too heavy does not mean "doesn't work", at least it wouldn't to design professionals in construction.

I suppose that's a problem of ineffective objective definition - which may be what is hanging the team at the moment. I'd run like a rabbit from being a 'thought leader' unless there were sufficient measures of authority in place to assure me that there were going to be some 'thought followers' charged with doing more than argue.

Charlie Martin said...

Sorry, Rick, I misled you: "heavyweight" in this case is a metaphor: too much process, to much paperwork being done per unit of useful work. This is a software product, it doesn't weigh anything really.

As far as your other question, there have been radical changes in the team, like firing management, getting rid of the outsourced development, and not least, hiring me. But the remaining people have essential knowledge of the problem domain and have to be keopt, as well as being valuable employees we'd like to retrain into the Good and True Path.

Rick Ballard said...

I wasn't clear in my reply. I was trying to say that 'process weight', that accretion of paper that bureaucracies love, cannot be eliminated using the same "pool" of employees - unless there is clearly perceived individual benefit to be derived from change. Like a big bonus or not being fired. The benefit has to be exhibited fairly early and fairly strongly (body under the bed) or the bureaucratic glacier will just keep grinding.

"But the remaining people have essential knowledge of the problem domain and have to be kept"

They all went on a sightseeing tour and the bus went off a cliff - what then?

gumshoe said...

seneca -

miles davis had a reputation for
many things...not all of them good.

in recording the widely acclaimed 1959 "Kind of Blue",
there were NO rehersals(and recording/studio time was *very* expensive then)...miles picked his
musicians,got them in the studio,
threw some *very* basic themes at them and hit "record".

this applies to what MHA noted and to an image of how to get the most for the skills you already have in your group:

-assemble skilled performers.

-identify what "performance is".

(and hang onto you hat).

it has the added benefit/feature
of demanding the same from *you*
(as "thought leader") as it does of your "thought followers":

performance.

"i'll play it first and tell you what it is later."

and in regard to you project:

"Don't play what's there,
play what's not there."

____________________________

OTOH -
what MHA said has another aspect:

the team MHA described doesn't have
a "formal method/process" and
yet he views that as key to their
performance and success....

...but a group who hasn't been exposed to that (non-formal)working "process" would need to be coached into it.

and,a "thought leader" who hasn't had some experience with that type of performance would need to
"crawl out from under the bed".

;0p

best of luck.

Charlie Martin said...

The benefit has to be exhibited fairly early and fairly strongly (body under the bed) or the bureaucratic glacier will just keep grinding.

Yep. Exactly where I am.

They all went on a sightseeing tour and the bus went off a cliff - what then?

Don't be a tease.

Rick Ballard said...

You could suggest the adoption of an offical project blog.