Cargo cults, competence and imposters

Way back in 2000 Steve McConnell wrote an article called Cargo Cult Software Engineering. If you’ve never read it, I highly recommend it.

(Note: The late 90s and early 00s were a time of much debate around the ‘best’ way to develop software. It was essentially an agile vs structured Software Development Method (SDM) argument. McConnell’s article is couched in the context of that debate, which actually still goes on today, and the debaters are still missing the point, which he makes so well.)

McConnell’s article opens with a delightful quote from the astonishingly amazing and sadly-missed (yeah, I’m a fan) Richard Feynman, thus:

In the South Seas there is a cargo cult of people. During the war they saw airplanes with lots of good materials, and they want the same thing to happen now. So they’ve arranged to make things like runways, to put fires along the sides of the runways, to make a wooden hut for a man to sit in, with two wooden pieces on his head for headphones and bars of bamboo sticking out like antennas—he’s the controller—and they wait for the airplanes to land. They’re doing everything right. The form is perfect. It looks exactly the way it looked before. But it doesn’t work. No airplanes land. So I call these things cargo cult science, because they follow all the apparent precepts and forms of scientific investigation, but they’re missing something essential, because the planes don’t land.

Process vs commitment – structured vs agile

McConnell goes on to describe how software development organisations are either process-oriented (e.g. the more highly-structured SDMs like RUP) or commitment-oriented (e.g. the agile SDMs like eXtreme Programming and Scrum.)

He then describes process imposter and commitment imposter organisations:

[Process imposter] organizations look at process-oriented organizations such as NASA’s Software Engineering Laboratory and IBM’s former Federal Systems Division. They observe that those organizations generate lots of documents and hold frequent meetings. They conclude that if they generate an equivalent number of documents and hold a comparable number of meetings they will be similarly successful.

[Commitment imposter] organizations look at successful companies like Microsoft; observe that they generate very little documentation; offer stock options to their employees; and then require them to work mountains of overtime. They conclude that if they, too, minimize documentation, offer stock options, and require extensive overtime, they will be successful.

Having the wrong argument

It should now be clear where the parallels to Cargo Cults come from. The people in both types of (ostensibly diametrically-opposite) imposter organisations are demonstrating the correct outward behaviours – they’re pulling the right levers, wearing the right gear and sitting in the right place. But they don’t get the underlying reasons why they are doing what they’re doing. And the planes don’t land.

Many of their projects end up crashing because these are just two different varieties of cargo cult software engineering, similar in their lack of understanding of what makes software projects work.

I’ll finish with McConnell’s final words:

We should not be debating process vs. commitment; we should be debating competence vs. incompetence. The real difference is not which style is chosen, but what education, training, and understanding is brought to bear on the project.

Rather than debating process vs. commitment, we should be looking for ways to raise the average level of developer and manager competence. That will improve our chances of success regardless of which development style we choose.

My 2 cents…

Steve McConnell seems to view the dangers of process and commitment imposters as being pretty much equal. I believe that the chances of doing agile wrong (aka being Fragile) are greater than the chances of messing up in a more structured environment. I already touched on that in this post. My next post will start to build on that premise.

Footnote

If it sounds like I’m anti-agile, that’s absolutely not the case. I’m very much an advocate of agile practices, as long as they’re used properly. Horses for courses, remember? I am however against their use when the people using them don’t know what they’re about.

Structured processes are more forgiving, and provide a good safety net. Treat structure like a pair of training wheels on a bicycle. You can remove them when you’re competent enough to balance without getting too grazed.