Nature doesn’t design upfront, nor should software engineers

Creationists (‘Intelligent Design’ proponents, whatever) argue that since life is too complex to have arisen by chance, there must be a God (or designer). They argue that the alternative to design is chance and that is just too improbable – it is like climbing an improbably tall, steep mountain in a single bound.


Or put another way, creationists only see two options: pure, blind chance on the one hand; and God on the other. Of course this premiss is disingenuous, Darwin demonstrated a third option – evolution by natural selection which supplies a strongly directional mechanism that is anything but random. Or as Richard Dawkins puts it: natural selection provides a way to navigate the gentle slopes up mount improbable, without the need to invoke the God Hypothesis.

Natural Selection provides a feedback loop to allow incremental changes that ratchet change up the slopes of mount improbable via a series of iterations. With each iteration being evaluated (by the environment) and either selected for, in which case it proceeds to the next iteration, or against in which case the change fails to be passed on the the next generation.

The approach of incremental improvement, ratcheting up the slopes of mount improbable, not only provides a mechanism or process to scale the slopes but also reduces the overall complexity of the system, by partitioning the problem into independent steps.

For example take three six-sided dice. The complexity of this system (the total number of possible states) is 63 or 216, this is 36 times more complex than a single six-sided die (which has six possible states). Now consider dividing the system into three partitions i.e. dealing with each die independently of the other two; the complexity of this system is the complexity of the first, plus the second, plus the third, or 6+6+6=18. Thus by partitioning a system into independent subsystems the overall complexity of a system is significantly reduced.

Partitioned dice

Now it strikes me that there are parallels here with software engineering. Software products are complex systems and attempting to climb a software mount improbable in a single bound seems to be foolhardy. Why attempt to design the entire system, at the start of the project, since it foists additional, unnecessary, complexity onto your project; it is safer to break your project into independent partitions.However, we need to be careful to try and ensure that each iteration (e.g. sprint) is independent of other iterations, otherwise we haven’t reduced our complexity (although this can in practice be difficult). And we should also ensure that we learn what has worked, and what hasn’t, at the end of each iteration and role these lessons into the next cycle of work.

It’s too easy to fail to genuinely review and learn from each iteration since we don’t like criticism and we are normally unwilling to throw away a month’s work (or part thereof) and instead continue to march blindly ahead. And that is blind faith, hoping that things will change on their own rather than continuing to move towards solutions that work and away from those that don’t.

6 responses to “Nature doesn’t design upfront, nor should software engineers”

  1. Are you suggesting the Flying Spaghetti Monster used Scrum?

  2. I like the idea that the Great Noodly One is in fact a Scrum Master; but I’m really suggesting that software should learn from the principles of evolution i.e. partitioning a project into independent subsystems and critical analysis of each iteration – progressing what works and killing what doesn’t.

  3. Well done interesting post. thanx for sharing R.

  4. […] indeed more unk unks. A project that is three times as big isn’t three times as complex – its 12 times more complex, at least. As Steve McConnell puts it, just because you can build a doghouse doesn’t mean you […]

  5. […] of events and the time given to different aspect of the product development life cycle. Agile’s iterative development cycle is its key strength, but it also makes for some tight deadlines and people often question whether […]

  6. […] wrong the company will lose more money) and the rate of occurrence because big projects are more complex and are more likely to contain unk-unks. The idea of a Hudson Bay start is to expose some of those […]

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: