Hudson Bay Start – reducing project risk

The Hudson’s Bay Company (HBC) was founded by King Charles II in 1670 and that makes it the oldest commercial company in the English-speaking world. For centuries the HBC controlled the fur trade throughout much of British-controlled North America, in an area known as Rupert’s Land, undertaking early exploration and functioning as the de facto government in many areas of the continent.

Hudson Bay Company traders

Fur trapping and exploration in Rupert’s Land in the 1600s must have been a high risk job to say the least. The trappers would have to survive on their own in the wilderness so having the right supplies in the correct amounts was literally a matter of life and death. To minimise this risk the trappers developed a technique known as the ‘Hudson Bay Start’.

The Hudson Bay Start was a test. A test to make sure the trappers had the right equipment before they headed off into the wilderness. They would pack up as if they were going to leave for the season but only canoe a short distance and camp overnight to make sure they had neither forgotten anything, nor taken too much. They would then return home, correct any mistakes, before heading out for real. Clearly this approach didn’t come for free – the trapping season was quite short – so while it reduced risk it did present an opportunity cost and might be seen as having an impact on their shipping schedule. A similar approach can be applied to large software projects.

Large scale software projects are high risk. The bigger the project the higher the risk, both in terms of impact (if it goes wrong the company will lose more money) and the rate of occurrence because big projects are more complex and are more likely to contain unk-unks. The idea of a Hudson Bay start is to expose some of those unk-unks and assumptions and therfore reduce some of your project risks, as Jerry Weinberg explains:

We take a trivial problem – really trivial like putting HELLO WORLD on a single screen, and run it through the entire development process – tools, people, documents, test plans, testing, or whatever they’re planning to use. It’s amazing the things that shake out in an hour or two. What’s even more amazing is how many clients are reluctant to do this – “It’s a waste of time; we’ve got an ambitious five-year project and we can’t spare a half day on an exercise.”

This is similar to spikes and tracer bullets in agile methodologies like Scrum. Except with tracer bullets you are working on a narrow implementation of a larger user story to help fine tune what needs to be built, helping you aim more accurately; and with spikes you are building quick and dirty throw away code to gain knowledge. Whereas with a Hudson Bay start you’re less interested in the actual product and more interested in making sure the development and deployment process works.

Making Agile User Centered

To integrate UX activities into an agile software development process can be tough. Just consider this discussion between Kent Beck the creator of Extreme Programming and Alan Cooper a leading proponent of Interaction Design.

I’ll divide what Alan is talking about into two things: a set of techniques, and the larger process into which they fit. While I’m 100 percent with the techniques themselves, I’m 100 percent against the process that he described for using them. The techniques are optimized for being thoughtful in a cognitively difficult, complicated area where you’re breaking new ground, and the thinking that’s embedded in the practices is absolutely essential to doing effective software development.

The process, however, seems to be avoiding a problem that we’ve worked very hard to eliminate. The engineering practices of extreme programming are precisely there to eliminate that imbalance, to create an engineering team that can spin as fast as the interaction team. Cooperesque interaction design, I think, would be an outstanding tool to use inside the loop of development where the engineering was done according to the extreme programming practices. I can imagine the result of that would be far more powerful than setting up phases and hierarchy.

To me, the shining city on the hill is to create a process that uses XP engineering and the story writing out of interaction design. This could create something that’s really far more effective than either of those two things in isolation. (Kent Beck)

The techniques that Cooper proposes seek to define the behavior of software products “from the point of view of understanding and visualizing the behavior of complex systems, not the construction of complex systems.” He views “interaction design as part of the requirements-planning or product-planning process rather than as something that comes after the product planning is done.”

The User

Despite the fact that its tough to integrate agile and user centered design approaches it is possible because there are similarities – most notably the desire to create useful software. Take, for example, Alistair Cockburn’s comparison of agile development to a cooperative game:

Software development is a (resource-limited) cooperative game of invention and communication. The primary goal of the game is to deliver useful, working software. The secondary goal, the residue of the game, is to set up for the next game.

And as the name suggests, user centered design is about placing people, users of a piece of software, at the centre of the design process – making software that is useful to them – to do this effectively UX professionals seek to understand user goals and motivation through user research.

Or put another way user centered design helps us understand how people use our products, while agile development let us build and deliver software that is more relevant to its users. Both approaches are trying to deliver meaningful software products to people.

The problem then is not because the two approaches are trying to achieve something different but rather the way of achieving it is different; and specifically the sequencing of events and the time given to different aspect of the product development life cycle. Agile’s iterative development cycle is its key strength, but it also makes for some tight deadlines and people often question whether there is sufficient time to fully consider the users’ needs? However, as Alan Cooper suggests, the time to define the user’s needs is not during the development but before the development starts.

I’ve written previously about using an envisaging team to scout out ahead. Not to design the solution but to understand the problem space – understand what the business needs, what users need, who the users are and what the technology ecosystem looks like – and report this back in a coherent fashion to the rest of the team. This isn’t Big Design Up Front by another name. The envisaging team’s work should be limited to discovery work only. This combined with regular rounds of user testing between each iteration – the results of which are fed into the next round of development – can mean that agile software development can be highly user focused.

Photo: The User, by Pete Ashton. Used under licence.

Living with Code

I’ve been responsible for projects of widely different sizes and complexity. From small production jobs – with no real software development (just visual design and integration) – through to large scale application developments, which we’ve then need to live with, to maintain and develop. And big projects, especially products (that need to be reused), aren’t scaled up small projects. They need to be managed differently not just because the projects are more complex but also because you will need to be able to live with the code for years to come.

Developing a large application isn’t the same as developing a small application. For starters large projects are significantly more complex than small projects – they have more interdependencies, there are more unknowns, indeed more unk unks. A project that is three times as big isn’t three times as complex – its 12 times more complex, at least. As Steve McConnell puts it, just because you can build a doghouse doesn’t mean you can build a skyscraper.

People who have written a few small programs in college sometimes think that writing large, professional programs is the same kind of work-only on a larger scale. It is not the same kind of work. I can build a beautiful doghouse in my backyard in a few hours. It might even take first prize at the county fair’s doghouse competition. But that does not imply that I have the expertise to build a skyscraper. The skyscraper project requires an entirely more sophisticated kind of expertise. The difference in complexity between student programs and professional programs can be just as great, and non-professional programmers -underestimate the difference in required expertise at their own peril.”

And to make matters worse, if you’ve not attempted to build a doghouse, skyscraper or fort before it’s likely that your estimates will be rubbish – since its likely that when planning your work – you’ll miss off important tasks, underestimate unfamiliar tasks, that so of thing. Clearly having team members that have worked on similar applications before helps, as does adopting an agile project management approach.

Agile software development isn’t necessary with small projects – because its reasonable to assume the requirements are stable, there shouldn’t be any unk unks and you’ve probably built a dog house before. But as applications get bigger, as projects become more complex, those assumptions don’t hold true. And you need to able able to manage the new information as it arises. But that’s not the whole story.

The code also needs to be easily maintained by your development team – they need code they can live with – and that doesn’t mean the team that build the application, because if you are building a product that will be used, maintained and developed over a number of years the software engineers that end up looking after the code won’t be the people that wrote it. With small projects – the code can usually be treated as throw away code – there’s no desperate need to supported it.

Clearly some technologies make writing supportable code easier than others. For example, while there is nothing inherently wrong with PHP – indeed it has many advantages, its easy to learn and quick to develop with – its all too easy to end up with spaghetti code, because PHP promotes conflating the business logic and presentation – this can make it painful to maintain, extend or debug. As Tim Bray points out:

…based on my limited experience with PHP (deploying a couple of free apps to do this and that, and debugging a site for a non-technical friend here and there): all the PHP code I have seen in that experience has been messy, unmaintainable crap. Spaghetti SQL wrapped in spaghetti PHP wrapped in spaghetti HTML, replicated in slightly-varying form in dozens of places.”

Don’t get me wrong, I’ve nothing against PHP you can write maintainable PHP code. The trouble is its also easy not to – so depending on the experience of the team it might be prudent to chose a technology that promotes good practice. You will want to extend the application – to deal with new requirements – so architect your application to make it easier to extend.