Hudson Bay Start – reducing project risk

The Hudson’s Bay Company (HBC) was founded by King Charles II in 1670 and that makes it the oldest commercial company in the English-speaking world. For centuries the HBC controlled the fur trade throughout much of British-controlled North America, in an area known as Rupert’s Land, undertaking early exploration and functioning as the de facto government in many areas of the continent.

Hudson Bay Company traders

Fur trapping and exploration in Rupert’s Land in the 1600s must have been a high risk job to say the least. The trappers would have to survive on their own in the wilderness so having the right supplies in the correct amounts was literally a matter of life and death. To minimise this risk the trappers developed a technique known as the ‘Hudson Bay Start’.

The Hudson Bay Start was a test. A test to make sure the trappers had the right equipment before they headed off into the wilderness. They would pack up as if they were going to leave for the season but only canoe a short distance and camp overnight to make sure they had neither forgotten anything, nor taken too much. They would then return home, correct any mistakes, before heading out for real. Clearly this approach didn’t come for free – the trapping season was quite short – so while it reduced risk it did present an opportunity cost and might be seen as having an impact on their shipping schedule. A similar approach can be applied to large software projects.

Large scale software projects are high risk. The bigger the project the higher the risk, both in terms of impact (if it goes wrong the company will lose more money) and the rate of occurrence because big projects are more complex and are more likely to contain unk-unks. The idea of a Hudson Bay start is to expose some of those unk-unks and assumptions and therfore reduce some of your project risks, as Jerry Weinberg explains:

We take a trivial problem – really trivial like putting HELLO WORLD on a single screen, and run it through the entire development process – tools, people, documents, test plans, testing, or whatever they’re planning to use. It’s amazing the things that shake out in an hour or two. What’s even more amazing is how many clients are reluctant to do this – “It’s a waste of time; we’ve got an ambitious five-year project and we can’t spare a half day on an exercise.”

This is similar to spikes and tracer bullets in agile methodologies like Scrum. Except with tracer bullets you are working on a narrow implementation of a larger user story to help fine tune what needs to be built, helping you aim more accurately; and with spikes you are building quick and dirty throw away code to gain knowledge. Whereas with a Hudson Bay start you’re less interested in the actual product and more interested in making sure the development and deployment process works.

Scrummaging for unk unks

In 2002 Donald Rumsfeld was ridiculed for his infamous “known unknown” speech. However, despite the ridicule he was in fact quoting an accepted project management theory. That of managing unk-unks.

unk-unks n. especially in engineering, something, such as a problem, that has not been and could not have been imagined or anticipated; an unknown unknown.

Reports that say that something hasn’t happened are always interesting to me, because as we know, there are “known knowns”; there are things we know we know. We also know there are “known unknowns”; that is to say we know there are some things we do not know. But there are also “unknown unknowns” — the ones we don’t know we don’t know.’

When NASA were working on the Apollo programme they realized that, despite their considerable expertise, they had a lot of new problems to solve – including some problems that hadn’t been identified. And because they knew that once the astronauts were in space/on the moon it would be too late to discover they hadn’t thought of something NASA coined the term unk-unk. The engineers then focused on identifying all the unk-unks and solving them before blast off.

With most software projects you are also faced with unk-unks. After all if you’re not solving a new problem why are you writing code? So what should you do about it?

Well you really only have two options. Either, you hedged against by developing on parallel lines and hope that one works out. This is the approach many designers take – they prototype lots of ideas knowing that only one will be picked, and no one can predict which one before all have been seen. Alternatively you can manage the unk unks as they emerge, by changing the approach in response to new information.

If you want to take this second approach then the team needs to be empowered to solve problems en route (rather than only being allowed to build to an existing design and plan). However, in my experience the senior management and/or product owner must also be intimately involved in the project, have sufficient knowledge to check the plausibility of changes, and be supportive of the mid-course corrections.

The project also need to be managed and structured to allow this to happen. And this means the project needs adopt an adaptive, agile methodology, such as Scrum. So the project team are encouraged to solve problems in little steps as they arise and adapt as they learn more.

Scaling up agile projects

Agile methods don’t scale, if you are embarking on a big serious project you had better adopt a ‘heavyweight’ method. That way you have certainty, control over your resources and an agreed plan. Right? Wrong.

Agile methods promote using small teams because it improves communications. With improved communications the team members are more likely to understand the problem they are being asked to solve and therefore the proposed solution. Also when the project requirements change the team can more easily adjust. But what about large projects? How can small teams can deliver big projects?

The answer? Divide the big project into small independent projects. Nothing too shocking about that. But there is still a question about how to organise the work and structure the project. Earlier in the year I attended Spa 2007 where Dave Thomas outlined one approach.

The objective is to let projects have freedom to create the solutions to specific problems but to coordinate activity between projects. The approach is to structure the work into four phases:

  • Envisioning;
  • Definition;
  • Development and;
  • Release.

Large Scale Projects

Envisioning

The envisioning team is tasked not with capturing requirements but rather with understanding the problem to be solved, defining the solution and finally building an executable prototype. The work is managed with Scrum but because the purpose is to build a prototype that meets the users’ needs the team works very closely with end users developing their solutions iteratively: adding fidelity and refining the design at each iteration.

If appropriate two envisioning teams can run in parallel developing ‘competitive’ ideas to explore different options.

Definition

The Envisioning Team provides a clear definition of what needs to be built; the Definition team designs how to build it. To do this the Definition Team is tasked with structuring the work and defining the initial Product Backlog. The aim is to reduce dependencies: not just between components, but between the overall requirements and the component designs. The work then is structured into a series of discreet teams:

  • Feature teams (user facing services)
  • Component Teams (core reusable components)
  • Release Engineering (continuous integration and QA)

Feature teams can be responsible for no more than defining the unit tests for component sets – or integrating components with service specific code to provide specific functionality and/or glue sets of components together.

The Release Engineering team is responsible for ensuring that all the components and features work and conform to the agree interoperability framework. They therefore take each release and ensure the ‘system’ works, rather than trying to configure the system at the end of the project.

NB Components can be built out of phase with the rest of the project.

Development

The first job for each of the development teams is to review the product backlog to ensure that the project estimates are OK and the scope is understood.

If any product backlog item can’t be delivered or if there are any project dependencies these get escalated to the ‘risk backlog’ – which is owned by the Product Owner. Its the Product Owners responsibility to clear the Risk Backlog and not allow the individual projects to be effected.

Release

Code is packaged, documentation written, marketing etc.

And of course throughout this whole process everything is kept in the code: code, requirements, risks, people, estimates…