Interesting stuff from around the web 2008-09-01

Google Chrome, Google’s Browser Project
Google Chrome, Google’s Browser Project

Some Google news

Google Chrome, Google’s Browser Project [blogoscoped.com]
Built with WebKit but with added cool features – some UX stuff but more interestingly a new JavaScript virtual machine, multi-threading and sensible security policies. And all Open Sourced.

Google’s undocumented favicon to png convertor [via Simon Willison]
Showing the favicon of a domain next to a link is a really nice trick, but it’s slightly tricky to achieve as IE won’t display a .ico file if you link to it from an img element, so you need to convert the images server-side. This undocumented Google API does that for you, meaning it’s much easier to add favicons as a feature to your site.

Some cool hacking…

Solar powered ice maker [Hack a Day]
The system uses solar heat to facilitate evaporation of a coolant. When the sun goes down and the coolant turns back to liquid, its temperature drops drastically due to extreme pressure differences. The unit can produce 14 pounds of ice per day with zero carbon footprint.

Archrs – an everyday story of web development [BBC – Radio Labs]
Tristan’s review of his work on a new Archer’s prototype. Unfortunately it didn’t get launched – which is a real shame it would have been great.

And finally some project management stuff

The Need for Leadership in Scrum [NetObjectives]
An interesting take on the role of management and leadership within organisations using Scrum. Where leadership as the “ability to create a solid vision of a better future for those people he/she is leading.” A leader must have a compelling desire to move towards that vision. And management as the “ability to match people’s tasks with their skills.”

Managing the code-garden

Kevin Barnes suggests that software engineering is a bit like gardening – software is never finished – you need to spend some time planning, some time adding new features and some time tending to what you have. Otherwise your code will become steadily more and more unmanageable.

Artfully planned decay

Basically, code is like a garden. We lay it out, plant it and then tend and maintain it for as long as it continues to warrant the attention.”

The trouble is how do you make sure that you allocate enough time to maintenance while continuing to add new features or enhancements? How do you make sure you don’t allocate all your time to adding the next new thing, nor spend weeks on end tidying your code because all you’ve done recently is add new features?

Likewise how do you give the team the time and space to innovate – to try out their ideas? One solution is to let anyone add a new item to the product backlog or requirements catalogue and then prioritise it alongside everything else. Well this is OK if it’s a big idea, but not great for smaller items nor for more geeky ideas if the Product Owner or the business at large doesn’t understand the value of such things. It also feels wrong – if someone has a good idea that can be implemented quickly then they should be able to so, after all going via the Product Baacklog route may take more time than simply implementing the feature. And that is as good a way to kill off innovation as anything else.

But likewise you need to provide appropriate project governance – if everyone did whatever they wanted then the business is unlikely to get what it needs from the software.

One solution is the idea of Gold Cards [pdf] as suggested by Tim MacKinnon. Gold cards are designed to address:

A lack of innovation because the customer does not necessarily explore options that are technically possible but not currently required. Consequently, cutting-edge knowledge may be slowly lost from the team.

Gold Cards allow developers time to explore technical possibilities in a controlled and focused way.

In Scrum the current sprint’s work items are written on cards and pinned to the wall – so everyone knows what’s being worked on and everyone knows what’s completed. Gold Cards are special cards that let you work on your own idea – it can be anything you want so long as it’s on the current project.

Each developer is allocated two Gold Cards at the beginning of each calendar month… Gold Cards can be used at any time during a month, but cannot be carried over into the next month. […]

Each card grants the developer who has it, one day of work on a topic of their choice.

In a similar fashion ‘Gardening Cards’ let you work on whatever is bugging you – not bugs which should be prioritised elsewhere – but those things that just annoy you about the way something is implemented, that missing feature that would make life much easier if it were there. It’s your chance to spend time tending to your garden, not planting new features.

So the idea is that you place one Gold Card and one Gardening Card on the wall. Each sprint everyone can spend one day on a gold card item and one on a bit of gardening. But because the project only has one gold card and one gardening card only one person can be doing each activity at a time – everyone else is working on the backlog as normal. It’s the Scrum Master’s responsibility to encourage everyone to take the time to work on these items.

Now clearly how many days you allocate to gardening and gold card items will depend on a number of different factors: team size, sprint length, age of the project and the state of the code (you wouldn’t allocate gardening time at the start of a project for instance). But generally when a project matures you should be allocating some time every sprint to this. The Scrum Master can plan around this because although they don’t know exactly what everyone will work on she does know how much time will be allocated.

The other advantage of this approach is that they also help fill the gaps – if your planning is a bit off, you encounter some unforeseen problems and some member of the team are being held up then they can remain productive by working on their gold card idea or fixing that pesky item that’s being bugging them for while rather than waiting for someone else.

Hudson Bay Start – reducing project risk

The Hudson’s Bay Company (HBC) was founded by King Charles II in 1670 and that makes it the oldest commercial company in the English-speaking world. For centuries the HBC controlled the fur trade throughout much of British-controlled North America, in an area known as Rupert’s Land, undertaking early exploration and functioning as the de facto government in many areas of the continent.

Hudson Bay Company traders

Fur trapping and exploration in Rupert’s Land in the 1600s must have been a high risk job to say the least. The trappers would have to survive on their own in the wilderness so having the right supplies in the correct amounts was literally a matter of life and death. To minimise this risk the trappers developed a technique known as the ‘Hudson Bay Start’.

The Hudson Bay Start was a test. A test to make sure the trappers had the right equipment before they headed off into the wilderness. They would pack up as if they were going to leave for the season but only canoe a short distance and camp overnight to make sure they had neither forgotten anything, nor taken too much. They would then return home, correct any mistakes, before heading out for real. Clearly this approach didn’t come for free – the trapping season was quite short – so while it reduced risk it did present an opportunity cost and might be seen as having an impact on their shipping schedule. A similar approach can be applied to large software projects.

Large scale software projects are high risk. The bigger the project the higher the risk, both in terms of impact (if it goes wrong the company will lose more money) and the rate of occurrence because big projects are more complex and are more likely to contain unk-unks. The idea of a Hudson Bay start is to expose some of those unk-unks and assumptions and therfore reduce some of your project risks, as Jerry Weinberg explains:

We take a trivial problem – really trivial like putting HELLO WORLD on a single screen, and run it through the entire development process – tools, people, documents, test plans, testing, or whatever they’re planning to use. It’s amazing the things that shake out in an hour or two. What’s even more amazing is how many clients are reluctant to do this – “It’s a waste of time; we’ve got an ambitious five-year project and we can’t spare a half day on an exercise.”

This is similar to spikes and tracer bullets in agile methodologies like Scrum. Except with tracer bullets you are working on a narrow implementation of a larger user story to help fine tune what needs to be built, helping you aim more accurately; and with spikes you are building quick and dirty throw away code to gain knowledge. Whereas with a Hudson Bay start you’re less interested in the actual product and more interested in making sure the development and deployment process works.

Scrummaging for unk unks

In 2002 Donald Rumsfeld was ridiculed for his infamous “known unknown” speech. However, despite the ridicule he was in fact quoting an accepted project management theory. That of managing unk-unks.

unk-unks n. especially in engineering, something, such as a problem, that has not been and could not have been imagined or anticipated; an unknown unknown.

Reports that say that something hasn’t happened are always interesting to me, because as we know, there are “known knowns”; there are things we know we know. We also know there are “known unknowns”; that is to say we know there are some things we do not know. But there are also “unknown unknowns” — the ones we don’t know we don’t know.’

When NASA were working on the Apollo programme they realized that, despite their considerable expertise, they had a lot of new problems to solve – including some problems that hadn’t been identified. And because they knew that once the astronauts were in space/on the moon it would be too late to discover they hadn’t thought of something NASA coined the term unk-unk. The engineers then focused on identifying all the unk-unks and solving them before blast off.

With most software projects you are also faced with unk-unks. After all if you’re not solving a new problem why are you writing code? So what should you do about it?

Well you really only have two options. Either, you hedged against by developing on parallel lines and hope that one works out. This is the approach many designers take – they prototype lots of ideas knowing that only one will be picked, and no one can predict which one before all have been seen. Alternatively you can manage the unk unks as they emerge, by changing the approach in response to new information.

If you want to take this second approach then the team needs to be empowered to solve problems en route (rather than only being allowed to build to an existing design and plan). However, in my experience the senior management and/or product owner must also be intimately involved in the project, have sufficient knowledge to check the plausibility of changes, and be supportive of the mid-course corrections.

The project also need to be managed and structured to allow this to happen. And this means the project needs adopt an adaptive, agile methodology, such as Scrum. So the project team are encouraged to solve problems in little steps as they arise and adapt as they learn more.

Scaling up agile projects

Agile methods don’t scale, if you are embarking on a big serious project you had better adopt a ‘heavyweight’ method. That way you have certainty, control over your resources and an agreed plan. Right? Wrong.

Agile methods promote using small teams because it improves communications. With improved communications the team members are more likely to understand the problem they are being asked to solve and therefore the proposed solution. Also when the project requirements change the team can more easily adjust. But what about large projects? How can small teams can deliver big projects?

The answer? Divide the big project into small independent projects. Nothing too shocking about that. But there is still a question about how to organise the work and structure the project. Earlier in the year I attended Spa 2007 where Dave Thomas outlined one approach.

The objective is to let projects have freedom to create the solutions to specific problems but to coordinate activity between projects. The approach is to structure the work into four phases:

  • Envisioning;
  • Definition;
  • Development and;
  • Release.

Large Scale Projects

Envisioning

The envisioning team is tasked not with capturing requirements but rather with understanding the problem to be solved, defining the solution and finally building an executable prototype. The work is managed with Scrum but because the purpose is to build a prototype that meets the users’ needs the team works very closely with end users developing their solutions iteratively: adding fidelity and refining the design at each iteration.

If appropriate two envisioning teams can run in parallel developing ‘competitive’ ideas to explore different options.

Definition

The Envisioning Team provides a clear definition of what needs to be built; the Definition team designs how to build it. To do this the Definition Team is tasked with structuring the work and defining the initial Product Backlog. The aim is to reduce dependencies: not just between components, but between the overall requirements and the component designs. The work then is structured into a series of discreet teams:

  • Feature teams (user facing services)
  • Component Teams (core reusable components)
  • Release Engineering (continuous integration and QA)

Feature teams can be responsible for no more than defining the unit tests for component sets – or integrating components with service specific code to provide specific functionality and/or glue sets of components together.

The Release Engineering team is responsible for ensuring that all the components and features work and conform to the agree interoperability framework. They therefore take each release and ensure the ‘system’ works, rather than trying to configure the system at the end of the project.

NB Components can be built out of phase with the rest of the project.

Development

The first job for each of the development teams is to review the product backlog to ensure that the project estimates are OK and the scope is understood.

If any product backlog item can’t be delivered or if there are any project dependencies these get escalated to the ‘risk backlog’ – which is owned by the Product Owner. Its the Product Owners responsibility to clear the Risk Backlog and not allow the individual projects to be effected.

Release

Code is packaged, documentation written, marketing etc.

And of course throughout this whole process everything is kept in the code: code, requirements, risks, people, estimates…

Nature doesn’t design upfront, nor should software engineers

Creationists (‘Intelligent Design’ proponents, whatever) argue that since life is too complex to have arisen by chance, there must be a God (or designer). They argue that the alternative to design is chance and that is just too improbable – it is like climbing an improbably tall, steep mountain in a single bound.

Mountain

Or put another way, creationists only see two options: pure, blind chance on the one hand; and God on the other. Of course this premiss is disingenuous, Darwin demonstrated a third option – evolution by natural selection which supplies a strongly directional mechanism that is anything but random. Or as Richard Dawkins puts it: natural selection provides a way to navigate the gentle slopes up mount improbable, without the need to invoke the God Hypothesis.

Natural Selection provides a feedback loop to allow incremental changes that ratchet change up the slopes of mount improbable via a series of iterations. With each iteration being evaluated (by the environment) and either selected for, in which case it proceeds to the next iteration, or against in which case the change fails to be passed on the the next generation.

The approach of incremental improvement, ratcheting up the slopes of mount improbable, not only provides a mechanism or process to scale the slopes but also reduces the overall complexity of the system, by partitioning the problem into independent steps.

For example take three six-sided dice. The complexity of this system (the total number of possible states) is 63 or 216, this is 36 times more complex than a single six-sided die (which has six possible states). Now consider dividing the system into three partitions i.e. dealing with each die independently of the other two; the complexity of this system is the complexity of the first, plus the second, plus the third, or 6+6+6=18. Thus by partitioning a system into independent subsystems the overall complexity of a system is significantly reduced.

Partitioned dice

Now it strikes me that there are parallels here with software engineering. Software products are complex systems and attempting to climb a software mount improbable in a single bound seems to be foolhardy. Why attempt to design the entire system, at the start of the project, since it foists additional, unnecessary, complexity onto your project; it is safer to break your project into independent partitions.However, we need to be careful to try and ensure that each iteration (e.g. sprint) is independent of other iterations, otherwise we haven’t reduced our complexity (although this can in practice be difficult). And we should also ensure that we learn what has worked, and what hasn’t, at the end of each iteration and role these lessons into the next cycle of work.

It’s too easy to fail to genuinely review and learn from each iteration since we don’t like criticism and we are normally unwilling to throw away a month’s work (or part thereof) and instead continue to march blindly ahead. And that is blind faith, hoping that things will change on their own rather than continuing to move towards solutions that work and away from those that don’t.