Making Agile User Centered

To integrate UX activities into an agile software development process can be tough. Just consider this discussion between Kent Beck the creator of Extreme Programming and Alan Cooper a leading proponent of Interaction Design.

I’ll divide what Alan is talking about into two things: a set of techniques, and the larger process into which they fit. While I’m 100 percent with the techniques themselves, I’m 100 percent against the process that he described for using them. The techniques are optimized for being thoughtful in a cognitively difficult, complicated area where you’re breaking new ground, and the thinking that’s embedded in the practices is absolutely essential to doing effective software development.

The process, however, seems to be avoiding a problem that we’ve worked very hard to eliminate. The engineering practices of extreme programming are precisely there to eliminate that imbalance, to create an engineering team that can spin as fast as the interaction team. Cooperesque interaction design, I think, would be an outstanding tool to use inside the loop of development where the engineering was done according to the extreme programming practices. I can imagine the result of that would be far more powerful than setting up phases and hierarchy.

To me, the shining city on the hill is to create a process that uses XP engineering and the story writing out of interaction design. This could create something that’s really far more effective than either of those two things in isolation. (Kent Beck)

The techniques that Cooper proposes seek to define the behavior of software products “from the point of view of understanding and visualizing the behavior of complex systems, not the construction of complex systems.” He views “interaction design as part of the requirements-planning or product-planning process rather than as something that comes after the product planning is done.”

The User

Despite the fact that its tough to integrate agile and user centered design approaches it is possible because there are similarities – most notably the desire to create useful software. Take, for example, Alistair Cockburn’s comparison of agile development to a cooperative game:

Software development is a (resource-limited) cooperative game of invention and communication. The primary goal of the game is to deliver useful, working software. The secondary goal, the residue of the game, is to set up for the next game.

And as the name suggests, user centered design is about placing people, users of a piece of software, at the centre of the design process – making software that is useful to them – to do this effectively UX professionals seek to understand user goals and motivation through user research.

Or put another way user centered design helps us understand how people use our products, while agile development let us build and deliver software that is more relevant to its users. Both approaches are trying to deliver meaningful software products to people.

The problem then is not because the two approaches are trying to achieve something different but rather the way of achieving it is different; and specifically the sequencing of events and the time given to different aspect of the product development life cycle. Agile’s iterative development cycle is its key strength, but it also makes for some tight deadlines and people often question whether there is sufficient time to fully consider the users’ needs? However, as Alan Cooper suggests, the time to define the user’s needs is not during the development but before the development starts.

I’ve written previously about using an envisaging team to scout out ahead. Not to design the solution but to understand the problem space – understand what the business needs, what users need, who the users are and what the technology ecosystem looks like – and report this back in a coherent fashion to the rest of the team. This isn’t Big Design Up Front by another name. The envisaging team’s work should be limited to discovery work only. This combined with regular rounds of user testing between each iteration – the results of which are fed into the next round of development – can mean that agile software development can be highly user focused.

Photo: The User, by Pete Ashton. Used under licence.

Living with Code

I’ve been responsible for projects of widely different sizes and complexity. From small production jobs – with no real software development (just visual design and integration) – through to large scale application developments, which we’ve then need to live with, to maintain and develop. And big projects, especially products (that need to be reused), aren’t scaled up small projects. They need to be managed differently not just because the projects are more complex but also because you will need to be able to live with the code for years to come.

Developing a large application isn’t the same as developing a small application. For starters large projects are significantly more complex than small projects – they have more interdependencies, there are more unknowns, indeed more unk unks. A project that is three times as big isn’t three times as complex – its 12 times more complex, at least. As Steve McConnell puts it, just because you can build a doghouse doesn’t mean you can build a skyscraper.

People who have written a few small programs in college sometimes think that writing large, professional programs is the same kind of work-only on a larger scale. It is not the same kind of work. I can build a beautiful doghouse in my backyard in a few hours. It might even take first prize at the county fair’s doghouse competition. But that does not imply that I have the expertise to build a skyscraper. The skyscraper project requires an entirely more sophisticated kind of expertise. The difference in complexity between student programs and professional programs can be just as great, and non-professional programmers -underestimate the difference in required expertise at their own peril.”

And to make matters worse, if you’ve not attempted to build a doghouse, skyscraper or fort before it’s likely that your estimates will be rubbish – since its likely that when planning your work – you’ll miss off important tasks, underestimate unfamiliar tasks, that so of thing. Clearly having team members that have worked on similar applications before helps, as does adopting an agile project management approach.

Agile software development isn’t necessary with small projects – because its reasonable to assume the requirements are stable, there shouldn’t be any unk unks and you’ve probably built a dog house before. But as applications get bigger, as projects become more complex, those assumptions don’t hold true. And you need to able able to manage the new information as it arises. But that’s not the whole story.

The code also needs to be easily maintained by your development team – they need code they can live with – and that doesn’t mean the team that build the application, because if you are building a product that will be used, maintained and developed over a number of years the software engineers that end up looking after the code won’t be the people that wrote it. With small projects – the code can usually be treated as throw away code – there’s no desperate need to supported it.

Clearly some technologies make writing supportable code easier than others. For example, while there is nothing inherently wrong with PHP – indeed it has many advantages, its easy to learn and quick to develop with – its all too easy to end up with spaghetti code, because PHP promotes conflating the business logic and presentation – this can make it painful to maintain, extend or debug. As Tim Bray points out:

…based on my limited experience with PHP (deploying a couple of free apps to do this and that, and debugging a site for a non-technical friend here and there): all the PHP code I have seen in that experience has been messy, unmaintainable crap. Spaghetti SQL wrapped in spaghetti PHP wrapped in spaghetti HTML, replicated in slightly-varying form in dozens of places.”

Don’t get me wrong, I’ve nothing against PHP you can write maintainable PHP code. The trouble is its also easy not to – so depending on the experience of the team it might be prudent to chose a technology that promotes good practice. You will want to extend the application – to deal with new requirements – so architect your application to make it easier to extend.

Scrummaging for unk unks

In 2002 Donald Rumsfeld was ridiculed for his infamous “known unknown” speech. However, despite the ridicule he was in fact quoting an accepted project management theory. That of managing unk-unks.

unk-unks n. especially in engineering, something, such as a problem, that has not been and could not have been imagined or anticipated; an unknown unknown.

Reports that say that something hasn’t happened are always interesting to me, because as we know, there are “known knowns”; there are things we know we know. We also know there are “known unknowns”; that is to say we know there are some things we do not know. But there are also “unknown unknowns” — the ones we don’t know we don’t know.’

When NASA were working on the Apollo programme they realized that, despite their considerable expertise, they had a lot of new problems to solve – including some problems that hadn’t been identified. And because they knew that once the astronauts were in space/on the moon it would be too late to discover they hadn’t thought of something NASA coined the term unk-unk. The engineers then focused on identifying all the unk-unks and solving them before blast off.

With most software projects you are also faced with unk-unks. After all if you’re not solving a new problem why are you writing code? So what should you do about it?

Well you really only have two options. Either, you hedged against by developing on parallel lines and hope that one works out. This is the approach many designers take – they prototype lots of ideas knowing that only one will be picked, and no one can predict which one before all have been seen. Alternatively you can manage the unk unks as they emerge, by changing the approach in response to new information.

If you want to take this second approach then the team needs to be empowered to solve problems en route (rather than only being allowed to build to an existing design and plan). However, in my experience the senior management and/or product owner must also be intimately involved in the project, have sufficient knowledge to check the plausibility of changes, and be supportive of the mid-course corrections.

The project also need to be managed and structured to allow this to happen. And this means the project needs adopt an adaptive, agile methodology, such as Scrum. So the project team are encouraged to solve problems in little steps as they arise and adapt as they learn more.

Wabi-Sabi – Design to age

We’ve recent returned home from camping in France, along with a fairly large number of caravaners. I mention this because I noticed that caravans and campervans don’t appear to be designed to age gracefully. I’m not talking about whether or not they are ‘design classics’ but rather that they aren’t designed to easily allow user modification nor to keep their looks as they age and pick up the inevitable scratches, nicks and general wear and tear. They appear to have only been conceived up to the point of sale – working on the assumption that they will remain perfect forever – with no apparent thought for how they will be affected by use. That’s just silly. Stuff should be designed to take into account general wear and tear.

Tiles

The Japanese have a useful concept to encapsulate this idea – Wabi-sabi (侘寂). Wabi-sabi acknowledges three core principles: nothing lasts, nothing is finished, and nothing is perfect. In other words aesthetics is founded in the acceptance of transience – that beauty is in the imperfect, impermanent, and incomplete.

It seems to me that the same ideas should be applied to our technology products. Too many of our technology products are designed like hi-tech caravans. Lovely and shinny at the point of sale (not that caravans are lovely, but you know what I mean!) – but they don’t stand up well to wear and tear, and they don’t let users easily modify the system to work the way they want to work. They appear to have been designed without thought for how they will age nor how their users might wish to hack the system – to womble – to use the system in unexpected ways.

For software products this means designing small components loosely joined together by a lightweight framework, that support open standards, protocols and APIs; with interfaces that are simple yet flexible enough for users to create their own journeys and workflow.

But Wabi-sabi also tell us something about how to build software. The acceptance of a state of ongoing imperfection implies that attempting to design a solution upfront will be doomed to failure. Instead we should embrace the idea of the perpetual beta, evaluate what works in the real world and modifying the system as a result.

Photo: Stone Tiles, by abundanceguy. Used under licence.

Scaling up agile projects

Agile methods don’t scale, if you are embarking on a big serious project you had better adopt a ‘heavyweight’ method. That way you have certainty, control over your resources and an agreed plan. Right? Wrong.

Agile methods promote using small teams because it improves communications. With improved communications the team members are more likely to understand the problem they are being asked to solve and therefore the proposed solution. Also when the project requirements change the team can more easily adjust. But what about large projects? How can small teams can deliver big projects?

The answer? Divide the big project into small independent projects. Nothing too shocking about that. But there is still a question about how to organise the work and structure the project. Earlier in the year I attended Spa 2007 where Dave Thomas outlined one approach.

The objective is to let projects have freedom to create the solutions to specific problems but to coordinate activity between projects. The approach is to structure the work into four phases:

  • Envisioning;
  • Definition;
  • Development and;
  • Release.

Large Scale Projects

Envisioning

The envisioning team is tasked not with capturing requirements but rather with understanding the problem to be solved, defining the solution and finally building an executable prototype. The work is managed with Scrum but because the purpose is to build a prototype that meets the users’ needs the team works very closely with end users developing their solutions iteratively: adding fidelity and refining the design at each iteration.

If appropriate two envisioning teams can run in parallel developing ‘competitive’ ideas to explore different options.

Definition

The Envisioning Team provides a clear definition of what needs to be built; the Definition team designs how to build it. To do this the Definition Team is tasked with structuring the work and defining the initial Product Backlog. The aim is to reduce dependencies: not just between components, but between the overall requirements and the component designs. The work then is structured into a series of discreet teams:

  • Feature teams (user facing services)
  • Component Teams (core reusable components)
  • Release Engineering (continuous integration and QA)

Feature teams can be responsible for no more than defining the unit tests for component sets – or integrating components with service specific code to provide specific functionality and/or glue sets of components together.

The Release Engineering team is responsible for ensuring that all the components and features work and conform to the agree interoperability framework. They therefore take each release and ensure the ‘system’ works, rather than trying to configure the system at the end of the project.

NB Components can be built out of phase with the rest of the project.

Development

The first job for each of the development teams is to review the product backlog to ensure that the project estimates are OK and the scope is understood.

If any product backlog item can’t be delivered or if there are any project dependencies these get escalated to the ‘risk backlog’ – which is owned by the Product Owner. Its the Product Owners responsibility to clear the Risk Backlog and not allow the individual projects to be effected.

Release

Code is packaged, documentation written, marketing etc.

And of course throughout this whole process everything is kept in the code: code, requirements, risks, people, estimates…

Facebook: new social network site; same old walled garden

Last year the buzz was around MySpace now its Facebook and before that Friends Reunited and Linkedin. I have to confess that I’ve never really grokked these services – I’ve played around with them a bit – but generally never really got that much out of them. Preferring to stick with email, IM, Flickr, del.icio.us and my blog.

Trends in Facebook, MySpace, Friends Reunited and Linkedin

The problem I’ve always faced with community sites, such as LinkedIn or Facebook, is that I’m never sure I want to maintain yet another online presence. I know that they provide tools to help bootstrap the site with data about your group of friends (importing contacts from your email account etc.) but there’s more to it than that. And in the back of my mind I know that all too soon there will be a new site out there doing more or less the same thing but with a twist that grabs everyone’s attention. And the reason this is a problem is, as Steve Rubel points outs, because they are walled gardens.

Despite the age of openness we live in, Facebook is becoming the world’s largest, and perhaps most successful, walled garden that exists today…

The problem, however, lies in this fact – Facebook gives nothing back to the broader web. A lot of stuff goes in, but nothing comes out. What happens in Facebook, stays in Facebook. As Robert Scoble noted, it’s almost completely invisible to Google. You can share only a limited amount of data on your public page – as he has here. That’s fine for many users, but not all.”

Walled gardens create barriers to getting information in and out of their system. This means that I know that I will need to go to extra effort to seed the site with information about me and my network; maintain duplicate information between different gardens as well as in the wild; and have difficultly getting data out and into something else. Walled gardens will always eventually die because they require that extra bit of effort, both from their community and, more widely, the developer community. As Jason Kottke notes like AOL before them Facebook’s walled garden approach places additional strain on the development community.

What happens when Flickr and LinkedIn and Google and Microsoft and MySpace and YouTube and MetaFilter and Vimeo and Last.fm launch their platforms that you need to develop apps for in some proprietary language that’s different for each platform? That gets expensive, time-consuming, and irritating. It’s difficult enough to develop for OS X, Windows, and Linux simultaneously… imagine if you had 30 different platforms to develop for.”

[what’s needed is]…Facebook inside-out, so that instead of custom applications running on a platform in a walled garden, applications run on the internet, out in the open, and people can tie their social network into it if they want, with privacy controls, access levels, and alter-egos galore.”

In other words we already have the platform – its the internet, in its raw, wild, untended form. And rather than trying to build walls around bits of it we should keep our content in the open, in applications such as Flickr, WordPress (other blogging software is also available) and email. But tie it all together into communities with technologies such as OpenID the “open, decentralized, free framework for user-centric digital identity” and Friend of a Friend (a project aimed at creating a Web of machine-readable pages describing people, the links between them and the things they create and do).

Indeed this approach is similar to that adopted in Plaxo 3.0 which now runs as a web service removing all reliance on Outlook. Plaxo now provides a synchronization and brokerage service between applications (e.g. Outlook or Apple’s Address Book) and services (AOL, Google) – your data is no longer within a walled garden but you do have access control over who can access your data.

Facebook is an amazing success – but like all walled gardens will eventually either die or be forced to open its garden gate and let the rest of the internet in (for example, let me replace the Facebook’s status application with Twitter, or Photos with Flickr). And in the meantime I’m happy to stick with my existing online presence.

Design for wombling

In the 1970s the Wombles were Britain’s great engineers. They designed, built and fixed stuff by focusing on what was available and what worked, rather than using the proper stuff. There are lessons here for both software engineers and interaction designers.

Take Backpack. One of the great things about Backpack is that you can easily use it to do so many useful things and it neatly fits around your workflow rather than making you work around its workflow. And I bet that the 37Signals designers never imagined all the ways people might use their applications – and of course they didn’t need to, they just needed to design a system that let people womble. To work the way they want and support them in doing so.

The advent of SMS messaging is an even more stark example. According to Cor Stutterheim from CMG SMS wasn’t even designed for customers:

It started as a message service, allowing operators to inform all their own customers about things such as problems with the network. When we created SMS (Short Messaging Service) it was not really meant to communicate from consumer to consumer and certainly not meant to become the main channel which the younger generation would use to communicate with each other.

But because it was reliable, flexible, convenient and cheap the technology was repurposed and used by mobile ‘phone users in their millions.

SMS was the triumph of the consumer- a grassroots revolution that the mobile industry had next to nothing to do with and repeatedly reacted to. This is in stark contrast to the top down technology and industry led approaches to other nonvoice services such as WAP. (ref)

And away from consumer products we have the ubiquitous TCP/IP. Although it wasn’t designed to support the Internet as we now know it TCP/IP is the backbone for much of our social and business life. TCP/IP is able to be flexible and useful because the hosts and not the network are responsible for reliability. The network only enforces interoperability. This means it’s possible to join almost any network together, no matter what their characteristics. In other words its extensible, scalable and decentralised.

As Nick Gall in the Podcast TCP/IP and Shipping Containers argues if you want a system that is extensible, scalable and decentralised you need a solid, but basic underlying architecture. Too much architecture in the framework, and you’re locked in, without sufficient freedom. Too little, and you have neither interoperability nor room to improvise. In other words the users of the system can’t womble.

The balance between simple, formalised, underlying structure with a high degree of overlying freedom is critical. Too little structure and you are left with an inaccessible, chaotic system. Too much structure and you have a rigid product that can’t be easily molded into something else.

So what are the characteristics of systems that Wombles will be able to use in unexpected ways? I think they include:

  • Independent subsystems tied together by a light weight, solid framework
  • Defined and enforced interoperability between subsystems
  • Platform neutrality
  • General purpose simple interfaces
  • Flexible user journeys and workflow
  • Accessibility
  • Open standards, open source, open protocols and open APIs
  • Data neutrality
  • And designed to respect that the data belongs to the user not the system

But I also suspect that the system itself can’t be built be ardent Wombles. If the system has be cobbled together then there’s a risk that it will lack the flexibility and structure necessary to allow its users to Womble – providing the light weight but solid framework may prove too difficult, or enforcing interoperability too cumbersome.

Photo: Flickr Hack #1 by Thomas Hawk. Used under licence.