Our development manifesto

Our development manifesto

Manifesto’s are quite popular in the tech community — obviously there’s the agile manifesto and I’ve written before about the kaizen manifesto and then there’s the Manifesto for Software Craftsmanship. They all try to put forward a way of working, a way of raising professionalism and a way of improving the quality of what you do and build.

Anyway when we started work on on the BBC’s Nature site we set out our development manifesto. I thought you might be interested in it:

  1. Peristence — only mint a new URIs if one doesn’t already exist: once minted, never delete it
  2. Linked open data — data and documents describe the real world; things in the real world are identified via HTTP URIs; links describe how those things are related to each other.
  3. The website is the API
  4. RESTful — the Web is stateless, work with this architecture, not against it.
  5. One Web – one canonical URI for each resource (thing), dereferenced to the appropriate representation (HTML, JSON, RDF, etc.).
  6. Fix the data don’t hack the code
  7. Books have pages, the web has links
  8. Do it right or don’t do it at all — don’t hack in quick fixes or ‘tactical solutions’ they are bad for users and bad for the code.
  9. Release early, release often — small, incremental changes are easy to test and proof.

It’s worth noting that we didn’t always live up to these standards — but at least when we broke our rules we did so knowingly and had a chance of fixing them at a later date.

What I wish I had made at bbc.co.uk if I stayed

In many ways I’ve been very lucky at the BBC I’ve helped make some cool stuff – well stuff I’m proud of. But since I’ve decided to leave I’ve started to wonder what else I would have like to have made, if I had stayed at the BBC.

There’s a bit of a health warning however, these are just ideas. I’ve no real idea if they are that practical and they almost certainly don’t fit into the current strategy.

Get Excited and make things
Get Excited and make things

My ideas…

Lab UK meets so you want to be a scientist

Lab UK is the part of the BBC’s website where you can participate in scientific experiments. They’ve done some cool stuff – including Brain Test Britain which had 67,000 people sign up and resulted in a paper in Nature [pdf].

The various experiments are tied into TV programmes and this is really important because it helps generate interest and get the number of participants required to make the experiment work. However, it also means that the experiments are designed in advance, by the scientists, and the public’s role is one of test subject.

The experiments do help build knowledge but they probably don’t help people understand science.

So here’s the idea – a bit like Radio 4’s “So You Want To Be A Scientist” the process would start with people suggesting ideas, questions they would like answering, the site would need to provide sufficient support to help people refine their ideas. It might even use material from the BBC archive to help explain some of the basics but at it’s heart it would be a collaborative process.

The ideas would then be voted on and the most popular would then be taken forward. With the help of scientists the experiment would be designed and build and carried out on the Lab UK platform, giving these amateur experiments potential access to a huge audience.

The process would be a rolling series of experiments designed and carried out by the public.

History through the eyes of the BBC

The BBC makes a lot of programmes about history – but much more significantly it has been part of or at least recorded a lot of our more recent history.

So rather than making a history site about the Romans, the Victorians or whatever I would use the BBC archive to tell the history of the world as seen through the eyes of the BBC.

Combining news stories, clips from programmes (broadcast or not), music and photographs the site would tell the story of the world since 18 October 1922.

The site would chart the major political, scientific, sporting, cultural and technological events since 1922 but also the minor events – the ones that remind us of our own past.

The site would provide a page for every day, month, year and decade since the BBC came into existence as well as pages for the people, organisations and events the BBC has featured in that time.

Basically a URI for everything the BBC has recorded in the last 90 odd years.

The site would also allow members of the public to add their thoughts and memories (shared under whatever licensing terms they wish) to enrich it further to create a digital public space for the UK.

One BBC nature

A few weeks ago we merged Wildlife Finder into the nature site and launched a new blog – and today we’ve taken the final step and brought Earth News into the fold to create a consolidated BBC nature site.

From a certain perspective this doesn’t represent a big change – after all we’re still publishing exclusive natural history news stories, video collections and video clips and information about: animals, plants, habitats (and the ancient earth’s habitats, such as Snowball Earth), adaptations & behaviours, places and ecozones, the geological time periods when they lived, the major mass extinction events, including the one that killed the dinosaurs, in fact lots of information on the history of life on earth and the fossil record. We even have a page about fish – and they don’t really exist!

However, from another perspective this is a really big change. It’s a big change because we’ve (hopefully) made everything so much simpler.

Screen grab of the new BBC nature site - features section

We’ve made it simpler by bringing everything together into one site and removed the various sub brands – if you love nature and natural history everything is now in one place: news stories, video clips from the archive, opinion pieces and more.

Bringing everything together has also allowed us to make a few additional changes which should help us more easily publish the content.

In addition to natural history news we have a features section where we can bring together articles and photo galleries (like this one) and a new blog Wonder Monkey written by Matt Walker. Matt has written a few posts so far including this one on the oddball midge that shouldn’t exist.

I really hope you like it. It represents the culmination of two years of work, during which time we launched and evolved both the site itself and the editorial proposition – there now are c.3,000 clips available online (many of which are available worldwide) about almost 900 animals (both prehistoric and living), 50 plants etc.

And of course wildlife data is for machines too.

However, after two years of development this represents the last major release, for a while at least. The site will continue to grow because we are continuing to create great new content as well as digging out the best bits from the archive – like this video collection looking back at David Attenborough’s Madagascar (starting with Zoo Quest 50 years ago). But there won’t be any major new features for a while, not that that’s a major problem – the site should offer a rich experience with amazing content.

As I said yesterday, I’m very proud of what we’ve produced and if I can marshal my thoughts I’ll try and write a post or two about how we went about building the site and the lessons I learnt on the way, until then enjoy the site.

Leaving the BBC

Leaving the BBC

After almost five years this will be my last month at the BBC.

The BBC has been a great place to work – I’ve worked with some amazing people, helped deliver some of the best work of my career and had the opportunity to speak at conferences around the world, including (amazingly) at the Web’s 20th birthday celebrations in CERN.

The BBC can certainly be a challenging place to work but I’m very grateful to Dan Hill and Matt Wood for offering me a job in the first place. I just hope I’ve not let them down because for every challenge, gripe and frustration there have also been opportunities to learn new things, work with brilliant people and help deliver great stuff that has, I think, had an positive impact on what the BBC does online.

So what have I been up to since I’ve been here?

The first project I worked on was /programmes a site that means that every programme the BBC broadcasts now has a web presence – one that both humans and machines can enjoy. The site is sometimes criticized as being a card catalogue of BBC programme metadata but its worth remembering that until the site launched the vast majority of programmes had no URI, had no webpage of any kind; /programmes changed that at a stroke. It was also the first truly dynamic web site on bbc.co.uk and whatever people might say about the aesthetics the site has the prettiest URIs of any site I know (something to thanks Michael Smethurst for).

The music site was my other project while in the FM&T bit of Audio & Music. Building on Musicbrainz the idea was to create a rich graph, linking music programmes with artist pages (available as HTML and RDF etc.) via ‘clickable tracklistings‘.

After a couple of years I left Audio & Music and joined ‘BBC Vision’ – the bit of the the BBC that does the telly – and took on a project known internally as ‘BBC Earth‘. And pretty much tried to replicate the music work but for natural history content.

I say I tried to replicate the music work that’s not really true, or rather its only true to a point. The core underlying concepts where the same, but the manifestation is quite different. For starters we sought to digitise and make available the TV archive but we also created original content – this broke down into exclusive natural history news stories, stories from TV and Radio production teams on location and, curated video collections.

I wanted the nature site to help people discover, explore and understand the natural world through the BBC’s content, I hope we’ve achieve that to some extent. Personally, and I know I’m biased, I think the site is brilliant and one of the best looking and useful semantic web sites around (we publish the data as RDF).

The credit for the site, however, should go to the team that actually made it. I was lucky, the core of the team has remained on the project throughout its development and I’m indebted to those, more talented than me, for making it what it is.

As I’ve said, I think the site is brilliant and I think the editorial, technical and design knowledge and skills of the team shine through, the site is theirs not mine.

There’s much I could write about this work – but I should really do it a bit more justice than the space available here and so I’ll save what I have to say for another post. Also there’s one last thing to push live on the site, to round off its development and it feels wrong to preempt that.

So what now? Well I’m joining Nature Publishing Group as Head of Platform for nature.com. As a failed scientist I’m very excited by the opportunities – Nature is the leading weekly, international scientific journal with a mission to:

Helping achieve that mission on the Web is a really exciting prospect and I hope the next five years prove as productive as the last. Wish me luck.

BBC iPhone apps? I think there’s a better way

Last month the BBC announced that it would launch a limited number of mobile app, starting with News and Sport  and then possibly an iPlayer app. Unsurprisingly the NPA promptly complained that the BBC would “damage the nascent market“, and now the BBC Trust as said that it wants to review the plan and that means a delay.

Well I don’t know about whether such a move by the BBC would have an impact on the market or not (although I agree with Martin, I think it was inevitable that the Trust has would review the plans), but I do think the BBC could tackle the problem in a different, more open way.

BBC News on mobile

There’s a lot of hype and hyperbole around mobile apps – and in some ways you can understand why, lots are downloaded and some folks are making money from them but I’m not so sure it’s going to last. I suspect that mobile apps are successful for a few reasons:

  1. They are hooked into a big marketing push. Apple et al. are all publicising their stores on your handset on the telly, on posters and in papers.
  2. The app stores are targeted and people know where to look, the Web could be The Store (as it’s been for other things) but that’s not how regular folks appear to see software nor do they want to dig about for what to install.
  3. The Web (mostly) only works when you’re online, apps (mostly) work offline too.
  4. Some stuff can only be built as a native app (rather than via the web), probably.

But as phones expose more of their API to the browser, as HTML 5 with its support for offline browsing and other goodies becomes adopted and, as libraries and support become available so the technical and user experience barriers start to become less relevant — it may once again be universally seen as sensible to develop web apps. Of course either the fear of being locked in or being locked out of the relationship with their customer might kick companies along a bit too.

So in the near future we should be able to build web apps every bit as good as mobile apps? Yes, but I would go further: for most of the things the BBC wants to do, the technology is already good enough. And with a web focused mind set you can start to invest in the sorts of things you can only do server side — just look at the sorts of things Google are building: word processors, voice communications, email clients, image recognition, maps etc. I think it’s better to embrace the future than play catch-up with the near past.

But what if I’m wrong and mobile apps are the future of content delivery? Well the BBC could still take a different approach – one where it licensed its content in such a way that others could build apps with its content. Of course, unless things changed, the app would need to be non-commercial and the use of the BBC logo and brand would be protected. Of course the non-commercial aspect might be reviewed under certain circumstances — indeed the BBC already licenses content to third parties both outside and inside the UK via its commercial arm BBC Worldwide, why not online? Although I can’t see any circumstance under which the BBC would allow use of its brand and logo since this is central to protecting its reputation, to avoid this sort of thing.

If the BBC did license its content in such as way as to allow others to build stuff then we might see all sorts of interesting innovation on all sorts of different devices and not just mobiles. Perhaps I’m missing something but I don’t see why the BBC needs to control the entire distribution chain, from encoding to eye balls, when distributing content over IP but not when broadcasting to your TV or radio. The BBC doesn’t make its own televisions nor radios instead it lets the market manage that bit, why not encourage the same sort of thing on the web?

Some thoughts on curation – adding context and telling stories

Just over two years ago I wrote a post about the importance of the resource and the URL — and I still stand by what I said there: the core of a website should be the resource and its URL. And if those resources describe real world things and they are linked together in the way people think about the world then you can navigate the site by hopping from resource to resource in an intuitive fashion. But I think I missed something important in that post — the role of curation, the role of storytelling.

When we started work on Wildlife Finder we designed the site around the core concepts that we knew people cared about and those that we had content about i.e. species, their habitats and adaptations, and we’ve been publishing resources about those concepts since last September. We’ve since published the model (Wildlife Ontology) describing how those concepts relate together. I’ve talked about this work as providing us with the Lego bricks because I also realised that we needed to use those Lego bricks to build stories, to help guide people through the content. Our first foray into online story telling with these Lego bricks are the Collections.

Collections allow us to curate a set of resources – to group and sequence clips and other resources to tell stories like the plight of the tiger or the years work of the BBC’s natural history unit. Silver Oliver has recently written about why he thinks this approach is important, why curation in a metadata driven information architecture — it’s a very good post — you should read it. But I thought I would share a bit about the intellectual framework behind how I think of this stuff. As with most of my ideas it’s not my ideas but one I’ve borrowed from someone brighter than me, in this case Nathan Shedroff who proposed a framework to think about how to build Lego bricks and then things with those bricks. A framework I’ve been using for few years now.

Wildlife Finder provides information by repackaging data from elsewhere – by organising programme clips, news stories etc. around natural history resources and concepts. This is good (I hope) because it provides useful additional context; but it’s not the whole story. In Shedroff’s model this process creates information — by adding context to data by presenting and organising it in a new, useful way. This is really what encyclopedias provide — structured information presented and organised in useful ways.  The next step is to take this information and build stories with it to build knowledge and facilitate conversations.

As I say, with Wildlife Finder, we have started to tell stories by localising the information into Collections, but of course, now we have a unified domain model (which links together programmes and concepts within the natural world) there are other ways in which we can add context and build knowledge on top these resources — in addition to collections. There are lots of ways we can create new experiences, but as you can see from the diagram above, we don’t hold a monopoly in terms of story telling — those that consume the information, our audiences and ‘users’ could also build stories. Although the BBC doesn’t really let people build their own stories other sites and organisations do, notably  Flickr who have a series of interesting approaches to let its users add context to photos through Groups, Galleries, Sets and Collections.

Apis and APIS a wildlife ontology

By a mile the highlight of last week or so was the 2nd Linked Data meet-up. Silver and Georgi did a great job of organising the day and I came away with a real sense that not only are we on the cusp of seeing a lot of data on the web but also that the UK is at the centre of this particular revolution. All very exciting.

For my part I presented the work we’ve been doing on Wildlife Finder – how we’re starting to publish and consume data on the web. Ed Summers has a great write up of what we’re doing I’ve also published my slides here:

I also joined Paul Miller, Jeni Tennison, Ian Davis and Timo Hannay on a panel session discussing Linked Data in the enterprise.

In terms of Wildlife Finder there are a few things that I wanted to highlight:

  1. If you’re interested in the RDF and how we’re modelling the data we’ve documented the wildlife ontology here. In addition to the ontology itself we’ve also included some background on why we modelled the information in the way we have.
  2. If you want to get you’re hands on the RDF/XML then either add .rdf to the end of most of our URLs (more on this later) or configure your client to request RDF/XML – we’ve implemented content negotiation so you’ll just get the data.
  3. But… we’ve not implemented everything just yet. Specifically the adaptations aren’t published as RDF – this is because we’re making a few changes to the structure of this information and I didn’t want to publish the data and then change it. Nor have we published information on the species conservation status that’s simply because we’ve not finish yet (sorry).
  4. It’s not all RDF – we are also marking-up our taxa pages with the species microformat which gives more structure to the common and scientific names.

Anyway I hope you find this useful.