Opening up the BBC’s natural history archive

The BBC’s Natural History Unit is responsible for some of the BBC’s most loved TV and radio programming — unfortunately until now it’s only been accessible as part of the regular schedule or via iPlayer. I say until now because today we launched the first phase of a new project which brings clips from the best of the NHU’s programmes online.

Pages for habitats, taxa and adaptations
URIs for habitats, taxa and adaptations

Over the last few months we’ve been plundering the NHU’s archive to find the best bits — segmenting the TV programmes, tagging them (with DBpedia terms) and then aggregating them around URIs for the key concepts within the natural history domain; so that you can discover those programme segments via both the originating programme and via concepts within the natural history domain — species, habitats, adaptations and the like.

The segments/ clips ‘belong’ to their originating programme — and as a result we’ve been adding information, about a bunch of programmes from the archive, to PIPs (the underlying database behind iPlayer and /programmes). The clip pages aren’t yet linked in with their owning episode, but they will be soon.

In addition to being able to discover these clips from within the context of the programme we are also providing URIs to aggregate information around the natural history domain, that is URIs for species, habitats, adaptations and ecozones.

URIs for species such as the Bush Elephant
URIs for species such as the Bush Elephant

Our hope is that by providing highly inter-linked, URIs we can help people gain a greater understanding of the natural world. For example, by being able to see the different animals and habitats that live within different ecozones you can gain an understanding of the diversity of of life in different parts of the world; or what different animals make up the Mammal or Bird Class; or more about a particular adaptation.

Ovoviviparous - what it is, what animals do it and BBC archived content about it
Ovoviviparous – what it is, what animals do it and BBC archived content about it

Of course we are doing more than providing access to programme segments, we have also plundered our sound archive so you can hear what the different habitats and species sound like (and obviously those sounds are separately addressable), we are then aggregating content from the other ‘BBC Earth’ projectsEarth News and Out of the Wild and elsewhere on the web.

It’s not just about BBC content.

You might have noticed that the slugs for our URIs (the last bit of the URL) are the same as those used by Wikipedia and DBpedia that’s because I believe in the simple joy of webscale identifiers, you will also see that much like the BBC’s music site we are transcluding the introductory text from Wikipedia to provide background information for most things. This also means that we are creating and editing Wikipedia articles where they need improving (of course you are also more than welcome to improve upon the articles).

We are also publishing data from bunch of other organisations. Information about habitats, ecozones and species distribution is provided by WWF’s Wildfinder; the species conservation status by IUCN’s Red List of Threatened Species and (where available) information about why a species is at threat coming for Zoological Society of London’s EDGE of Existence programme. Finally information about a species adaptations and behaviours are provided by Animal Diversity Web.

Adopting this approach means that we are able to contribute distinctive content to the Web while at the same time helping people find what is already there.

There is a lot more we need to do, including linking in with current programmes and making everything available as RDF, JSON and for mobile devices. That’s all on it’s way but in the meantime I hope you find what’s there useful, informative and entertaining.

33 responses to “Opening up the BBC’s natural history archive”

  1. To paraphrase O Wilde – to preside over the transformation of one major domain of BBC content into a living linked data archive in the course of the year may be regarded as good fortune; to do both looks like design. Chapeau!

    Like the new derivadow.

  2. Great stuff – really brings it all to life, I had great fun exploring the different ways of navigating between species etc.

    …and if it works so well for this domain, should help sell the approach wider as well ;-)

  3. Absolutely fantastic! Big up Tom Scott!

  4. This is really excellent and nice to see dbpedia identifiers in there. Congrats to you and the team!

  5. Thanks everyone. I’m very proud of what we’ve achieved.

  6. […] This post was Twitted by kehan […]

  7. […] Derivadow.com: Opening up the BBC’s natural history archive News of a new online BBC project from the Natural History Unit which themes content including clips from their programmes around subjects like deserts and elephants. […]

  8. […] But it’s an absolute pleasure to celebrate one amazing piece of work: the project to open up the BBC’s natural history archive. The project’s been led by Tom Scott and it’s his blog post I’m linking to […]

  9. […] Opening up the BBC’s natural history archive […]

  10. really great stuff

    one little niggle: with the “WHAT DO THEY SOUND LIKE?” audio player – when you try to share the url, the player is not big enough to show the url and when you choose to use a service like facebook to share, it does not pre-populate the link field. Using chrome

    Otherwise thanks

  11. actually – just noticed the iplayer video links dont work either – they just send you to a a page telling you ‘No longer available on BBC iPlayer’. So can’t really share any of the media it seems yet

    is that what ” The clip pages aren’t yet linked in with their owning episode, but they will be soon.” means?

    cheers

  12. @juniorbonner sorry about the bugs and thank you for reporting them.

    We know the audio player is a bit small, and will be fixed soon. I hadn’t spotted the Facebook bug – thanks, I’ll raise that bug.

    In the meantime if you add /sounds to the end of the URL e.g. http://www.bbc.co.uk/nature/species/Lion/sounds then you get a list of available sound URLs.

    Likewise the iPlayer video links is a known bug and the relevant team are working on a fix. The problem is that it’s linking to the owning episode rather than the clip page.

    The solution here is to open the clip in a new window/ tab so that you get to a page like this one: http://www.bbc.co.uk/programmes/p003c3d9

    “The clip pages aren’t yet linked in with their owning episode, but they will be soon.” thing is about not linking from episode pages link this one: http://www.bbc.co.uk/programmes/p002sh97 not linking to clips http://www.bbc.co.uk/programmes/p003c3d9

  13. […] Opening up the BBC’s natural history archive « Derivadow.com The story behind it. (tags: bbc nature data Archive) […]

  14. Integration with the Encyclopedia of Life would be nice. Each species page in the Natural History archive could link to the corresponding page in EOL.

    1. Yes indeed – I want to automate the process rather than maintaining the links by hand, there’s an item on the backlog to do just that.

  15. Hi Tom.

    Is there a global list of the tags anywhere – this is something I’d love to link to from Freebase? If not, I’ll just link up the ones I find :-)

    Cheers!

  16. Back from a little holiday and trying to work out some encoding problems at the Freebase end, and it looks like you may have one at your end :-)

    As far as I can tell, anything with an apostrophe in is erroring out: see eg http://www.bbc.co.uk/nature/species/Eleonora%27s_Falcon as linked from http://www.bbc.co.uk/nature/species

    I don’t think this was happening before I went on holiday a week ago, but I could be wrong??

    1. Philip, looks like we’re having a little config issue with some of our servers, we’re on the case. Thanks for reporting it.

    2. The apostroph bug is all fixed, everything should be working again as normal. Sorry about that.

  17. Right, I think I’ve now got everything linked from Freebase. See eg the Snow Leopard page there, and note the “BBC Natural History” weblink on the right-hand side.

    The only thing I know which isn’t working is anything with a non-ASCII character in the key, which is triggering a bug in the Freebase web client so the keys don’t appear there.

    If you want to link back to the Freebase pages, it should just be a matter of linking to http://www.freebase.com/view/wikipedia/en/, except you’ll have to deal with Freebase’s slightly quirky encoding, which replaces everything outside the set [0-9A-Za-z_-] with a $xxxx encoding, where xxxx is the hex-encoding of the Unicode codepoint of the character; for example, apostrophe becomes $0027.

    1. That should be http://www.freebase.com/wikipedia/en/KEY except I wrote KEY in angle brackets and it got stripped…

    2. Very nice – we’ve tracked down the encoding bug and the fix should be live tomorrow morning.

      How are you keeping things up to date? Are you spidering those pages on a regular basis? If not FYI we’ve a bunch of new taxa going live next week.

      And indeed I would like to link to Freebase. Backlog item created.

      1. great work tom, thanks from canada

      2. I’m not spidering the pages on a regular basis per se, but there’s a script on my laptop which I can run whenever to pick up any changes. Are you going to be adding any “categories” other than the 8 you listed above? (just as that requires a small amount of work at the Freebase end).

      3. Philip – yes we have plans to add more categories – but we’re also working on a /animals (and /plants) page which will display all ranks for which we have content (i.e. you won’t need to spider /species /genus etc.)

      4. This may be a good time for a request then :-)

        At the moment, I’m not finding it completely trivial to link to bbc.co.uk: say I’ve got the key “Snow_Leopard”. I can’t determine solely from that key the URL to link to, as I’ve got to know that it’s a species (and also that species are found in the /species/ directory on bbc.co.uk). Would it be at all possible to create something like http://www.bbc.co.uk/nature/key/Snow_Leopard which would redirect to the correct page?

      5. I’ve just spidered the new links and I think there may be one small problem: the /nature/species page is linking to /nature/species/Doria’s_Tree-Kangaroo rather than /nature/species/Doria’s_Tree-kangaroo (note case of “K”). The former page doesn’t have the Wikipedia article or “Elsewhere on the web” sections (and breaks my script as there’s no “Doria’s Tree-Kangaroo” article at Wikipedia :-) ). The /genus/Tree-kangaroo page has the correct link.

        [ There’s also no video available on either of the Doria pages. I’m not sure if that’s intentional or not ]

  18. […] example: http://www.bbc.co.uk/nature/species/African_Bush_Elephant. The BBC’s Tom Scott explains: Over the last few months we’ve been plundering the NHU’s [Natural History Unit's] […]

  19. Philip – sorry about that, we’ll get that fixed.

    1. Phil – it should be fixed now.

  20. […] Opening Up The BBC’s Natural History Archive — the BBC are releasing programme segments and a whole lot of metadata around their programming. Audio and video segmented, tagged with DBpedia terms, and aggregated into a URI structure based on natural history concepts: species, habitats, adaptations, etc. Gorgeous! […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: