Category Archives: Work

When’s the sailing on? Introducing genre schedules…

You might find this helpful - you’ve been able to browse schedules by radio station or TV channel for a while now which is great, but what if you don’t care where the BBC broadcasts your favourite Olympic sport, music genre or SciFi programme?

To help you find programmes, no matter which station or channel they are broadcast on, we’ve started publishing schedules for all our genres (sport, music etc.). These views are being used as part of the BBC’s Olympics coverage - specifically to drive the TV schedule and individual sport pages. But as you might be able to guess a little bit of URL hacking gives you more. All you need to do is add /schedules to the end of any genre aggregation, so for example:

A schedule for your favourite music genre :
www.bbc.co.uk/programmes/genres/music/classicpopandrock/schedules

What’s happening on the Olympics:
www.bbc.co.uk/programmes/genres/sport/olympics/schedules

Drama on radio:
www.bbc.co.uk/radio/programmes/genres/drama/schedules

Or science and the environment on Radio 4:
www.bbc.co.uk/radio4/programmes/genres/factual/sciencenatureandenvironment/schedules

And obviously you can get this data both as HTML and as XML, JSON, YAML just by adding .xml .json or .xml to the end of the URL. So the TV coverage for sailing as XML is:

www.bbc.co.uk/programmes/genres/sport/sailing/schedules.xml

Hope that helps you find your favorite programme.

UPDATE

As Duncan has previously discussed on the BBC’s Radio Lab blog you can also get this data in your Calendar via iCal - just add .../upcoming.ics to the above URL. For example:

www.bbc.co.uk/programmes/genres/sport/sailing/schedules/upcoming.ics

www.bbc.co.uk/radio4/programmes/genres/factual/sciencenatureandenvironment/schedules/upcoming.ics

Interesting BBC data to hack with

If you are interested in our on going drive to make our data available in interesting and useful ways you might be interested in Nick, Patrick, Duncan and Sean’s recent work.

XML views of radio AOD availability

If you head over to bbc.co.uk/radio/aod/availability/:network.xml then you will get an XML file (updated every 3 hours) with details about what’s available to listen to now and in the next 48 hours. So for example:

http://www.bbc.co.uk/radio/aod/availability/radio1.xml

Will give you data about Radio 1 data [obviously]. The file contains a bunch of metadata about the episodes including details of the stream URLs.

What you will notice is that we’re not pointing you directly at the URLs for the audio instead we’re directing you to our ‘Media Selector’ which we use to maintain the availability window. So if you follow the media selector link you will get back a lump of XML with details of the available media. By the way you’ll want to use the /mediaSelection/media/@encoding = real. Ignore the MP3 that’s a ’secure’ stream used in iPlayer.

Programme schedules as XML, YAML, JSON and Text

Duncan has already written about his work to implement iCal views on the Radio Labs blog:

iCalendar is a standard for calendar data exchange. It is most notably used by Apple’s iCal application, Microsoft’s Outlook and Google Calendar, to import and export calendar information. We thought that some of the views in /programmes would also work well in the iCal format, so we have exposed a few for you to subscribe to, and play with.

These are available for your regular schedule and for genres [upcoming cricket programmes for example] and for individual programmes. For example:

Upcoming episodes of Eastenders
webcal://bbc.co.uk/programmes/b006m86d/episodes/upcoming.ics
Upcoming (New) episodes of Eastenders
webcal://bbc.co.uk/programmes/b006m86d/episodes/upcoming/debut.ics
Episodes of Eastenders available to watch again
webcal://bbc.co.uk/programmes/b006m86d/episodes/player.ics

In addition to iCal you can also get this data as plain text, XML, JSON or YAML. So the upcoming drama programmes as XML can be found here:

http://www.bbc.co.uk/programmes/genres/drama/schedules/upcoming.xml

Or the Radio 1 schedule as plain text as:

http://www.bbc.co.uk/radio1/programmes/schedules.txt

Artist pages as RDF

We’re busy doing a load of work on the music site right now which will be launching really soon. When we do in addition to lots of HTML we’ll also me making the data available for machines - including RDF. But if you are into this sort of thing here’s a sneaky peak at what will be released:


<?xml version="1.0" encoding="utf-8"?>
<rdf:RDF xmlns:rdf      = "http://www.w3.org/1999/02/22-rdf-syntax-ns#"
         xmlns:rdfs     = "http://www.w3.org/2000/01/rdf-schema#"
         xmlns:foaf     = "http://xmlns.com/foaf/0.1/"
         xmlns:mo       = "http://purl.org/ontology/mo/"
         xmlns:mf       = "http://purl.org/ontology/mo/mf#"
         xmlns:owl      = "http://www.w3.org/2002/07/owl#"
         xmlns:time     = "http://www.w3.org/2006/time#"
         xmlns:dc       = "http://purl.org/dc/elements/1.1/"
         xmlns:timeline = "http://purl.org/NET/c4dm/timeline.owl#"
         xmlns:event    = "http://purl.org/NET/c4dm/event.owl#">

<rdf:Description rdf:about="/music/artists/cc197bad-dc9c-440d-a5b5-d52ba2e14234.rdf">
  <rdfs:label>Description of the artist Coldplay</rdfs:label>
</rdf:Description>

<mo:MusicGroup rdf:about="/music/artists/cc197bad-dc9c-440d-a5b5-d52ba2e14234#artist">
  <foaf:name>Coldplay</foaf:name>

  <mo:image rdf:resource="/music/images/artists/7col_in/cc197bad-dc9c-440d-a5b5-d52ba2e14234.jpg" />

  <mo:musicbrainz rdf:resource="http://musicbrainz.org/artist/cc197bad-dc9c-440d-a5b5-d52ba2e14234" />
  <mo:homepage rdf:resource="http://www.coldplay.com/" />
  <mo:fanpage rdf:resource="http://www.pleasureunit.com/coldplay/index.php" />
  <mo:wikipedia rdf:resource="http://en.wikipedia.org/wiki/Coldplay" />
  <mo:imdb rdf:resource="http://www.imdb.com/name/nm1095892/" />
  <mo:myspace rdf:resource="http://www.myspace.com/coldplay" />

  <mo:member rdf:resource="/music/artists/18690715-59fa-4e4d-bcf3-8025cf1c23e0#artist" />
  <mo:member rdf:resource="/music/artists/d156ceb2-fd90-4e82-baea-829bbdf1c127#artist" />
  <mo:member rdf:resource="/music/artists/6953c4db-7214-4724-a140-e87550bde420#artist" />
  <mo:member rdf:resource="/music/artists/98d1ec5a-dd97-4c0b-9c83-7928aac89bca#artist" />

</mo:MusicGroup>

<mo:SoloMusicArtist rdf:about="/music/artists/18690715-59fa-4e4d-bcf3-8025cf1c23e0#artist">
  <foaf:name>Guy Berryman</foaf:name>
</mo:SoloMusicArtist>
<mo:SoloMusicArtist rdf:about="/music/artists/d156ceb2-fd90-4e82-baea-829bbdf1c127#artist">
  <foaf:name>Jon Buckland</foaf:name>
</mo:SoloMusicArtist>
<mo:SoloMusicArtist rdf:about="/music/artists/6953c4db-7214-4724-a140-e87550bde420#artist">
  <foaf:name>Will Champion</foaf:name>
</mo:SoloMusicArtist>
<mo:SoloMusicArtist rdf:about="/music/artists/98d1ec5a-dd97-4c0b-9c83-7928aac89bca#artist">
  <foaf:name>Chris Martin</foaf:name>
</mo:SoloMusicArtist>

</rdf:RDF>

We would really welcome your feedback on any of this.

And finally a bit of URL hackery…

Our decision to use opaque IDs for our programmes [episodes, series and programme brands] means that we can provide persistent URL - which is a good thing. The downside is that you can’t guess the URL. To fix this you can now enter URLs like this:

www.bbc.co.uk/programmes/eastenders and you will be redirected to www.bbc.co.uk/programmes/a-z/by/eastenders/all

That’s a disambiguation page for all programmes with Eastenders in the title. If however there’s just one programme with that title, like for example www.bbc.co.uk/programmes/chrismoyles then you will be redirected to that brand page (www.bbc.co.uk/programmes/b006wkqb).

Photo: Data storage - old and new, by Ian-s. Used under licence.

Helping machines play with programmes - XTech presentation

Here are the slides from my recent presentation at XTech on our work on the BBC Programmes Ontology.

I’ve also written a post over at the BBC’s Radio Lab blog about the machine readable serializations to represent the concepts described within the ontology.

We have been following the Linked Data approach - namely thinking of URIs as more than just locations for documents. Instead using them to identify anything, from a particular person to a particular programme. These resources in-turn have representations, which can be machine-processable (through the use of RDF, Microformats, RDFa, etc.), and these representations can hold links towards further web resources, allowing agents to jump from one dataset to another.

more…

There are also details about how to get your hands on XML, JSON and YAML serializations of the schedule views and RDF serializations of the objects (episode, series and programme brand).

We would really welcome feedback on what we’ve done - especially the RDF views which are still in development.

Highly connected graphs: Opening BBC data

Mike Butcher wants the BBC to open up. To make its data available, to provide lots of APIs, well as he puts it:

Dear BBC,

What we want is your data, a lot more APIs, developer tools and your traffic.
We’ve paid for it already in the license fee.
Now get on with it.

Yours Sincerely,
The UK’s Startups

I do largely agree with Mike’s central premise - the BBC does need to make its data more accessible, it does need to provide more APIs. And as Matt has already noted there are people at the BBC working to open things up. Now I don’t want to get into the debate about what the BBC does well vs what it doesn’t - but I did want to highlight some of the work that the team I work in is doing and to give some perspective on why Mike’s objective isn’t as simply as it might appear.

I work in the “FM&T for A&Mi” bit of the BBC (as James has rechristened it) - in other words the ‘new media’ team embedded within the radio and music department. We’re currently working on a couple of projects (programmes and a revamped music site) that I hope might give some of the UK Startups some of what Mike is after. And in due course we’ll be adding more data that will make more startups happy (hopefully).

So what are we doing? It’s probably easiest to start by looking at the current programmes beta - the objective is to ensure that every programme the BBC broadcasts has a permanent, findable web presence. The site provides data for the eight BBC TV channels, ten national radio stations and the six stations covering Scotland, Northern Ireland and Wales.

To enable the sharing of this data in a structured way, we are using the linked data approach to connect and expose resources i.e. using web technologies (URLs and HTTP etc.) to identify and link to a representation of something, and that something can be person, a programme or an album release. These resources also have representations which can be machine-processable (through the use of RDF, Microformats, RDFa, etc.) and they can contain links for other web resources, allowing you to jump from one dataset to another.

OK so that’s the theory - what are we doing?

Currently the pages are marked up with microformats: hCalendar on the schedule views and hCard for cast and crew on episode pages. That’s OK - but hardly the APIs that Mike is after. But what’s coming very soon will hopefully be a bit closer to the mark. Our plan is to make all resources available in a variety of formats: XML, Atom, RSS 2, JSON, YAML, RDF etc. (We’ll announce these on Backstage as they become available.)

And to help folk get direct access to the actual data backing BBC Programmes, we designed a Semantic Web ontology covering programmes data, The Programmes Ontology. This ontology provides web identifiers for concepts such as brand, series, or episode and is released under a creative commons license so anyone can use it.

But there are limitations of such a web interface. To provide a more expressive API we are also investigating using D2R Server, a Java application for mapping relational databases to RDF to make the data accessible through SPARQL. SPARQL allows you to carry out more complex queries than would be possible with simple RDF representations of the resources - think of SPARQL as SQL for the semantic web. It allows you to semantically connect to external data sources such as DBpedia to provide extra information that is not present in our dataset, such as date and place of birth of cast members.

So what about music data? We’re not as far ahead with this work as we are with programmes but you still shouldn’t have to wait too long.

As I’ve written about before we are using MusicBrainz to provide GUIDs for artists and releases and; to give us core metadata about those music resources. The use of MusicBrainz IDs means that we can relate all the BBC resources about an artist together; and others who also use these GUIDs (e.g. metaweb) can use them to find our resources.

In terms of making the data accessible - it’s a similar story to programmes. We’re currently marking up relevant pages with microfromats (hReview and hCard) but the plan is to publish BBC music resources in a variety of different representations (XML, Atom, RSS 2, JSON, APML etc.)

What resources are we talking about? In addition to the core data from MusicBrainz we’re also talking about album reviews (released under a Creative Commons License) and data from our music programmes, for example:

  • views aggregating the artists played on each station into both charts (most played that day, week and since the prototype started running) and artist clouds;
  • programme views which are similar to the station aggregation views but for each programme and links through to each episode;
  • programme episodes with track listings which link through to artist pages;
  • artist pages with biographies pulled in from Wikipedia (MusicBrainz include links to Wikipedia) and links back to the programmes that have featured that artist.
  • And much more…

As you can see we’re not only making the data available as discrete resources we are also linking them together - and making that data available in both human readable and machine readable views. This is a big job - it involves a lot of data, a lot of systems (both web and production systems) and it all needs to work under a high load.

And what about the future? As Michael recently presented at the Semantic Camp our plans are to join programmes, music, events, users and topics. All available on the web, for ever, for people to explore and open for machines to process. If you would like to find out more then Nick and I will be discussing this further at XTech next month in Dublin.

So I hope, in our way we are ‘getting on with it’ - sorry for the delay.

Photo: Please open door slowly, by splorp. Used under licence.

Programmes Ontology at XTech 2008

This year’s XTech conference in Dublin, Ireland will see a gathering of software engineers, information designers, and managers that work with web and standards-based technologies looking at the worlds of web development, open source, Web 2.0 and open standards.

Over the last few months our corner of the BBC has been working on a couple of projects that aim to expose BBC metadata for others to use: Programmes and Music. Our work on the new music site isn’t quite ready for public consumption yet, but programmes has been up and running since last autumn. So a few of us thought that we should share what we’ve been up to, why we’ve done what we’ve done and hear what folk think of it all.

Our hope is that by exposing our data in a variety of accessible forms (including ATOM, JSON, XML and RDF) other developers can do interesting stuff with it. If you would like to find out a bit more you can read my previous post about the ontology, the abstract for XTech or better still come and listen to Nick and myself in Dublin on the 8th May.

Photo: Somewhere in Dublin, by pierofix. Used under licence.

New prettier /programmes

Over the last few weeks the /programme’s team, but mainly Jamie, have been working to bring the site inline with the BBC’s new online visual language (as rolled at on the BBC’s home page). Today we’ve launched the upgrade.

programmes.png

In addition to the new shinier, wider UI we’ve also linked up the schedules which should be helpful. And fixed and tidied up a bunch of things under the hood which has improved the applications performance.

Episode Page

Hope you like it!

Innovation Labs in search of creativity

I’ve being spending the last few days on the shores of Loch Lomand at the BBC’s Innovation Labs. Although the BBC has been running the labs for a while now this round is the first time I’ve got involved.

Loch Lomand

Having worked on both sides of the fence so to speak - having worked in the private sector for a number of years pitching for work and now at the BBC listening to others pitching their ideas I’ve found the whole experience really interesting.

The idea behind the Innovation labs is to help foster creativity by getting companies with a track record in innovation to pitch their ideas in response to briefs set by the BBC. The ten best in each region are then invited to come along to a week long creative workshop where they work with mentors to develop their ideas before pitching to us BBC folk. If successful the project gets funded and built or at least prototyped.

What this means is that the companies get some funding to develop their idea and time with BBC staff and mentors to help hone their idea. That’s really good - it increases the chances of getting projects that meet the BBC’s objectives, good for the BBC; good for the companies, who have a better chance of succeeding and winning work; and good for the license fee payer who see an improving online service.

But… when we started the process I couldn’t help thinking that we were missing an opportunity, I thought that what would be really good is to spend less time developing the idea and more prototyping the idea.

I have always found innovation to be a contact sport. It works best when there are constraints, objectives and tangible deliverables. Spending too long conceptualising your idea can leave you short of Oxygen, you need to get down from the abstract and deal with the actual. Doing so helps your idea to grow and mature. Rapid prototyping helps not only test the validity of your idea but also helps develop it.

There were some really very bright people at the Innovation Labs, cool ideas and some fun technology. I would have loved to have had a relatively short burst of effort working up the ideas and then a healthy slug of time prototyping the idea, bringing together a team of editorial and technical staff from the BBC with the design and technologists from the companies.

However, having just finished listening to the companies pitch their project to us and seen how all the projects have being improved since Wednesday lunchtime. I have to admit that I was at least in part wrong. The process does add value, the ideas are better as a result of the process and I’m sure the services we will be able to launch are going to be really great. So watch out for lizards on Big Ben, improved user interaction and navigation around our programmes and much more.

Photo: If only Loch Lomand always looked like this..., by Ben Matthews. Used under licence.

An ontology for programmes

What is an ontology? Well if you Google for it you’ll find lots of references to “an explicit specification of a conceptualization“. More usefully an ontology is a way of formally describing what something is and how it relates to other things.

British Library

Over the last few years their popularity has waxed and wained: Yahoo tried to categorize web sites into predefined hierarchical categories - but as Clay Shirky points out that didn’t work so well because “there is no shelf“:

Yahoo, faced with the possibility that they could organize things with no physical constraints, added the shelf back. They couldn’t imagine organization without the constraints of the shelf, so they added it back. It is perfectly possible for any number of links to be in any number of places in a hierarchy, or in many hierarchies, or in no hierarchy at all. But Yahoo decided to privilege one way of organizing links over all others, because they wanted to make assertions about what is “real.” [...]

One reason Google was adopted so quickly when it came along is that Google understood there is no shelf, and that there is no file system. Google can decide what goes with what after hearing from the user, rather than trying to predict in advance what it is you need to know.

In other words rather than having a site located in one predefined location within a classification scheme you have linked resources. Obvious huh? But in that case why does anyone bother with trying to define a classification system? Because there are situations where having a classification scheme does make sense. Clay suggests these situation include the following:

  • Small corpus
  • Formal categories
  • Stable entities
  • Restricted entities
  • Clear edges

The whole of the internet doesn’t fit these criteria, which is why Yahoo had problems - but somethings do - and one that does are TV and radio programmes, not at the subject matter level but at a meta level, at a level of how programmes are structured: episodes, series, broadcasts and the like.

A few years ago the Audio and Music part of the BBC, where I now work, started to look at how to model programmes - Tom Coates post ‘the age of pointing at things‘ describes much of this early work. Since then we’ve significantly re engineered the underlying technology and rolled it out for all BBC programmes.

As I discussed previously the idea behind BBC Programmes is to ensure that every programme brand, series and episode broadcast by the BBC has a permanent, findable web presence. The Programmes Ontology was developed by Yves, Michael, Patrick and Nick to expose this data following the Linked Data approach, enabling the interchange of programme information on the Semantic Web. This then is the current ontology for programmes:

Programme ontology

The ontology provides identifiers for concepts such as brand, series and episode and is divided in two main parts. Firstly, it describes categorical information about programmes, and the relations between those categories. For example, it allows the description of a programme brand, a series constituting it and the episodes in that series. And secondly it describes the services and broadcast events.

Internally we’ve used this ontology and a D2R server to map the underlying relational databases to RDF through SPARQL which resulted in over 5 million RDF triples accessible via multiple views (brand, series or episode) in addition to direct queries via the SPARQL interface.

SPARQL is a bit like SQL for RDF and the so provides an interface that allows access to the data in ways that would be difficult to achieve via the Programmes web interface. In addition SPARQL can be used to connect and combine data sources external to the BBC, such as DBpedia, which can be used to contextualise the programmes data with information not held within our dataset e.g. the date and place of birth of cast members.

Finally to facilitate its uptake we have released the ontology under a Creative Commons license. So if you need to model TV or Radio programmes please feel free to use it.

Photo: The British Library, by Steve Cadman. Used under licence.

QR codes for BBC programmes and some other stuff

We’re on a bit of a roll at the moment with our work on /programmes, so quick on the heals of the work to integrate iPlayer and programme credits onto our episode pages I’m chuffed to announce another batch of new features: QR-Codes, schedules and um calls to inaction.

Torchwood episode page

QR-Codes, if you’ve not come across them before, are a bit like two-dimensional bar codes created by the Japanese company Denso-Wave. But unlike bar codes you can encode any string you fancy (up to 4,296 alphanumeric characters) into an image and this means you can convert a URL into a QR-Code.

You can then print QR-Codes on posters, t-shirts, scarfs, whatever. People can then decode them by taking a picture, using specific software installed on their mobile phone, to read and convert the code back to the URL - so they can easily visit the web site.

Anyway - every BBC programme now has a QR code. You can get it by adding …/qrcode to the end of the URL for any programme page. For example, this is the URL for Platoon:

	http://www.bbc.co.uk/programmes/b007mcgw

And this is its QR-Code:

	http://www.bbc.co.uk/programmes/b007mcgw/qrcode

QR Code

A more mainstream feature that’s also gone live today are schedule pages for some radio networks. For example:

	http://www.bbc.co.uk/asiannetwork/programmes/schedules

And because of the way the technology works we also get a bunch of new pages for free:

	http://www.bbc.co.uk/asiannetwork/programmes/genres/music

And finally we’re flagging when episodes are next due to be broadcast, when we don’t have any media and links to the iPlayer so you can download the video.