Category: Metadata

Linked things

Linked things

So this is the question: do you always need separate URIs for non-information resources and the information resource? That is do you need an identifier for both the document and the thing the document is about? Your answer to that question will depend a lot on your attitudes to the semantic web project.

Now until recently I would have said “yes you do need both”, but recently I’ve been thinking that perhaps it’s not quite so black and white.

Before I get into why I think it probably makes sense to backtrack a little and explain the background to the question. After all for many people this question seems odd: why on earth would you need a URI for anything other than the web page, the document?

In the real world we give all sorts of things identifiers: people have passports and National Insurance Numbers; buildings get Post Codes; books ISBNs etc. We do this because it’s useful to be able to unambiguously identify stuff. To be able to point, discuss and share information about things.

On the Internet we have email addresses and URIs on the Web. OpenID for example is predicated on the notion that a person can have an URI to identify themselves. And the Linked Data project gives URIs for not just people, but all sorts of things: people, places, animals, music, and through dbpedia the myriad of things described in Wikipedia.

Once you have an identifier for a thing you can make assertions about that thing. How big it is, where it is (in the real world), when it was created, who owns it, anything. You can also describe how those things relate to other things – this person is friends with this person and works for this company, which is at this address etc.

Now many people will tell you (indeed I probably will too) that you need to distinguish the statements you make about the thing in the real world from the statements about the document. For example, a URI for me might return a document with some information about me, but the creation date for that document and the creation date for me are two different things. And because you don’t want to get confused it’s better to have a URI for the thing and another one for the document making assertions about the thing. Make sense?

For those that are interested there are a couple of different ways of achieving this separation. For the purposes of this post it’s not important to know how to do this, but if you’re interested have a look at this paper by Richard.

But here’s the thing, many people will tell you that this is all too complex and frankly unnecessary, indeed you may well be thinking the same thing right about now.

Some people will tell you that the whole non-information resource thing isn’t necessary – we have a web of documents and we just don’t need to worry about URIs for non-information resources; others will claim that everything is a thing and so every URL is, in effect, a non-information resource.

Michael, however, recently made a very good point (as usual): all the interesting assertions are about real world things not documents. The only metadata, the only assertions people talk about when it comes to documents are relatively boring: author, publication date, copyright details etc.

If this is the case then perhaps we should focus on using RDF to describe real world things, and not the documents about those things.

On the Web there are a number of different ways of making an assertion about a thing (as identified via a URI): you can state how it relates to other things, you can link it to a piece of data (e.g. RDF literals) or you can link it to a document which makes some statements about the thing (e.g. a news article).

The question is: is there much utility in defining non-information resources in this third scenario: do you need URIs for the documents? Obviously they still need a URL so you can link to it and you should make that document available in a variety of representations but do you need a separate identifier for the non-information resource?

I think not.

This is how I’ve started to think about it: RDF is a great way of describing how (real world) things relate to each other and for this you need URIs for non-information resources. And because you’re dealing with real world things (I know documents are real world things too, but going down this path is how we ended up with the confusion we have today) you will hopefully have interesting and useful links to other things, useful chunks of data and links to useful documents about that thing. Those documents could be in any format – they could be an HTML document, a (Flash) movie, MP3 file, even a csv file. The point is the documents decorate the tree they are discoverable via the RDF graph but they don’t need to be published as RDF themselves.

An RDF graph of things is therefore a great way to: discover documents, to make assertions and share what we know about how those things. Or put another way RDF is a way of building a vocabulary to describe how web resources related to real world objects. I my however me wrong and I would be interested to hear what others think.

There’s only metadata and URIs

On the web I reckon there’s only metadata and URIs or perhaps there’s no metadata and only data. Either way the metadata, data/content distinction isn’t helpful.

Linked Data allows you to bind HTTP URIs to an object and to information about that object. This is useful because it’s more useful to talk about real world things — things like people, places and events — the things that people think about. Despite this I have numerous conversations, and have done for years, about what ‘metadata’ to use to describe a document. Typically what this really means is: “what keywords to use so that some technomagical solution can use that ‘metadata’ to personalise/ recommend content”.

Self-portraiture + metadata by Saltatempo's. Some rights reserved
Self-portraiture + metadata by Saltatempo's. Some rights reserved

Beyond the obvious — keywords on their own are never going to achieve the sorts of solutions non-technical people imagine — it also forces an unhelpful schism. It makes people think about their content and your metadata, or that metadata is somehow outwith the content they are creating. The trouble is that one persons data is another persons metadata. Is the title of a story metadata or content? Is a news story content or metadata about a real world event? The answer depends on your perspective.

It seems to be that a more useful way to think about things is to have URIs to identify things and then have information/documents/data/metadata/whatever that make assertions about those things. Sometimes those bits of information will be simple data points, for example, for an album release they might include information/metadata about who performed or wrote the piece (obviously linking to URIs to identify the person who did perform or write it, with appropriate predicates) while other bits of metadata might be more verbose: reviews of the album or the lyrics etc. and then again some might be media things (recordings of the album etc.).

And of course because we’re talking about a graph of data, those documents making assertions about a thing can in turn also have metadata/data/documents which make assertions about them, for example, who wrote it, comments about it etc.

Imagine what might happen if a news website took this approach? You would mint a URI for the event (or reuse one that already existed) and then write news stories about it, each with their own URL, each making assertions about that event. It would create a news service which was truly native to the Web, rather than a facsimile of the printed press. Imagine then what it would be like if we could link-up all the news stories on the web which also made assertions about that event. As a user of such a site/ set of sites I could find everything about a given thing (a person, event or place).

Of course, as Dan Brickley, put it:

concepts and events are still social and technological artefacts, but they are designed to help interconnect descriptions of butterflies, documents (and data) about butterflies, and people with interest or expertise relating to butterflies.

In other words what matters is a way of identifying things, a way of interconnecting them and a way of describing them — subdividing those ways of describing them into ‘data’ and ‘metadata’ is unhelpful, or at the very least adds nothing useful.

It is however useful to separate our concept of something from our conception of it. As Stephen Pinkers puts it:

…if you look up William Shakespeare in a dictionary it says “English playwright, lived in the 17th century, wrote Romeo and Juliet and Hamlet, etc.” Is that what the name William Shakespeare means, and is that what the concept William Shakespeare is? That sounds plausible, but it turns out not to be true. If we were to learn that William Shakespeare didn’t write any of the plays attributed to him — let’s say that we learned he didn’t even live in Stratford, that there was a clerical error and he really lived in Warwick. He would still be William Shakespeare, and we wouldn’t posthumously dub the real author of Shakespeare’s plays William Shakespeare. We would just say we were mistaken about what we believed about William Shakespeare.

So what is the concept of William Shakespeare, the meaning of the word William Shakespeare? Basically, when Mr. and Mrs. Shakespeare christened their son William, and the name stuck, and then everyone who knew him, and then who knew someone else, who knew someone else, and passed it down to us — that unbroken chain of transmission of the name from the moment of first dubbing is what gives William Shakespeare its meaning. There’s a sense in which to have a concept necessarily means to be connected to the world through this chain of transmission of a name going back to the moment of first dubbing.

So while I don’t think it’s helpful to separate data from metadata it is helpful separate concept from conception.

Linked Data making the web human centric

The lovable Mr Stephen Fry recently noted [iTunes link] that the challenge isn’t to help people become “computer literate” instead it is to make computers “human literate”. And when you think of the last 25 years, as an industry, we’ve done a pretty good job. 

Apple launched the Macintosh in 1984
Apple launched the Macintosh in 1984

1984 saw Apple launch the Macintosh and with it the world was introduced to the GUI. And then in 1989 TimBL invented the web and changed the world. I’m not suggesting for a moment that everything is OK in the world of interaction design, just that we have come a very long way.

The genius of the Web and the Macintosh is their ability to abstract information to make it more useful, as TimBL put it when talking about the Giant Global Graph:

[The Net] made it simpler because [instead] of having to navigate phone lines from one computer to the next, you could write programs as though the net were just one big cloud, where messages went in at your computer and came out at the destination one. The realization was, “It isn’t the cables, it is the computers which are interesting”. The Net was designed to allow the computers to be seen without having to see the cables.

Simpler, more powerful. Obvious, really.

And then with the development of the Web we could go one step further:

“It isn’t the computers, but the documents which are interesting”. Now you could browse around a sea of documents without having to worry about which computer they were stored on. Simpler, more powerful. Obvious, really.

That’s where we are, more or less, right now, except there’s a realisation that we can keep going, keep making the web more useful and easier to use because:

“It’s not the documents, it is the things they are about which are important”

To achieve this we need to be able to identify those things we’re interested in and the relationship between them, in a way that is above the level of documents, if we do this then we get reuse of data around the concept. That’s just what Linked Data is all about, allowing us to break free of the document layer by focusing on URLs.

By thinking about the web as a web of (identifiers for) interconnected things, not a web of pages means that when I watch a TV programme online it’s not the page on iPlayer (other players are available) that matters to me instead it’s the URI of the programme and it’s that URI that I bookmark. This means that whatever device I use, my iPhone, laptop or IP enabled TV, it will use the device appropriate view. But because we’re talking about URIs and HTTP isn’t just a different way of tuning into a set of presets it also means, as Nicholas Negroponte puts it:

My VCR of the future will say to me when I come home, “Nicholas, I looked at five thousand hours of television while you were out and recorded six segments for you which total forty minutes. Your high school classmate was on the ‘Today’ show, there was a documentary on the Dodecanese Islands, etc…” It will do this by looking at the headers. The bits about the bits change broadcasting totally. They give you a handle by which to grab what interests you and provide the network with a means to ship them into any nook or cranny that wants them.

Designing the web in this way, by thinking about what real world objects people care about, giving them all URIs and then linking them up and linking them to the rest of the web – building the web the linked data way – means you can use the network to not only deliver content but also let people discover more content, mash content together to create new stories.

This as I see it, abstracting the problem above the document layer, is a very sensible way to help make computers more ‘human literate’ because people can stop thinking about webpages and instead start thinking about the stuff that matters to them – whether that be a TV programme, a music track, a book, a person, or a holiday. And whether they access that thing on their desktop computer, mobile phone or IP enabled TV set.

Interesting stuff from around the web 2009-01-12

Teaching a six year old about Triples, by Leigh Dodds. Some rights reserved.
Teaching a six year old about Triples, by Leigh Dodds. Some rights reserved.

Content modeling & FOAF — this is really very good stuff — you should read these then subscribe to Paul’s blog

Paul’s experiments with FOAF and Doctor Who
“I chose the first ever episode of the show, from 1963. This featured four main characters, and thanks to the workshop from Yves and the others, I had an inkling of an understanding of how to create FOAF profiles. The results can be seen here (best viewed if you use a Firefox plugin like Tabulator). So far so good. I then linked each character to the other, using the simple ‘knows’ relationship. Finally, to get my linked open data brownie points, I linked each character to its DBpedia equivalent, using the OWL ’same as’ relationship.” Modeling Doctor Who is going to be cool esp if Paul looks at the different incarnations of the Doctor (different actors + different version).

Part three of Paul’s investigation into fictional content modeling and FOAF
One thing that has come up in the discussions, though, is that there’s perhaps two elements to what I’m trying to achieve. The first is to link existing ontologies and, if needed, build a new one, to help describe the narrative content of ’stories’ within the context of television and radio programmes. The second is to experiment (and for me to learn) with existing ontologies, again, linking them up, to build dynamic and interesting webpages that work on linked data principles.

SUDS an ontology (an extension to FOAF) for Soap operas [author stream presentation]
Celia Romaniuk’s presentation on SUDS and it’s application to Eastenders. 

Visualising radio — I can’t but applaud the team and I love the technology but does the idea make sense? I’m not so sure…

… a bit like ‘enhanced podcast’ (you know podcasts with pictures) because my iPod lives in my pocket as far as I’m concerned the images are just a waste of bandwidth; likewise visual radio. That said it’s well worth checking out.

Visual radio launch announced [BBC Radio Labs]
Snippets of information delivered over http to visualise the Chris Moyles show.

Visualising Radio, pushing, not pulling [whomwah.com]
A bit of technical background from Duncan. “So, the big news here is that we are pushing and not pulling. Ordinarily, you would request a webpage using your browser and a some data would be returned. The only way that you would then subsequently see any changes to that data, would be if you requested it again, or if something on that page requested it for you, maybe via polling.”

These are just great…

In Search of Cultural Identifiers [BBC – Radio Labs]
Getting this stuff right is really the first job in any web project. Identify the objects you want to talk about and model the relations between those objects. The key point is to ensure the things you model map to user’s mental models of the world. User centric design starts here and if you choose to model and expose things that users can’t easily comprehend no amount of product requirements or personaes or storyboards will help you out. For want of a better label we A&M types often refer to this work as ‘cultural identifiers’. One identifier, one URL, one page per cultural artifact, all interlinked. It’s something that Wikipedia does better than anyone. One page per concept, one concept per page. bbc.co.uk could be a much more pleasant place to be if we can build something similar for the BBC.

Teaching a Six Year Old About Triples [Lost Boy]
“I suggested to him that we try drawing it out. I thought that this might help him get a better mental picture. I explained to him that we could try writing down the characters names and start drawing lines between them to illustrate the relationships. He got it straight away.” This is brilliant and shows just how intuitive RDF is.

Wikipedia over DNS [DgL]
“I had written some code to take wikipedia articles and summarise them. I wanted to offer this for use in various places, now the obvious way to offer it is just a web service (via REST, SOAP, etc), but that’s boring and I had a cunning plan. Why not offer it over DNS – it is basically a huge associative array and DNS is designed for this stuff. So I wrote a little nameserver which returns the results as TXT records. There are some obvious limitations for example responses are limited to around 430 bytes (it only does UDP). It has advantages too, it gets cached at your nameserver and it is also faster than HTTP (no need to setup a TCP session).” Such a cool idea and lovely URLs too.

The web as a CMS

If you run a website you’re going to want to manage your content. You might use an Enterprise CMS, an open source CMS, a blogging platform or a bespoke app, and as you might expect at the BBC the same rules apply. Except some of us have been trying out something a bit different — using the web as a content management system.

Coffee Shop Study by notashamed. Some rights reserved

As anyone that reads this blog will realise I’m a bit of of Linked Data nut. I believe that the future of web design lies not in webpages but in URLs and resources. If you’re not sure what I’m on about then I suggest you read Tim Berners Lee’s article on the Giant Global Graph or Tom Coates’s presentation Native to the Web of Data.

I and the teams working on BBC programmes and music believe that the web is wonderful because of links — because people can go on journeys of discovery, browsing by meaning — by following links to the things that interest them. We also believe that where existing webscale identifiers exist or where existing ontologies, taxonomies and metadata exists we should reuse those and link to them. This focus on URLs and resources, and existing services has meant the services that we’ve been building are a bit different, as we’ve stated previously:

A core feature of the site is the integration with external services – notably MusicBrainz and Wikipedia. We are using these services to provide core information (discographies, biographical information, membership etc) about artists and releases. We are then combining this data with information from within the BBC – including details about which BBC programmes have played that artist.

The use of community, non-BBC, maintained databases obviously means that much of the data the BBC is publishing on bbc.co.uk/music/beta comes from external services. But what may not be immediately obvious is that the BBC isn’t forking this data, as others have done. Because that would be silly.

wikipedia

So if we want to create a new artist, or edit any of this content then rather than editing that data via some internal CMS we edit MusicBrainz or Wikipedia. Just like you or anyone else, as Nick puts it:

From now on if I want to indulge my love of Frank Sinatra, I’ll just edit the Wikipedia page, knowing it will turn up on the BBC. Collaboration is the future, and not just in music.

Indeed with MusicBrainz we’ve been actively contributing content since June 2007.

[…] In exchange, MusicBrainz receives a monthly license fee that will allow MetaBrainz to hire some engineering help in the coming months to work on new features and to improve the existing infrastructure. This is quite significant since MusicBrainz has been resource constrained for many months now — having paid people on staff will ensure a more reasonable amount of progress moving forward.

Even cooler, the BBC online music editors will soon participate in the MusicBrainz community contributing their knowledge to MusicBrainz. The goal is to have the BBC /music editorial team round out and add new information to MusicBrainz as they need to use it in their MusicBrainz enabled applications internally.

As of today the BBC music team has contributed over 2,800 edits to MusicBrainz. None of this is to say that we don’t need content management solutions for our own, internal (meta)data we clearly do, but it does means that a significant proportion of these services are drawing on data from elsewhere on the web, and that means that if we want to edit it we edit the web improving those services and those provided by the BBC (including search).

Interesting stuff from around the web 2008-08-21

Tag on Tim Berners-Lee's original NeXT machine -- first Web server [by Robert Scoble Used under licence.]
"Tag on Tim Berners-Lee original NeXT machine - the first Web server" by Robert Scoble, used under licence
An ontology to modeling dynamic online presence [The Online Presence Project]
Having a presence ontology could be used to facilitate the integration of online presence data from various sources, or help in transferring data from one service to another. Interesting join to the Music Ontology.

MOAT (Meaning Of A Tag) provides a Semantic Web framework to publish semantically-annotated content from free-tagging
It aims to solve this by providing a way for users to define meaning(s) of their tags using URIs of Semantic Web resources (such as URIs from dbpedia, geonames … or any knowledge base), and then annotate content with those URIs rather than free-text tags.

Wikipedia Miner lets you query Wikipedia to resolve terms to their relevant topics and navigate the categories and links that organize them
Wikipedia Miner is a toolkit for navigating and making use of the structure and content of Wikipedia. It aims to make it very easy for you to integrate Wikipedia’s content into your own applications.

‘Semantic Web at the BBC’ workshop [BBC Internet Blog]
Nice write up from Silver – including Michael and Matt’s ‘brief introduction to the semantic web’, Alex talking about MOAT and Dan Brickley considering how OpenID might fit in.

Music APIs – Yahoo! make theirs available and someone else starts to use the BBC’s

iPlayer Real Converter
Easy access to the BBC radios’ Real streams. Nice work and handy for those that like their audio via Real.

Y! Music Battle of the APIs in the Key of Ruby [Yahoo! Developer Network Blog]
Yahoo! release a RESTful API to Yahoo! Music. Limited to 5,000 requests per day and for non commercial use. Nice though.

Finally some more news on the LHC

CERN are going to start up the Large Hadron Collider – Here’s how It plans to stop it [IEEE Spectrum]
The experiment will hurtle two hair-thin beams of hundreds of trillions of protons around a ring-shaped accelerator at 99.99 percent the speed of light, each unimpeded beam is capable of melting a 500-kilogram block of copper. Since malfunction could lead to a catastrophic accident we should be thankfully CERN has spent nearly two decades devising an interlocking system of fail-safes.

The all new BBC music site where programmes meet music and the semantic web

We (well Nick, Patrick, Tony, Deanna, Sacha and Guy) have recently been working on a major rewrite of the BBC’s music site [beta] – and since its just gone live as a public beta I thought it would be a good time to explain a bit about what we’ve done and what we hope to do with the site. I would also love to hear what you think about what we’ve done so far – especially as we move from public beta towards a replacement for the current site.

Madonna's artist page on the BBC's new beta music site
Madonna's artist page on the BBC's new beta music site

Our work so far has focused on providing core information for every artist the BBC plays on our daytime radio network shows. That’s shows like Annie Mac and Chris Moyles on Radio 1, or Steve Lamacq on 6Music. We’ve focused on these shows because we’ve integrated the site with the radio playout system VCS dira! which are basically giant iPods in the basement of Broadcasting House. Unfortunately the specialist shows as well as national and local radio shows don’t use these so we haven’t got track listing data in the right format for these programmes. But we have a plan and will be adding more shows in due course.

So what have done so far?

For starters we decided that we wanted to have a single canonical page for every artist. We decided to do this because we want to aggregate everything we know about an artist at a single URL. But this means that we need unique, persistent, unambiguous URLs. Thankfully, as Michael has already discussed, Musicbrainz gives us unique identifiers that allow us to provide just such a set of URLs.

The core of the new site then is built around Musicbrainz. In addition to giving us web scale identifiers Musicbrainz is also being used to give core music metadata e.g. discographies, related artists and related links. Some of those related links are Wikipedia links and we are using those to go and fetch the introductory text for each artist’s biography from Wikipedia.

The approach we’re using to keep the Wikipedia data up to date is, I think, quite neat. Patrick has written a bot to monitor the Wikipedia IRC channel for updates – when it spots an update we fetch the new Wikipedia content. Oh and obviously all this data is rendered dynamically using the same MVC framework we are using for /programmes which means that updates happen almost instantaneously.

This brings me to the next major feature – integration with /programmes. As I’ve said we have integrated with VCS to give us track listing information for our daytime radio shows – we are matching this data with both our internal database of programme metadata and Musicbrainz. This lets us know which radio stations and shows have played which artists.

We also want to make this data available for others to use and so have designed the site to provide a RESTful API, following the principles of Linked Data:

…namely thinking of URIs as more than just locations for documents. Instead using them to identify anything, from a particular person to a particular programme. These resources in-turn have representations, which can be machine-processable (through the use of RDF, Microformats, RDFa, etc.), and these representations can hold links towards further web resources, allowing agents to jump from one data-set to another.

If you would like to have a play with these we have RDF, XML, JSON and YAML representations of the resources – just add .xml .rdf .json or .yaml to the end of the artist url.

Where next?

Nick, Michael and I have previous spoken about our plans for linking programmes, music, events, topics and users. Well this is our first foray into this world. Information about programmes and music is interesting, it’s useful; but it’s not as interesting nor as useful as when the two are intelligently linked. Joining the two worlds means that you can aggregate information about the programmes that have played an artist [as we’ve done], you can put track listings on episode pages, you can have charts of which artists are played most on all BBC Radio programmes, on Radio 1, by Zane Lowe. You can also aggregate all episodes that you can currently listen to that feature a given artist. Or show which programme first played a given artist and how often the BBC has played them since. Basically the interesting stuff happens at the joins between the nodes because that’s where the context lives.

By exposing the information that is created by joining programmes and music we can provide context and serendipity. We can help you find out about the music you’ve just listened to and introduce you to new shows that also features the music you like. So that’s what we’re working on.

We also need to provide more data about each artist – from both inside and outside the BBC. That means bringing the album reviews into the fold, hooking up external news feeds and the like. But whatever we do it’s worth bearing in mind that this is on a much larger scale than the current music site. This new site has in the order of 388,398 artist pages, 157,677 external links and 93,912 artist to artist relationships.

It would be great to hear what you think about what we’ve done and our plans for the future. You are welcome to leave a comment here, or via the Backstage mailing list.