WoDOOM 2012: A brief review of the First International Workshop on Debugging Ontologies and Ontology Mappings


I travelled to Galway (Ireland) in early October for the First International Workshop on Debugging Ontologies and Ontology Mappings, or WoDOOM 2012 in short, which was co-located with EKAW 2012. With around 20 attendees and 4 speakers, the half-day workshop was fairly small, but it was definitely an interesting start for, hopefully, more workshops to come.

The invited speaker was Bijan Parsia, who gave a rather awesome talk laying out the landscape of what we generally refer to as ‘errors’ in OWL ontologies. We can categorise errors into logical and non-logical errors. Logical errors include the ‘classical’ errors such as incoherence and inconsistency, wrong entailments, missing entailments, but also less obvious problems such as tautologies and ‘concept idleness’. Non-logical errors are problems that we might not think of straight away when we talk about debugging; these include wrong naming of concepts and properties, structural irregularities, and performance problems.

The first research paper by Valentina Ivanova, Jonas Laurila Bergman, Ulf Hammerling and Patrick Lambrix was dealing with the debugging of ontology alignments based on an interesting use-case (ToxOntology, an ontology describing toxicological information of food). The main idea was to validate mappings based on the structural relations of concepts in the  ontology. Valentina also demoed a prototype of the RepOSE tool which nicely combines the “accept/reject” task of debugging alignments with a graph-based user interface (see screenshot below), making the job slightly less painful.


Next up was Tu Anh Nguyen from the Open University who presented her work on justification-based debugging using patterns and natural language. The approach taken to measuring the cognitive complexity of justifications is very appealing: They first identified a set of frequently occurring patterns in justifications which were sub-sets of justifications of maximally 4 axioms, using justifications from around 500 ontologies. The  50 most frequent patterns were then translated into natural language and evaluated using a mechanical turk style web service by presenting the ‘rule’ to a user, then asking them to decide whether a given entailment followed from that rule. This is quite close to what we did in our complexity study, but with the advantage that the natural language rules could be presented to a much wider audience than our DL/OWL Manchester syntax patterns. The result of the user study was a ranking of the most frequent rules, which can be used to rank the complexity of OWL justifications – at least in their natural language form. It would obviously be interesting to find out whether the complexity measure translates directly to Manchester syntax as used in Protege, for example.

And finally, I presented my paper “Declutter your justifications“, which deals with grouping multiple justifications based on their structural similarities. My talk followed on quite nicely from Tu Anh’s presentation, as she basically solved the problem of “obvious proof steps” using her natural language approach to testing justification sub-patterns. The slides for my presentation are available here.

In summary, this first WoDOOM turned out really well, and the papers presented were very interesting. I also have to admit that I was very pleased with the rate of 75% female speakers / first authors, which is pretty awesome. I’m hoping that we’ll have some more papers next year, as at least two had a very similar approach to debugging (justifications!), especially given Bijan’s highlighting other errors which are currently not considered in most debugging approaches.

[Photo of Galway by Phalinn Ooi, cc-licensed]

Videolectures from ISWC 2011 online

The video lectures from ISWC 2011 have been online for a while, so if you’re interested in checking out our talks, you can do this here: http://videolectures.net/iswc2011_research/

The Manchester talks are:

Try and sit through the first 5 minutes of my talk, I *will* slow down eventually.


At this year’s International Semantic Web Conference ISWC 2011, Manchester will be heavily present with 4 papers in the research track, of which 2 focus on justifications.

In the first one, which we presented in similar form at DL, we discuss our user study on the cognitive complexity of OWL justifications. It is quite interesting that, despite the fairly large body of work on explanation for entailments, there have only been few attempts at analysing the effectiveness of explanation approaches with regard to how understandable they are to the average OWL user. Starting with fine-grain justifications (Kalyanpur, Parsia, Cuenca-Grau, DL 2006) which were then defined formally as laconic & precise justifications (Horridge, Parsia, Sattler, ISWC 2008 – won best paper award at the conference), there has been a line of research dealing with making justifications easier to understand by removing superfluous parts (i.e. parts of the axioms in the justification that are not necessary for the entailment to hold). The notion of (non-)laconicity is based on the assumption that superfluous parts distract the user and therefore make it harder to understand why the entailment holds – which, intuitively, seems sound. Moving away from distracting parts only, we want to have a general picture of how easy or difficult justifications are to understand, and why. These ideas are captured in a complexity model (again, Horridge, Parsia, Sattler) which considers certain properties of a justification and the respective entailment, and gives us a score for the cognitive complexity, or hardness of the justification. The considerations behind this model, issues related with cognitive issues, and the validation of the model are discussed in the paper “The Cognitive Complexity of OWL Justifications”, which we’re presenting at ISWC in October.

The second paper is part of my own PhD research and deals with “The justificatory structure of the NCBO BioPortal Ontologies”. Again, this is a topic which has hardly been touched yet by other researchers who deal with explanation in the form of minimal entailing subsets (i.e. MinAs, MUPS  – if for unsatisfiable classes, justifications… maybe we should simply call them MEHs – minimal entailment-having subsets?). While we generally focus on a) finding efficient mechanisms for computing all MEHs, errmm, justifications, or b) analysing the cognitive complexity of individual justifications, there is only a small body of work that looks at multiple justifications. This seems an obvious step, since we know that considering individual justifications for an entailment in isolation does not give us the full picture of why an entailment holds in an ontology. The consequences can be only partial understanding, ineffectual repair attempts, or over-repair (removing or modifying more than necessary). Further, we even know that those multiple justification have relations between them, as they can share axioms, entail each other, be subsets of one another (if we consider justifications for multiple entailments), etc.  To which degree multiple justifications occur in an ontology, and what relations there are between them, can tell us more about the ontology than the simple metrics we see in ontology editors – in the paper I call it ‘making implicit structure explicit’. An analysis of the prevalence of multiple justifications and their relations in a set of BioPortal ontologies is the focus of the paper, which, again, will be turned into a talk at ISWC.

And, in a amusing move, we have had the research track session which contains the two talks named after us: The ISWC organisers decided to call it “MANCHustifications”. You know where you can find us.

Off to DL…

We’ve been busy preparing talks and posters for the Description Logics workshop 2011 in Barcelona, which will start on Wednesday 13th July. I’m really looking forward to meeting researchers from the DL community and to hearing some interesting talks! The workshop program is packed with talks and presentations, so I’m sure there will be plenty to discuss.

The Manchester group will be present with two posters (Samantha Bail, Bijan Parsia and Ulrike Sattler. ‘Extracting Finite Sets of Entailments from OWL Ontologies’ and Chiara Del Vescovo. The Modular Structure of an Ontology: Atomic Decomposition with Labels’) and two talks (Matthew Horridge, Samantha Bail, Bijan Parsia and Ulrike Sattler. ‘The Cognitive Complexity of OWL Justifications’ and Rafael S. Gonçalves, Bijan Parsia and Uli Sattler. ‘Analysing Multiple Versions of an Ontology: A Study of the NCI Thesaurus’).

I’m planning to write a short review of the workshop on this blog during / after my stay in Barcelona – the weather forecast looks pretty bad, so I presume I’ll have time in my hotel room to write quite a bit.