Guiding Theme B2: Content comparison through large-scale inference and reasoning

Guiding Theme B2 deals with the selection of statements from heterogenous documents that should be part of a summary. This involves the development of methods for measuring the importance of statements.

Research results of the first Ph.D cohort

The main focus of the first phase was on content selection methods for multi-document summarisatization, which is the first step in summarizing a document, whether it is abstractive or extractive. The goal of content selection is the identification and extraction of the key elements from the source documents. We focused on optimization-based content selection, as this yields very promising results and allows us to closely cooperate with guiding theme D2, which is concerned with analyzing and defining suitable evaluation metrics for multi-document summarisatization.

In particular, we formulated content selection as an optimization problem, where the goal is to choose a set of information nuggets which have some desired properties while meeting a length constraint. The objective function of the optimization methods should approximate as closely as possible the quality judgment of a summary. To this end, we developed objective functions to approximate the ROUGE metric (Peyrard and Eckle-Kohler, 2016a) and the Jensen-Shannon divergence (Peyrard and Eckle-Kohler, 2016b). We also explored the use of genetic algorithms to generate training data (Peyrard and Eckle-Kohler, 2017a) and used it to optimize towards the recent Automatic Pyramid metric (Peyrard and Eckle-Kohler, 2017b) and, in a collaboration with C3, towards human judgments (Peyrard et al., 2017).

Ongoing project of the 2nd Ph.D cohort

In the second phase, we broaden content selection to pairwise comparison and evaluation of content, e.g. regarding the similarity or entailment of statements. This allows to distinguish between similar content extracted from different documents, which should be summarized together, and completely independent content, which cannot be summarized as one. The content comparison thus allows a second content selection phase, in which entailed content is not further considered for summarization. Since predicting the similarity or entailment of statements is a difficult task, additional background knowledge is required, which can be learned or extracted at large scale.

In order to apply reasoning to natural language statements in combination with knowledge bases, two connected problems have to be investigated:

• 1) Finding a suitable representation of the two statements that can be used in combination with an appropriate reasoning paradigm. This may be a logical representation, e.g. the parse structures of the statements as obtained using semantic parsing methods, or a numerical representation in the form of distributional semantics such as word or sentence vectors. Accordingly, purely logical or probabilistic reasoning methods may be applied.
• 2) Choosing a (combination of) knowledge base(s), e.g. Wikidata, DBPedia, Freebase, Wordnet, etc, that provides background knowledge usable as resource for the chosen reasoning paradigm.

In addition to creating a new state of the art approach for pairwise classification problems by injecting additional knowledge and applying reasoning methods, research of the second Ph.D. cohort aims at achieving explainability of the classification. For example, an explanation as to why a statement entails another in terms of the reasoning steps and background knowledge involved.

This project provides opportunities for collaboration with Area C (especially semantic role labelling in C3) as well as with Area A in terms of semantic relation extraction (in particular A2).

People

• PI (First Cohort): Prof. Dr. Iryna Gurevych
• PI (Second Cohort): Dr. Claudia Schulz
• Former PI: Dr. Judith Eckle-Kohler
• First Cohort PhD student: Maxime Peyrard
• Second Cohort PhD student: Prasetya Ajie Utama

Publications

go to TU-biblio search on ULB website