Guiding Theme C2: Methods for contextual and constraint-based ranking

The goal of this guiding theme is the development of suitable ranking algorithms that will be used in other parts of the project, with a particular focus on multi-document summarization. In particular, we aim at the development of algorithms for learning to rank information items such as sentences according to various criteria such as their importance, a key ingredient in extractive summarization algorithms.

Research results of the first Ph.D. cohort

The main focus of the first phase was to support multi-document summarization by ranking sentences according to their importance. In particular, we developed supervised and unsupervised machine learning methods for estimating the intrinsic importance of text units, and used them as the backbone of the CPSum summarization system. Unlike conventional approaches, CPSum does not rely on centrality or structural features as indicators for information importance, but learns to rank sentences according to their perceived information importance directly from a background corpus (Zopf et al., 2016a). Furthermore, we developed a methodology to evaluate automatically generated summaries without reference summaries (Zopf, 2018a). The basic summarization algorithm of CPSum, which generates contextual rankings of sentences for addressing both importance and redundancy jointly (Zopf, 2015; Zopf et al., 2016b), can not only learned information importance but also use a wide variety of different annotation types that are contributed by other guiding themes, such as named entities (A1), events and relations between them (A2), opinions (A3), concepts (B2), motifs (C1), and frames (C3), to estimate importance of text units. We published a joint work that integrates the results from these work packages into our summarization system (Zopf et al., 2018b).

Ongoing project of the 2nd Ph.D. cohort

In the second phase, we continue this work and extend it in two ways. On the one hand, in the first phase of AIPHES, we partly relied on crowd-sourcing for obtaining training signals for information importance, which we now intend to replace with incidental signals that can be obtained from background corpora or other information, an approach that have already started to explore in (Zopf et al. 2016a) and (Zopf, 2018a). On the other hand, instead of focusing solely on information importance, we want develop a unified framework for exploiting incidental signals to rank sentences according to different criteria such as importance, validity, trustworthiness and more.

Our framework will use preference learning and ranking as the underlying formalism that allows us to encode and aggregate incidental supervision signals. In this particular setting, several challenges need to be addressed: (i) identifying the informative cues capable of generating the incidental supervision signals from available resources; (ii) exploiting these signals in order to produce good models to rank sentences according to different criteria, such as validity, importance, style or redundancy; (iii) learning to adequately aggregate local rankings in order to produce a final ranking that take into account the preferences determined by different criteria. This problem can be seen as constraint-based ranking.

This guiding theme will benefit from various information identification methods as inputs such as those to be developed in A1, A3, B2 and C1. A2 can additionally provide information about relations between events, such as causal or temporal relations, that can be interpreted as incidentals signals. Moreover, the results of this guiding theme can be used as support for C3, D1 and D2.

People

• PI: Prof. Dr. Johannes Fürnkranz
• Co-Supervisor: Dr. Eneldo Loza Mencía
• First Cohort PhD student: Markus Zopf
• Second Cohort PhD student: Aissatou Diallo

References

• Fürnkranz, J. und Hüllermeier, E., editors (2011). Preference Learning. Springer-Verlag.
• Fürnkranz, J., Hüllermeier, E., Loza Mencía, E., und Brinker, K. (2008). Multilabel Classification via Calibrated Label Ranking. Machine Learning, 73(2):133–153.
• Hüllermeier, E. und Fürnkranz, J. (2010). On Predictive Accuracy and Risk Minimization in Pairwise Label Ranking. Journal of Computer and System Sciences, 76(1):49–62.
• Hüllermeier, E., Fürnkranz, J., Cheng, W., und Brinker K. (2008). Label Ranking by Learning Pairwise Preferences. Artificial Intelligence, 172(16-17):1897–1916.

Publications

go to TU-biblio search on ULB website