Harvesting Paragraph-level Question-Answer Pairs from Wikipedia

ACL 2018

Harvesting Paragraph-level Question-Answer Pairs from Wikipedia

Jan 28, 2021
|
35 views
|
|
Code
Details
Abstact: We study the task of generating from Wikipedia articles question-answer pairs that cover content beyond a single sentence. We propose a neural network approach that incorporates coreference knowledge via a novel gating mech-anism. As compared to models that only take into account sentence-level information (Heilman and Smith, 2010; Du et al., 2017; Zhou et al., 2017), we find that the linguistic knowledge introduced by the coreference representation aids question generation significantly, producing models that outperform the current state-of-the-art. We apply our system (composed of an answer span extraction system and the passage-level QG system) to the 10,000 top ranking Wikipedia articles and create a corpus of over one million question-answer pairs. We provide qualitative analysis for the this large-scale generated corpus from Wikipedia. Authors: Xinya Du, Claire Cardie (Cornell University)

Comments
loading...