Automatically Extracting High-Quƒality Negative Examples for Answer Selection in Quƒestion Answering

摘要

We propose a heuristic called “one answer per document” for automaticallyrnextracting high-quality negative examples for answerrnselection in question answering. Starting with a collection ofrnquestion–answer pairs from the popular TrecQA dataset, we identifyrnthe original documents from which the answers were drawn.rnSentences from these source documents that contain query termsrn(aside from the answers) are selected as negative examples. Trainingrnon the original data plus these negative examples yields improvementsrnin e‚ectiveness by a margin that is comparable to successivernrecent publications on this dataset. Our technique is completelyrnunsupervised, which means that the gains come essentially for free.rnWe con€rm that the improvements can be directly aŠributed to ourrnheuristic, as other approaches to extracting comparable amountsrnof training data are not e‚ective. Beyond the empirical validationrnof this heuristic, we also share our improved TrecQA dataset withrnthe community to support further work in answer selection.