STRATIFICATION OF TOKEN TYPES FOR DOMAIN-ADAPTABLE QUESTION ANSWERING SYSTEMS

摘要

A method determines a relevancy of answers to questions based on token relevance in a system capable of answering questions. One or more processors receive a question that is composed of a set of tokens T (T1, T2, . . . , Tn). The processor(s) select tokens T′ (T′1, T′2, . . . , T′m) from the tokens T (T1, T2, . . . , Tn), where each T′j from T′ is a noun, and classify each T′j as a noun type. The processor(s) scan a corpus to identify passages with candidate answers to the question, and analyze the identified passages utilizing noun entries in the passages classified as the noun type. The processor(s) train an artificial intelligence (AI) system to associate a relevancy to the question for the identified passages based on noun types, and then utilize the trained AI system to provide an answer to the question based on an output of the trained AI system.