Dialog system evaluation method, dialog system evaluation apparatus, and program

摘要

PROBLEM TO BE SOLVED: To automatically estimate, without manual intervention, a chat dialog system having no task.SOLUTION: A reference sentence database 10 has stored an input sentence and a plurality of reference sentences predetermined for the input sentence. An output sentence acquisition unit 11 inputs an input sentence into a dialog system 9 having no particular task, and acquires a system output sentence from the dialog system 9. An inter-sentence similarity calculation unit 13 calculates a degree of inter-sentence similarity between the system output sentence and each of the reference sentences. An evaluation value calculation unit 12 calculates a system evaluation value to evaluate the system output sentence on the basis of the plurality of reference sentences.SELECTED DRAWING: Figure 1