Ambiguity detection methods for improving handwritten mathematical character recognition accuracy in classroom videos

摘要

In classroom videos, recognizing mathematical content, handwritten on the whiteboard presents a unique opportunity in the form of audio content spoken by the instructor. This recognized audio content can be used to improve the character recognition accuracy by providing evidence in corroboration or contradiction of the output options generated by the primary, video based recognizer. However, such audio-video based disambiguation also has the potential to introduce errors in what may have been the correct output from the video based recognizer. In this paper, we focus on improving the character recognition accuracy by developing ambiguity detection methods that can be used to determine the set of potentially incorrect outputs from the video based recognizer and, for each such output, determining the subset of possibly correct output options that must be forwarded for audio-video based character disambiguation. In this paper, we propose, implement and evaluate a number of such ambiguity detection methods.