METHOD AND SYSTEM FOR PROCESSING DATA USING ELEMENT-WISE MULTIPLICATION AND MULTIMODAL RESIDUAL LEARNING FOR VISUAL QUESTION-ANSWERING

摘要

Disclosed are a method for processing data using element-wise multiplication and multimodal residual learning for visual question and answer and a system thereof. The method for processing data comprises: a first step of extracting a visual feature vector on an input image and extracting a question vector on an input question; a second step of calculating a first result value by reflecting linear mapping and hyperbolic tangent mapping for deep residual learning to the question vector; a third step of calculating a second result value by reflecting the linear mapping and the hyperbolic tangent mapping to the visual feature vector; a fourth step of calculating a third result value on the element-wise multiplication between the first result value and the second result value; and a fifth step of calculating a fourth result value on the element-wise multiplication between the third result value and the result value made by reflecting the linear mapping to the question vector.