CNN-based temporal detection of motion saliency in videos

摘要

The problem addressed in this paper appertains to the domain of motion saliency in videos. However this is a new problem since we aim to extract the temporal segments of the video where motion saliency is present. It turns out to be a frame-based classification problem. A frame will be classified as dynamically salient if it contains local motion departing from its context. Temporal motion saliency detection is relevant for applications where one needs to trigger alerts or to monitor dynamic behaviours from videos. It can also be viewed as a prerequisite before computing motion saliency maps. The proposed approach handles situations with a mobile camera. It involves two main stages consisting first in cancelling the global motion due to the camera movement, then in applying a deep learning classification framework. We have investigated two ways of implementing the first stage, based on image warping, and on residual flow respectively. Experiments on real videos demonstrate that we can obtain accurate classification in highly challenging situations. (C) 2019 Elsevier B.V. All rights reserved.