Online dense activity detection

Abstract

Dense activity detection is a subtask of activity detection that aims to localise and identify multiple human activities in video clips. Existing methods adopt offline frameworks that require video frames to be available when activity detection begins. These offline methods are unable to be applied to online scenarios. An online framework is proposed for dense activity detection. The framework has two stages: warm-up and detection. Warm-up is the initialisation of dense activity detection, which generates a contextual model called an online aggregated-event. After that, the method moves into the detection stage, which consists of two modules: coarse label prediction and refined label prediction. Coarse label prediction predicts activity labels by taking the online aggregated-event as a priori; then, prediction is refined by two techniques, human–object interaction detection and online relation reasoning. The proposed method is evaluated using two dense activity datasets: Charades and AVA. The experimental results show that the proposed method has better performance than existing offline methods after the whole video input is added to the algorithm.