Automatic Event Detection in Videos using Deep Neural Networks

Abstract

Within a large range of applications in computer vision, Human Action Recognition has become one of the most attractive research fields.This thesis investigates possibilities of applying automatic event detection on large dataset of simulation videos captured during medical training sessions. In a typical training session different scenario-based event can occur and the students undergoing the training must take actions accordingly. These events and actions are manually annotated by an observer using an app or by watching the video after a session. These hand-crafted annotations are later used for evaluating such sessions. This is a manual work which requires human intervention and can quickly become tedious, time consuming and difficult over time (especially when there are a lot of things going on in a particular training or simulation setting). Hence, this thesis aims to solve the challenges by : 1) Providing a baseline approach for automatically detecting events occurring inlong untrimmed videos 2) Activity localization

This thesis is focused mainly on detecting "Washing Hands" activity performed by health care providers and medical students under different settings. The proposed system approach consists of activity recognition and generation of activity timelines using 3D CNNs.The dataset used in this thesis originally contained more than 4000 untrimmed videos with associated annotations, of which only 60% of the data was found tobe relevant but required reliable annotations before it could be fed into the deep neural network. Hence, as an initial step into this thesis a reusable Data Curation tool was developed and used extensively for generation of ground truth annotations.

This thesis proposes a generalized methods for data curation and activity recognition. An overall classification accuracy of 68% was achieved in this work using theproposed method.