Adapt and Generalize Deep Learning Methods for Activity Recognition on Newborn Resuscitation Videos

Abstract

Low- and middle-income countries have nearly 99 % of deaths of children under 28 days. Complications during childbirths, such as birth asphyxia, account for most of these deaths. To prevent this, strengthening the quality of resuscitation training and providing better quality health service are a major focus.

Major Projects like Safer Births has had a focus to establish new knowledge and develop new technology to reduce the number of deaths. They have collected videos of newborn resuscitation at Haydom Lutheran hospital in Tanzania. This data is useful for evaluating the official resuscitation guidelines by investigating the treatment process and compliance with guidelines. With this data a timeline over the activities like suction, ventilation and stimulation can be extracted. However, issues like low brightness, non-standard camera placement and low frame rate videos persist in these data.

The aims of this thesis is to improve upon previous work involving the Safer Births data and put all the previous work into a single pipeline. Earlier work has used object detection networks to locate and classify objects in the videos to post-process the videos to a format that the activity recognition network could predict on.

The Convolutional Neural Network (CNN) from earlier work like RetinaNet and Inception I3D was tested and had promising results, but still had some issues with detecting objects like the suction penguin and a large amount of False Positives. To improve object detection different kinds of pre-processing methods were applied. The methods to correct the lighting resulted in poorer results, but training a new model with gradient based input together with the original model resulted in a loss of False Positives among most classes. Combining previous binary activity recognition models resulted in a promising gain in suction, but a loss in the ventilation prediction. Adding a more efficient optical flow method for usage with activity recognition encountered issues with training data generation and resulted in worse performance than previous work. Creating new methods in python from previous work was completed with a new pipeline that could predict and generate a timeline, but has worse performance and needs more work.