Optimization of object detection in newborn resuscitation videos

Abstract

99% of deaths of children under 28 days of age takes place in low- and middle-income countries. Most of these deaths are due to complications during childbirth. Newborn resuscitation heavily revolves around ventilation, as the cause of newborn cardiac arrests are predominantly asphyxial. Having properly trained and equipped health care providers is recognized as a key element in lowering newborn mortality. Since 2013 the Safer Births project has been collecting data during resuscitation episodes at Haydom Lutheran hospital in Tanzania. The Safer Births project is a collaborative project between several actors, including the University of Stavanger, Laerdal Global Health and Stavanger University Hospital. The goal of the project is to gain knowledge and develop equipment in order to better train and equip health care providers to save lives at birth. Among the collected data are video recordings of the resuscitation episodes. The objective of the thesis is to locate and classify objects in the video recordings. This is done through object detection networks: Supervised machine learning systems based on deep convolutional neural networks. The object detection will supplement a larger video interpretation project which final aim is to provide timelines for different activities surrounding newborn resuscitation. The resulting information will be used in analysis of resuscitation activities and the responses of the newborn. A baseline for the object detection was available through previous work, where the YOLOv3 object detection network was used. The previous results were encouraging, but some objects were found hard to detect. The largest potential for improvement lay in the detection of a suctioning device, called the suction penguin. From tests conducted on the same set of images, object detection networks RetinaNet and SSD MultiBox collectively outperformed YOLOv3 for all different object classes. The best overall results was achieved through training the networks on original and augmented image frames from the video recordings, in addition to synthetic images. Here, the best attained average precision for the objects were 82.77 for the heart rate sensor, 78.34 for health care provider hands, 69.75 for the bag-mask resuscitator and 44.22 for the suction penguin. Detection rates on the suction penguin are promising considering the networks were not trained on split images, which for the previous work heavily improved the detection of this object.