Particle swarm optimization with soft search space partitioning for video-based markerless 3D human pose tracking

Abstract

The task for this master’s thesis was to develop, implement, and evaluate an algorithm for markerless human pose tracking. The algorithm should be based on the freely available HumanEva framework to enable a quantitative comparison to the state of the art. This framework uses an articulated body model to estimate the pose in an analysis by synthesis approach. Some of the hard problems of markerless motion capture come from self-occlusions and 3D-2D mapping ambiguities. These problems are alleviated by using multiple cameras. But the hardest problem remains: The high number of parameters that define the pose of the body model. This thesis proposes a new algorithm called soft partitioning particle swarm optimization (SPPSO) which formulates pose tracking as an optimization of the 31 parameters that define the pose of the body model. The optimization objective is a fitness function which represents the match between the body model and the video frames. To tackle the dimensionality problem, SPPSO divides the optimization into two stages that exploit the hierarchical structure of the model. The first stage only optimizes the six most important parameters that define the global orientation and position of the model. In contrast to hard hierarchical partitioning schemes, soft partitioning refines the estimation of these parameters in the second optimization stage. In addition to presenting SPPSO, the thesis also provides a literature review of the current research in the field with an emphasis on approaches that use particle swarm optimization. The performance of SPPSO was evaluated in various tracking experiments on the Lee walk sequence, a standard dataset from the HumanEva framework which contains multi-view video and ground truth motion capture data. The most important result of these experiments is that SPPSO performs better than the annealed particle filter, a common benchmark algorithm, at a frame rate of 20fps, and equally well at 60fps. The better performance at the lower frame rate is attributed to the explicit exploitation of the hierarchical model structure. The experiments also showed that SPPSO performs better than a single-stage global optimization and better than a variant with hard par- titioning. An important conclusions from the literature review is that any future developments should combine a stochastic global optimization, such as SPPSO, with a local refinement stage to make tracking more accurate. The global optimization stage allows such an algorithm to recover from tracking failure and the local refinement efficiently improves the tracking accuracy.