Multiple Pose Human Body Database (LSP/MPII-MPHB)

Multi-pose human body detection has many important applications in practice, e.g, in human gesture estimation, one often needs to detect the position of the human body first to provide a reference location for other human parts, such as head, hands, feet, and so on. One of the most studied topics related to this is the pedestrian detection. But pedestrian detection by itself concerns mainly about the human body in upright positions, while people in general will not always be upright - they can be bending, sitting, lying, or in other poses (see Fig. 1 for some illustration of human body in different poses), highlighting the need for detecting human body under arbitrary poses becomes necessary.

Fig. 1. Illustration of human body under different poses. (Images from MPII Human Pose [3] with the bounding boxes annotated in this work.)

 

For the study of object detection, there are many public data sets for researchers to use such as Pascal VOC 2007, VOC 2010, ILSVRC 2010, and ILSVCR 2012. However, these do not focus on the category of the human being among many other categories of objects. Actually, there currently does not exist a large-scale data set tailored for the task of human body detection. Fortunately, there have been many datasets built for pose estimation like FLIC, LSP [2] and MPII Human Pose [3]. Although there are many annotations of the locations of different body parts in these datasets, they all lack the annotation of bounding boxes about the whole human body, and such bounding boxes needed for model training and performance evaluation can not be reliably derived from the available annotations of body parts. Therefore, these data sets cannot be directly used for human body detection. To this end, we annotate a new dataset named LSP/MPII-MPHB (Multiple Poses Human Body) for human body detection, by selecting over 26K challenging images in LSP and MPII Human Pose and annotating human body bounding boxes on each of the selected images.

The resulting dataset, named LSP/MPII-MPHB, contains 26,675 images and 29,732 human bodies. There is at least one human body per image, and some may contain multiple people. Among these images, 2,000 are from LSP and 24,675 are from MPII Human Body. We compute the size ratio of the ground-truth bounding box to the whole image and count the frequency histogram, as shown in Fig. 2. One can see that almost 70% ground-truth’s size ratio is less than 10%, indicating that it is challenging to detect the human beings in the MPHB data set.

Fig. 2. Distribution of the size ratio of human beings in the MPHB dataset.

 

Human bodies in LSP/MPII-MPHB are under various poses. They can be roughly divided into six categories, i.e., bent, kneeling, lying, partial, sitting and upright. Fig. 3 gives an illustration of each of these. Accordingly, Table 1 gives the standard of how we partition these images and the number of images belonging to each category. The table shows that the number of images in each category of poses is unbalanced - the predominant category is "bent" while the poses of "kneeling" and "lying" are relatively few.

Fig. 3. The image examples of 6 human body pose images.

Pose Type	Description	Number
Bent	at least one of body part is bent (e.g., stoop or horse step)	10,229
Kneeling	at least one of the two knees touching something	1,053
Lying	sleeping or swimming, in horizontal position and on its back	1,123
Occlusion	only part of the body	4,040
Sitting	buttocks touching something	5,739
Upright	standing without any bend	4,492

Table 1: The description and number of 6 human body pose images

We design an evaluation protocol by dividing these images into three partitions. Particularly, the number of images(human bodies) are respectively 8,385(9,732), 8,110(8233), 10,180(11,767) for training set, validation set and test set. For performance evaluation, a detector outputs a list of bounding boxes with associated confidence (rank) and detections were assigned to ground-truth objects and judged to be true/false positives by measuring bounding box overlap. Following common practice in this field (cf., the Pascal VOC [4] and ILSVRC [5]), when testing we calculate the IoU (Intersection-over-Union) for each test image and instances whose IoU (Intersectionover-Union) with the ground-truth is more than 0.5 are considered as positive. The final performance is summarized using the average precision (AP) metric.