Research on Mask Wearing Detection of Natural Population Based on Improved YOLOv4
^††thanks: Xuecheng Wu, Mengmeng Tian, and Lanhang Zhai are all with school of Cyber Science and Engineering, Zhengzhou University. (Corresponding authors: Xuecheng Wu.)

Coordinate Attention [2] is a new attention mechanism proposed by Qibin Hou and others at the National University of Singapore. This attention mechanism innovatively embeds specific positional information into inter-channel attention. This mechanism solves the common problems in SE, BAM and CBAM, which has better results and avoids introducing significant computational overhead. In this paper, after the convolution transformation layer denoted as DarknetConv2D-BN-Mish in the backbone, a Coordinate Attention Module is added before the residual convolution transformation blocks in order to strengthen the semantic representation of the shallow feature maps and obtain richer feature information over a larger region, which can further improve the performance of backbone. Compared with the original YOLOv4, the improved YOLOv4 combined with the Coordinate Attention Module can locate and identify the objects of interest more accurately.

Coordinate Attention first performs maximum average pooling in the horizontal and vertical directions and then conducts transformation to encode the specific positional information accurately. Finally, the specific positional information is fused by weighting on the feature channels. The Coordinate Attention Module is divided into two steps: coordinate information embedding and coordinate attention generation.

In the step of coordinate information embedding, the Coordinate Attention Module first divides the global average pooling into a pair of 1D feature encoding operations so that the Coordinate Attention Module can capture the remote spatial interactions with precise positional information. Given the input $X$, we deploy the two pooling kernels of size (H, 1) and (1, W) to encode each specific feature channel along with the horizontal and vertical directions. The abovementioned two transformations perform feature aggregation along with the horizontal and vertical directions and yield a pair of direction-aware feature maps. As a result, the output of the $c$-th channel at height $h$ and width $w$ can be formulated as Eq. 1 and Eq. 2, respectively.

The $q_c^h$ and $q_c^w$ in Eq. 1 and Eq. 2 denote the outputs which are obtained after the average pooling operation along the horizontal and vertical directions, respectively.

In the coordinate attention generation, as shown in Eq. 3, we first stack the specific pair of direction-aware feature maps generated in the step of coordinate information embedding and compress the number of feature channels by a $1\times 1$ convolution transformation operation. We then encode the precise positional information in the horizontal and vertical directions through a BatchNorm layer and a ReLU layer. Afterward, $f$ is divided into two separate feature tensors $f^h$ and $f^w$ along the spatial dimensions, and two convolution transformation operations are utilized respectively to transform $f^h$ and $f^w$ to feature tensors with the same number of channels to the input $X$. Then we can get $g^h$ and $g^w$ by normalized weighting, as shown in Eq. 4 and Eq. 5.

In Eq. 3, $F_1(\cdot )$ denotes first stacking the two input feature tensors, followed by a $1\times 1$ convolution transformation and a BatchNorm layer. $\delta (\cdot )$ represents the ReLU activation function and $f\in R^{C/r\times (H+W)}$ is the encoded intermediate feature map. In Eq. 4 and Eq. 5, $F_h(\cdot )$ and $F_w(\cdot )$ respectively denote a $1\times 1$ convolution transformation operation. $\sigma (\cdot )$ represents a sigmoid function. The outputs $g^h$ and $g^w$ are then extended and utilized as the Coordinate Attention weights, respectively. The final output of Coordinate Attention is shown in Eq. 6.

In Eq. 6, $x_c(i,j)$ denotes the original input feature tensor, $g_c^h\left(i\right)$ and $g_c^w\left(j\right)$ denotes the horizontal attention weight and the vertical attention weight, respectively. And $y_c(i,j)$ denotes the weighted feature maps.

Although the neck network in the original YOLOv4 can strengthen the semantic representation capability of feature maps to a certain extent, it is susceptible to the natural scenes, mutual occlusion of people, and inconspicuous feature discrimination after people wearing masks during detection. Therefore, we make a series of neck network structural improvements to the original neck network. First, we increase the number of convolution transformation layers before and after the Spatial Pyramid Pooling layer from three to five. Second, inspired by the original YOLOv4, we increase the number of convolution transformation layers from one to three before $L3$ and $L4$ are fed into the neck network, respectively. The improved neck network deepens the capacity and depth of the overall network, obtaining larger receptive fields and richer semantic feature information, significantly improving the model performance.

K-means clustering [3] is an iterative solution clustering analysis approach. The nine anchor boxes in the original YOLOv4 are computed initially by clustering the MS-COCO dataset utilizing K-means clustering. The MS-COCO contains 80 types of objects, and the sizes of the different objects varies greatly. If it is directly employed in the mask wearing detection, the sizes of some anchor boxes are unreasonable. Therefore, the K-means clustering is deployed again to perform clustering calculations on the NPMD dataset, which is collected by us, to obtain the sizes of the nine anchor boxes suitable for our dataset. In detail, first, we initialize the number of categories and cluster centers. We then calculate the distance, 1-IoU, between each bounding box and all cluster centers. After that, we choose the nearest cluster center as its category and utilize the average of each category cluster as the category center for the next iteration. We repeat the above two steps until the center position of each category does not change anymore. Finally, we can get the reasonable sizes of nine anchor boxes for our NPMD dataset.

Currently, there are very few datasets on the detection of mask wearing in natural scenes, the detection environment is too single, and there is a general lack of images in the category of incorrect mask wearing. Therefore, we propose NPMD (Natural Population Mask Detection) dataset, which mainly includes images that meet our requirements crawled by the web crawler, those images selected from public datasets such as MAFA and RMFD which meet our requirements, and some images obtained by extracting frames from online videos. In addition, we have made some images by ourselves to expand the NPMD dataset further. The final original dataset contains 7,854 images in three categories: mask-wearing correctly, without mask-wearing, and mask-wearing incorrectly, involving multiple public natural scenes such as stations, subway stations, and supermarkets. Three common examples of mask-wearing incorrectly include uncovering the nose, uncovering the mouth and nose, and mask on the chin. The dataset is in the Pascal VOC format and annotated employing the LabelImg tool. Because the total amount of images in the original NPMD dataset is relatively small and the number of mask wearing incorrectly categories is also relatively small, we have adopted a variety of random geometric data augmentation, such as affine rotation transformation, Gaussian filtering, random color enhancement, and median blur. The random data enhancement method is utilized to further expand the total amount of images in NPMD dataset. After data enhancement, the total number of images in our NPMD is 11447. The number of each label in NPMD after random data augmentation is shown in Tab. I. In this approach, we can further reduce the risk of overfitting during the model training and improve the robustness and generalization of our proposed YOLOv4.

The evaluation indicators chosen in our experiments are Average Precision ($AP$) and Mean Average Precision ($mAP$). The $AP$ measures the accuracy of model performance from the accuracy rate $P$ and the recall rate $R$. The accuracy rate represents the proportion of samples that are actually positive and predicted to be positive to all samples that are predicted to be positive. The recall rate represents the proportion of samples that are actually positive and predicted to be positive to all samples that are actually positive. The general formulas of $P$ and $R$ are shown as Eq. 7 and Eq. 8, respectively.

In the abovementioned formulas, $TP$ represents a positive sample detected as correct; $FP$ represents a negative sample detected as a positive sample; $FN$ represents the positive sample detected as a negative sample.

$AP$ is calculated by the integral of the accuracy-recall rate curve. The higher the $AP$, the better the model performance, and its general formula is shown in Eq. 9.

$mAP$ is the average value of APs for each category, which is utilized to measure the average detection accuracy of multiple object categories. The value of $mAP$ can reflect the comprehensive performance of the detectors in all categories. The general formula of $mAP$ is shown in Eq. 10.

We propose a series of ablation experiments to verify the effectiveness of improved YOLOv4. We conduct four sets of ablation studies, and the specific schemes are as follows: the first group, the original YOLOv4, which is deployed as a control to determine whether our improvement component is effective or not; the second group is deployed to determine the effectiveness of the Coordinate Attention Module; the third group is used to determine the effectiveness of the neck network structural improvement; the last group is utilized to determine the effectiveness of K-means clustering for performance enhancement. The detailed results of ablation experiments are shown in Tab. II.

Through the ablation experiments, we can clearly observe that each improvement component positively impacts the improved YOLOv4. The Coordinate Attention Module has significantly increased the AP by 2.67%. Furthermore, we can conclude that the reason for the performance enhancement of improved YOLOv4 is that the Coordinate Attention Module is applied to the front-end of the backbone for guidance, resulting in a more assertive semantic representation of shallow-level feature images. At the same time, the improvement of the network structure further expands the receptive fields of deep-level feature images in disguise and then can extract more profound and richer feature information. The employment of K-means clustering makes the sizes of the nine anchor boxes we set more aligned with our NPMD dataset, further improving the model performance.

First of all, the APs of improved YOLOv4 for three categories are 96.12%, 93.99%, and 95.84%, respectively. The mAP of our improved YOLOv4 achieves 95.32%. Compared with the original YOLOV4, the mAP has increased by 4.06% with a little bit of computational overhead.

In order to comprehensively evaluate the performance of our improved YOLOv4, we then compare the improved YOLOv4 with other CNN-based state-of-the-art object detectors on the NPMD dataset. The detailed results are shown in Tab.3. Specifically, “Cat.1”, “Cat.2” and “Cat.3” represent Mask Wearing Correctly, Without Mask Wearing, and Mask Wearing Incorrectly, respectively. We can see that compared with Faster R-CNN, SSD, YOLOv3 and the original YOLOv4, the improved YOLOv4 has significantly improved the APs of the three categories, and the mAP is also better than the abovementioned four SOTA detectors. Our improved YOLOv4 further improves the detection ability of small targeted objects and the processing of detailed features. The effectiveness of our proposed improved YOLOv4 is further confirmed by comparing it with the current mainstream object detectors.