In this research, we built a system to detect the use of facial masks in images, especially in public places. Our system used YOLOv4 for detecting the facial masks usage. This research was conducted using dataset with outdoor and indoor test scenarios. In previous research that has been conducted with the title "A Deep Learning Based Assistive System to Classify COVID-19 Face Mask for Human Safety with YOLOv3" conducted in 2020, images of faces with masks and for the rest of the faces without masks, the resulting accuracy is 96% with more than 4000 training times. We used three labels for the dataset, namely correct mask use, incorrect mask use, and not using mask. We used 1984 images for training set and 992 images for testing set. From our experiment, the best results were obtained when making adjustments to the IoU value of 0.5 which has an F1-Score value of 0.9850 when testing for all classes.