Indonesia’s geographical location, with its tropical climate and fertile soil, provides great potential for producing various types of fruit. In 2022, Indonesia’s fruit production reached 55,424,366 tons, making it the 7th largest fruit producer in the world. However, challenges arise in the long distribution chain, which can result in reduced freshness and fruit damage. Traditional fruit quality sorting mechanisms face problems in terms of efficiency and effectiveness. Therefore, a more efficient method is needed, and one solution is to use computer vision with image classification. Image classification using the ResNet50 and MobileNetV2 Convolutional Neural Network (CNN) models has proven to provide good recognition performance. In this research, a combined dataset from FruitNet and FreshandRotten, consisting of 13,246 images with 5 different types of fruit, was used. The dataset is divided into fresh and rotten classes.This study focuses on comparing the MobileNetV2 architectural model, which is known to be lighter than ResNet50. The results obtained for the best parameters used ImageNet weights with trainable set to false, a learning rate of 0.001, and a batch size of 64. The MobileNetV2 model achieved an accuracy of 96.99% and an F1- Score of 97.01%, while ResNet50 achieved an accuracy of 97.74% and an F1-Score of 97.75%. This shows that there is no significant difference in performance between MobileNetV2 and ResNet50. However, in terms of model size, MobileNetV2 is superior, with a size of 9.1MB compared to ResNet50, which is ten times larger at 90.1MB. This highlights the advantage of MobileNetV2 over ResNet50.