Automated essay scoring is an important tool in modern education, aim ing to reduce the time and effort required for manual grading. However, exist ing systems often struggle with consistency, fairness, and accuracy across diverse datasets and linguistic patterns. To address these challenges, this study investigates the application of Large Language Models (LLMs) for automated essay scoring, focusing on BERT, RoBERTa, ALBERT and GPT2. The models were evaluated using key metrics such as Mean Squared Error (MSE), Normalized Mean Squared Error (NMSE), and R2. Each model will have a Roman numeral, BERT (I), AL BERT (II), RoBERTa (III) and GPT2 (IV). Then, Each model will be fine tuned in three scenarios (A,B,C). The results show that BERT I-B that with text vector simplification, which can help the model to understand the data more easily and im prove performance is outperforms all other models, achieving the highest accuracy and demonstrating the most balanced performance across multiple datasets. On the Watermarking & Steganography dataset, BERT I-B achieved the lowest MSE (3.712121), lowest NMSE (0.577150), and the highest R2 Score (0.809663), indi cating its superior ability to understand complex data patterns. Additionally, BERT I-B recorded an MSE of 4.607143 and an NMSE of 0.534900 on the Coding & Compression dataset, which remains competitive compared to other models. These f indings highlight BERT I-B’s strong performance in both complex and less com plex datasets, providing a reliable solution for automated essay grading. This re search contributes to educational technology by offering an efficient, scalable, and accurate grading system. Future work could focus on enhancing the model’s in terpretability and expanding the datasets to include a broader range of responses, further improving the system’s generalizability.
Keywords: Automated Essay Scoring, Large Language Models, Fine Tuning, GPT2, BERT.