DETEKSI CACAT KODE PADA BAHASA PEMROGRAMAN PYTHON MENGGUNAKAN MODEL BAHASA CODET5-SMALL BERBASIS FOCAL LOSS

Hanny Handayani Sucinta, NIM.: 24206051005 (2026) DETEKSI CACAT KODE PADA BAHASA PEMROGRAMAN PYTHON MENGGUNAKAN MODEL BAHASA CODET5-SMALL BERBASIS FOCAL LOSS. Masters thesis, UIN SUNAN KALIJAGA YOGYAKARTA.

Preview	Text (DETEKSI CACAT KODE PADA BAHASA PEMROGRAMAN PYTHON MENGGUNAKAN MODEL BAHASA CODET5-SMALL BERBASIS FOCAL LOSS) 24206051005_BAB-I_IV-atau-V_DAFTAR-PUSTAKA.pdf - Published Version Download (6MB) | Preview
	Text (DETEKSI CACAT KODE PADA BAHASA PEMROGRAMAN PYTHON MENGGUNAKAN MODEL BAHASA CODET5-SMALL BERBASIS FOCAL LOSS) 24206051005_BAB-II_sampai_SEBELUM-BAB-TERAKHIR.pdf - Published Version Restricted to Registered users only Download (9MB) | Request a copy

Abstract

Code defect detection is an important component of software quality assurance because it helps identify potentially faulty code before such defects affect testing or production environments. In Python, this task is particularly challenging due to the dynamic nature of the language and the presence of class imbalance in defect detection datasets. This study aims to develop a binary classification model for detecting defects in Python code using CodeT5-small with a Sequence-to-Sequence architecture. The dataset was obtained from PyTraceBugs and processed through preprocessing, deduplication, tokenization, and stratified data splitting based on label and length-bin to preserve both class distribution and code-length distribution. To address class imbalance, WeightedRandomSampler was applied at the data distribution level, while Focal Loss was used at the loss function level. The model was evaluated using accuracy, precision, recall, F1-score, MCC, AUC, confusion matrix, a length-only baseline, and a length-matched test. The evaluation on the test set achieved an accuracy of 80.14%, defective-class precision of 67.89%, recall of 86.51%, F1-score of 76.08%, MCC of 60.79%, and AUC of 83.73%. Compared with the length-only baseline, the proposed model showed clear improvements across the main evaluation metrics. In the length-matched test, the model maintained a recall of 83.39% and an F1-score of 77.30%, indicating that its performance was not solely driven by code length. The findings suggest that the combination of CodeT5-small, WeightedRandomSampler, and Focal Loss improves sensitivity toward the defective class while providing a more robust evaluation against potential code-length bias.

Share this knowledge with your friends :

Actions (login required)

View Item