Institutional Repository UIN Sunan Kalijaga YogyakartaDETEKSI CACAT KODE PADA BAHASA PEMROGRAMAN PYTHON MENGGUNAKAN MODEL BAHASA CODET5-SMALL BERBASIS FOCAL LOSSNIM.: 24206051005Hanny Handayani SucintaauthorCode defect detection is an important component of software quality assurance because it helps identify potentially faulty code before such defects affect testing or production environments. In Python, this task is particularly challenging due to the dynamic nature of the language and the presence of class imbalance in defect detection datasets. This study aims to develop a binary classification model for detecting defects in Python code using CodeT5-small with a Sequence-to-Sequence architecture. The dataset was obtained from PyTraceBugs and processed through preprocessing, deduplication, tokenization, and stratified data splitting based on label and length-bin to preserve both class distribution and code-length distribution. To address class imbalance, WeightedRandomSampler was applied at the data distribution level, while Focal Loss was used at the loss function level. The model was evaluated using accuracy, precision, recall, F1-score, MCC, AUC, confusion matrix, a length-only baseline, and a length-matched test. The evaluation on the test set achieved an accuracy of 80.14%, defective-class precision of 67.89%, recall of 86.51%, F1-score of 76.08%, MCC of 60.79%, and AUC of 83.73%. Compared with the length-only baseline, the proposed model showed clear improvements across the main evaluation metrics. In the length-matched test, the model maintained a recall of 83.39% and an F1-score of 77.30%, indicating that its performance was not solely driven by code length. The findings suggest that the combination of CodeT5-small, WeightedRandomSampler, and Focal Loss improves sensitivity toward the defective class while providing a more robust evaluation against potential code-length bias.005.12 Software System Analysis and Design/Sistem Analisa dan Desain Perangkat Lunak2026-05-18UIN SUNAN KALIJAGA YOGYAKARTA;FAKULTAS SAINS DAN TEKNOLOGIThesis

For work being deposited by its own author: In self-archiving this collection of files and associated bibliographic metadata, I grant Institutional Repository UIN Sunan Kalijaga Yogyakarta the right to store them and to make them permanently available publicly for free on-line. I declare that this material is my own intellectual property and I understand that Institutional Repository UIN Sunan Kalijaga Yogyakarta does not assume any responsibility if there is any breach of copyright in distributing these files or metadata. (All authors are urged to prominently assert their copyright on the title page of their work.)

For work being deposited by someone other than its author: I hereby declare that the collection of files and associated bibliographic metadata that I am archiving at Institutional Repository UIN Sunan Kalijaga Yogyakarta) is in the public domain. If this is not the case, I accept full responsibility for any breach of copyright that distributing these files or metadata may entail.

Clicking on the deposit button indicates your agreement to these terms.