Comparison of Random Forest, Decision Tree, and XGBoost Models in Predicting Student Academic Success

Nurbaeti Nurbaeti, Neny Sulistiyaningsih, Ria Rismayati

Sari


Students' academic success is influenced by various academic and non-academic factors. Machine learning (ML) offers an effective approach to predicting academic outcomes by analyzing complex data patterns. However, most previous studies are limited to graduation prediction and rarely incorporate non-academic features or multiple feature selection techniques. This study aims to compare the performance of three ML algorithms Random Forest, Decision Tree, and XGBoost in classifying students’ academic success using a dataset from the UCI Machine Learning Repository, consisting of 4424 records and 37 features. The data underwent cleaning, label transformation, and feature selection using PCA, SelectKBest, and Variance Threshold. Models were trained using a holdout method (80% training, 20% testing) and evaluated based on accuracy, precision, recall, and F1-score. The results show that Random Forest with Variance Threshold achieved the highest accuracy (0.77) and F1-score (0.84) on majority classes. XGBoost followed with 0.75 accuracy, while Decision Tree showed the lowest performance. All models struggled to classify the minority class, indicating challenges related to data imbalance. This research highlights the importance of algorithm choice and effective feature selection in academic classification tasks. It also emphasizes the need for data balancing strategies to reduce class bias. The findings can help educational institutions design data-driven interventions to improve learning outcomes and reduce dropout rates.

Teks Lengkap:

PDF

Referensi


E. F. Agyemang, “ScholarWorks UTRGV Predicting Students ’ Academic Performance Via Machine Learning Algorithms : An Empirical Review and Practical Application,†vol. 15, no. 1, pp. 86–102, 2024.

Orji Fidelia and vassileva Julita, “c,†Univ. Saskatchewan, 2022, [Online]. Available: https://www.researchgate.net/publication/364471938_Machine_Learning_Approach_for_Predicting_Students_Academic_Performance_and_Study_Strategies_based_on_their_Motivation

Y. Chen, J. Sun, J. Wang, L. Zhao, X. Song, and L. Zhai, “Machine Learning-Driven Student Performance Prediction for Enhancing Tiered Instruction,†pp. 1–19, 2025, [Online]. Available: http://arxiv.org/abs/2502.03143

N. Sharma, S. Appukutti, U. Garg, J. Mukherjee, and S. Mishra, “Analysis of Student’s Academic Performance based on their Time Spent on Extra-Curricular Activities using Machine Learning Techniques,†Int. J. Mod. Educ. Comput. Sci., vol. 15, no. 1, pp. 46–57, 2023, doi: 10.5815/ijmecs.2023.01.04.

G. Airlangga, “A Comparative Analysis of Machine Learning Models for Predicting Student Performance : Evaluating the Impact of Stacking and Traditional Methods,†vol. 4, no. 2, pp. 491–499, 2024.

R. Sharma, S. S. Shrivastava, and A. Sharma, “Predicting Student Performance Using Educational Data Mining and Learning Analytics Technique,†J. Intell. Syst. Internet Things, vol. 10, no. 2, pp. 24–37, 2023, doi: 10.54216/JISIoT.100203.

A. I. Gufroni, P. Purwanto, and F. Farikhin, “Academic Performance Prediction Using Supervised Learning Algorithms in University Admission,†Int. J. Informatics Vis., vol. 9, no. 1, pp. 184–194, 2025, doi: 10.62527/joiv.9.1.2974.

D. Andrade-Girón et al., “Predicting Student Dropout based on Machine Learning and Deep Learning: A Systematic Review,†EAI Endorsed Trans. Scalable Inf. Syst., vol. 10, no. 5, pp. 1–11, 2023, doi: 10.4108/eetsis.3586.

R. Rismaya, D. Yuniarto, and D. Setiadi, “Penerapan Algoritma Machine Learning dalam Prediksi Prestasi Akademik Mahasiswa,†2025.

S. K. Ghosh and F. Janan, “Prediction of student’s performance using random forest classifier,†Proc. Int. Conf. Ind. Eng. Oper. Manag., pp. 7089–7100, 2021, doi: 10.46254/an11.20211238.

M. YILDIZ and C. BÖREKCİ, “Predicting Academic Achievement with Machine Learning Algorithms,†J. Educ. Technol. Online Learn., vol. 3, no. 3, pp. 372–392, 2020, doi: 10.31681/jetol.773206.

U. P. Inyang and E. A. Johnson, “Intelligent Ensemble Learning Framework for Prediction of Students Academic Performance Using Extreme Gradient Boosting and Random Forest Algorithms,†vol. 13, no. 3, pp. 1–19, 2025.

M. R. Borna, H. Saadat, A. T. Hojjati, and E. Akbari, “Analyzing click data with AI: implications for student performance prediction and learning assessment,†Front. Educ., vol. 9, no. December, 2024, doi: 10.3389/feduc.2024.1421479.

O. Ojajuni et al., Predicting Student Academic Performance Using Machine Learning, vol. 12957 LNCS, no. June 2023. Springer International Publishing, 2021. doi: 10.1007/978-3-030-87013-3_36.

K. P. Pengkomputeran, I. Media, and T. Mara, “Student Dropout Prediction Using Random Forest and XGBoost Method,†vol. 9, no. 1, pp. 147–157, 2025.

S. Rajendran, S. Chamundeswari, and A. A. Sinha, “Predicting the academic performance of middle- and high-school students using machine learning algorithms,†Soc. Sci. Humanit. Open, vol. 6, no. 1, 2022, doi: 10.1016/j.ssaho.2022.100357.

A. Asselman, M. Khaldi, and S. Aammou, “Enhancing the prediction of student performance based on the machine learning XGBoost algorithm,†Interact. Learn. Environ., vol. 31, no. 6, pp. 3360–3379, 2023, doi: 10.1080/10494820.2021.1928235.

H. Kaur, T. Kaur, and R. Garg, “A Prediction Model for Student Academic Performance Using Machine Learning,†Inform., vol. 47, no. 1, pp. 97–108, 2023, doi: 10.31449/inf.v47i1.4297.

L. Amatullah, Y. Widiastiwi, and N. Chamidah, “Penerapan Klasifikasi Random Forest Terhadap Data Gangguan Spektrum Autisme (ASD) Pada Anak-Anak Menggunakan Seleksi Fitur Principal Component Analysis,†Semin. Nas. Mhs. Ilmu Komput. dan Apl., vol. 3, no. 2, pp. 356–364, 2022.

Generosa Lukhayu Pritalia, “Analisis Komparatif Algoritme Machine Learning dan Penanganan Imbalanced Data pada Klasifikasi Kualitas Air Layak Minum,†KONSTELASI Konvergensi Teknol. dan Sist. Inf., vol. 2, no. 1, pp. 43–55, 2022, doi: 10.24002/konstelasi.v2i1.5630.

K. N. Azizah, I. Kurniawan, and F. Nhita, “Model Klasifikasi berbasis Ekspresi Gen Non-Small Cell Lung Carcinoma (NSCLC) pada Wanita Bukan Perokok Menggunakan Metode Ensemble,†Log. J. Penelit. Inform., vol. 1, no. 1, p. 1, 2023, doi: 10.25124/logic.v1i1.6389.

Jan Melvin Ayu Soraya Dachi and Pardomuan Sitompul, “Analisis Perbandingan Algoritma XGBoost dan Algoritma Random Forest Ensemble Learning pada Klasifikasi Keputusan Kredit,†J. Ris. Rumpun Mat. Dan Ilmu Pengetah. Alam, vol. 2, no. 2, pp. 87–103, 2023, doi: 10.55606/jurrimipa.v2i2.1470.

Z. Jin, J. Shang, Q. Zhu, C. Ling, W. Xie, and B. Qiang, “RFRSF: Employee Turnover Prediction Based on Random Forests and Survival Analysis,†Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol. 12343 LNCS, pp. 503–515, 2020, doi: 10.1007/978-3-030-62008-0_35.

R. H. Pambudi, B. D. Setiawan, and Indriati, “Penerapan Algoritma C4.5 Untuk Memprediksi Nilai Kelulusan Siswa Sekolah Menengah Berdasarkan Faktor Eksternal,†Pengemb. Teknol. Inf. dan Ilmu Komput., vol. 2, no. 7, pp. 2637–2643, 2020, [Online]. Available: http://j-ptiik.ub.ac.id

A. F. B. Sajiwo, B. Rahmat, and A. Junaidi, “Klasifikasi Indeks Standar Pencemaran Udaran (Ispu) Menggunakan Algoritma Xgboost Dengan Teknik Imbalanced Data (Smote),†J. Inform. dan Tek. Elektro Terap., vol. 12, no. 3, 2024, doi: 10.23960/jitet.v12i3.4699.




DOI: http://dx.doi.org/10.30811/jaise.v5i3.7138

Refbacks

  • Saat ini tidak ada refbacks.


Indexing :

Creative Commons License
Journal of Artificial Intelligence and Software Engineering (JAISE) licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.