OVERCOMING BIAS FROM MISSING VALUES IN MICROCREDIT DATA BY COMPARING MICE AND IMPUTATION METHODS

Abstract

Missing values are one of the main problems in financial data processing, especially in microcredit data. The presence of incomplete data can cause bias, disrupt variable distribution, and reduce the performance of classification models in determining creditworthiness. This study aims to compare two imputation approaches, namely Simple Imputation (median for numerical attributes and mode for categorical attributes) and the MICE (Multiple Imputation by Chained Equations) Method, in an effort to reduce bias due to missing values and improve classification prediction performance. The dataset used is Loan Payments Data from Kaggle, which contains 500 rows of data and 11 attributes, namely Loan_ID, loan_status, Principal, terms, effective_date, due_date, paid_off_time, past_due_days, age, education, and Gender. After the data cleaning process, outlier handling, and imputation using both methods, the data was predicted using two classification models, namely Logistic Regression and Random Forest. Model performance was evaluated using the Accuracy and AUC (Area Under the ROC Curve) metrics. The results showed that the MICE method produced higher and more stable performance compared to Simple Imputation. Logistic Regression increased from an accuracy of 66.67% to 82.00%, and AUC from 71.02% to 95.03%. The Random Forest model on Simple Imputation data achieved 100% accuracy and 100% AUC, but these overly perfect values potentially indicate overfitting, a condition where the model memorises specific patterns in the training data and is less able to generalise. On the MICE imputation data, Random Forest still achieved high performance with an accuracy of 98.00% and an AUC of 99.55%, which is considered more realistic and stable. These findings indicate that the MICE method is more effective in reducing bias due to missing values and improving the reliability of microcredit risk classification results.