Evaluation of Machine Learning Models for Customer Churn Prediction Using LIME-Based Explainable AI

Abstract

Customer attrition forecasting has become a critical challenge in highly competitive industries such as telecommunications, where retaining existing customers is more cost-effective than acquiring new ones. Although machine learning techniques have been widely applied to identify customers at risk of churn, many models operate as black boxes, limiting their interpretability and usability. To address this issue, this study proposes an integrated framework that combines predictive modeling with Explainable Artificial Intelligence (XAI) using the Local Interpretable Model-Agnostic Explanations (LIME) technique. Unlike conventional approaches that treat explainability as a post-hoc analysis, the proposed framework embeds LIME directly into the modeling pipeline to ensure both accurate and interpretable predictions. The method consists of several stages, including data preprocessing, feature selection, model training, performance evaluation, and model interpretation. Experiments were conducted using the Telco Customer Churn dataset obtained from Kaggle. Three classification algorithms, namely Logistic Regression, Decision Tree, and Random Forest, were evaluated using accuracy, precision, and recall metrics. The results show that Logistic Regression achieved the highest accuracy of 0.8211, followed by Random Forest with 0.7928 and Decision Tree with 0.7289. Furthermore, LIME-based analysis identifies contract type, internet service, monthly charges, tenure, and additional services such as online security and technical support as key factors influencing churn. These results demonstrate that integrating machine learning with XAI enhances model transparency and provides actionable insights for more effective customer retention strategies.