LightGBM-, SHAP-, and Correlation-Matrix-Heatmap-Based Approaches for Analyzing Household Energy Data: Towards Electricity Self-Sufficient Houses

Abstract

The rapidly growing global energy demand, environmental concerns, and the urgent need to reduce carbon footprints have made sustainable household energy consumption a critical priority. This study aims to analyze household energy data to predict the electricity self-sufficiency rate of households and extract meaningful insights that can enhance it. For this purpose, we use LightGBM (Light Gradient Boosting Machine)-, SHAP (SHapley Additive exPlanations)-, and correlation-heatmap-based approaches to analyze 12 months of energy and questionnaire survey data collected from over 200 smart houses in Kitakyushu, Japan. First, we use LightGBM to predict the ESSR of households and identify the key features that impact the prediction model. By using LightGBM, we demonstrated that the key features are the housing type, average monthly electricity bill, presence of floor heating system, average monthly gas bill, electricity tariff plan, electrical capacity, number of TVs, cooking equipment used, number of washing and drying machines, and the frequency of viewing home energy management systems (HEMSs). Furthermore, we adopted the LightGBM classifier with ℓ1 regularization to extract the most significant features and established a statistical correlation between these features and the electricity self-sufficiency rate. This LightGBM-based model can also predict the electricity self-sufficiency rate of households that did not participate in the questionnaire survey. The LightGBM-based model offers a global view of feature importance but lacks detailed explanations for individual predictions. For this purpose, we used SHAP analysis to identify the impact-wise order of key features that influence the electricity self-sufficiency rate (ESSR) and evaluated the contribution of each feature to the model’s predictions. A heatmap is also used to analyze the correlation among household variables and the ESSR. To evaluate the performance of the classification model, we used a confusion matrix showing a good F1 score (Weighted Avg) of 0.90. The findings discussed in this article offer valuable insights for energy policymakers to achieve the objective of developing energy-self-sufficient houses.