Internet of Things (IoT) and cloud computing are used in many real-time smart applications such as smart health-care, smart traffic, smart city, and smart industries. Fog computing has been introduced as an intermediate layer to reduce communication delay between cloud and IoT Devices. To improve the performance of these smart applications, a predictive maintenance system needs to adopt an anomaly detection and root cause analysis model that helps to resolve anomalies and avoid such anomalies in the future. The state of art work on data-driven root cause analysis suffers from scalability, accuracy, and interpretability. In this paper, a multi-agent based improved data-driven root cause analysis technique is introduced to identify anomalies and its root cause. The deep learning model LSTM autoencoder is used to find the anomalies, and a game theory approach called SHAP algorithm is used to find the root cause of the anomaly. The evaluation result shows the improvement in accuracy and interpretability, as compared to state-of-the-art works.