Author:
Mills Nishan,Issadeen Zafar,Matharaarachchi Amali,Bandaragoda Tharindu,De Silva Daswin,Jennings Andrew,Manic Milos
Abstract
AbstractBig Data is steadily expanding beyond the boundaries of its foundational constructs of three primary Vs, Volume, Velocity and Variety, and two secondary Vs, Veracity and Value. The advent of 5G networks, Edge computing and IoT technologies has transformed Big Data into this modern context. With these new manifestations of Big Data, the focus is not only on the data itself but on the context that it applies to its immediate environment as well as the human and societal perception of this context. It is increasingly challenging for conventional AI algorithms to process and transform this data, analyse and visualise a broad spectrum of insights, and then formulate the explainability of such insights in terms of bias, transparency, safety, ethics, and causality. Self-structuring Artificial Intelligence (SSAI) addresses the limitations of conventional AI by adapting to the inherent structure of the data, incrementally learning and abstracting from this structure. SSAI has not been investigated in a cloud-based setting for generating explainable insights from these new types of Big Data. In this paper we propose a cloud-based architecture for explainable Big Data analytics using SSAI in highly-connected 5G and Edge computing environments. The proposed architecture is empirically evaluated on a commercial scale Big Data use case of Smart Grid for Smart Cities. The results of these experiments confirm the functionality and effectiveness of the proposed architecture.
Publisher
Springer Science and Business Media LLC
Reference39 articles.
1. Hoffmann J, Borgeaud S, Mensch A, Buchatskaya E, Cai T, Rutherford E, Casas DdL, Hendricks LA, Welbl J, Clark A, et al. Training compute-optimal large language models. 2022. arXiv preprint arXiv:2203.15556
2. Biderman S, Schoelkopf H, Anthony QG, Bradley H, O’Brien K, Hallahan E, Khan MA, Purohit S, Prashanth US, Raff E et al. Pythia: a suite for analyzing large language models across training and scaling. In: International conference on machine learning. PMLR. 2023. p. 2397–430.
3. De Silva D, Burstein F, Jelinek HF, Stranieri A, et al. Addressing the complexities of Big Data analytics in healthcare: the diabetes screening case. Aust J Inf Syst. 2015. https://doi.org/10.3127/ajis.v19i0.1183.
4. Nawaratne R, Bandaragoda T, Adikari A et al.: Incremental knowledge acquisition and self-learning for autonomous video surveillance. In: IECON 2017-43rd annual conference of the IEEE industrial electronics society. IEEE. 2017. p. 4790–5.
5. Hashem IAT, Yaqoob I, Anuar NB, Mokhtar S, Gani A, Khan SU. The rise of “Big Data” on cloud computing: review and open research issues. Inf Syst. 2015;47:98–115. https://doi.org/10.1016/j.is.2014.07.006.