Affiliation:
1. AI Graduate School, Gwangju Institute of Science and Technology, Gwangju 61005, Republic of Korea
2. Marine Security and Safety Research Center, Korea Institute of Ocean Science & Technology, Busan 49111, Republic of Korea
Abstract
The amount of data in the maritime domain is rapidly increasing due to the increase in devices that can collect marine information, such as sensors, buoys, ships, and satellites. Maritime data is growing at an unprecedented rate, with terabytes of marine data being collected every month and petabytes of data already being made public. Heterogeneous marine data collected through various devices can be used in various fields such as environmental protection, defect prediction, transportation route optimization, and energy efficiency. However, it is difficult to manage vessel related data due to high heterogeneity of such marine big data. Additionally, due to the high heterogeneity of these data sources and some of the challenges associated with big data, such applications are still underdeveloped and fragmented. In this paper, we propose the Vessel Data Lakehouse architecture consisting of the Vessel Data Lake layer that can handle marine big data, the Vessel Data Warehouse layer that supports marine big data processing and AI, and the Vessel Application Services layer that supports marine application services. Our proposed a Vessel Data Lakehouse that can efficiently manage heterogeneous vessel related data. It can be integrated and managed at low cost by structuring various types of heterogeneous data using an open source-based big data framework. In addition, various types of vessel big data stored in the Data Lakehouse can be directly utilized in various types of vessel analysis services. In this paper, we present an actual use case of a vessel analysis service in a Vessel Data Lakehouse by using AIS data in Busan area.
Funder
Institute of Information & Communications Technology Planning & Evaluation
Korea Coast Guard
Subject
Electrical and Electronic Engineering,Computer Networks and Communications,Hardware and Architecture,Signal Processing,Control and Systems Engineering
Reference37 articles.
1. (2023, January 11). Data Lakehouse. Available online: https://databricks.com/glossary/data-lakehouse.
2. Orescanin, D., and Hlupic, T. (2021, January 27). Data Lakehouse—A Novel Step in Analytics Architecture. Proceedings of the 44th International Convention on Information, Communication and Electronic Technology, Opatija, Croatia.
3. (2023, January 11). Vessel Monitoring System. Available online: https://en.wikipedia.org/wiki/Vessel_monitoring_system.
4. Lytra, I., Vidal, M.E., Orlandi, F., and Attard, J. (2017, January 27). A Big Data Architecture for Managing Oceans of Data and Maritime Applications. Proceedings of the International Conference on Engineering, Technology and Innovation, Madeira, Portugal.
5. Lin, B. (2020, January 23). Overview of High Performance Computing Power Building for the Big Data of Marine Forecasting. Proceedings of the 2020 International Conference on Big Data and Informatization Education (ICBDIE), Zhangjiajie, China.
Cited by
2 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献