Author:
Lv Taizhi,He Chenyong,Zhang Juan,Song Zhiyang
Abstract
Abstract
There are nearly 50 million AIS data generated every day in the Yangtze River Basin. Efficient storage and query of AIS data become the key problem of smart maritime administration. To overcome this problem, a distributed storage and query scheme based on Hadoop platform is proposed. Firstly, the original AIS message is parsed, the abnormal data is removed and the parsed data is pushed to the Kafka message queue. Secondly, the messages in the queue are pushed to different servers to complete the storage of real-time cache, historical trajectory and regional information. Finally, the query interface is provided to realize the real-time query of ship status, trajectory and regional ship in a certain period of time. The real applications show that compared with the traditional relational database, the solution can meet the requirements of real-time query and has high scalability.
Subject
General Physics and Astronomy
Reference6 articles.
1. The Significance of Incorporating Unidentified Vessels into AIS-based Ship Emission Inventory;Zhang;Atmospheric environment,2019
2. Modelling Ship Density Using a Molecular Dynamics Approach;Liu;Journal of Navigation,2019
3. A Distributed Framework for Trajectory Similarity Query Based on HBase and Redis;Qin;Information,2019
4. MHBase: A Distributed Real-Time Query Scheme for Meteorological Data Based on HBase;Ma;Future Internet,2016
5. A Distributed Message Delivery Infrastructure for Connected Vehicle Technology Applications;Du;IEEE Transactions on Intelligent Transportation Systems
Cited by
5 articles.
订阅此论文施引文献
订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献
1. Research on educational big data analysis based on Hive and Spark;2023 8th International Conference on Information Systems Engineering (ICISE);2023-06-23
2. Research on ship data analysis based on Spark platform;2023 5th International Conference on Communications, Information System and Computer Engineering (CISCE);2023-04-14
3. A method for detecting abnormal behavior of ships based on multi-dimensional density distance and an abnormal isolation mechanism;Mathematical Biosciences and Engineering;2023
4. Data Pipeline Framework for AIS Data Processing;2022 Seventh International Conference on Informatics and Computing (ICIC);2022-12-08
5. Research on ship tracking and control for COVID-19 epidemic prevention based on multi-source information fusion;2022 5th International Conference on Advanced Electronic Materials, Computers and Software Engineering (AEMCSE);2022-04