Adaptive Cross-platform Scheduling Framework for NWP in Hybrid Clouds

Author:

Ding Fan1

Affiliation:

1. Lanzhou University of Technology

Abstract

Abstract

Numerical Weather Prediction (NWP) requires real-time, high-accuracy processing, straining traditional high-performance computing clusters with limited resources, complex operations, and long queue times. Hybrid clouds merge the security of local clusters with the scalability of public clouds, providing a viable solution for high-performance computations. However, it also poses challenges: parallel programming for local clusters is not suitable for the various settings of hybrid clouds; complex parallelization policies increase communication overhead and complicate scheduling; and traditional static resource binding can lead to load imbalance in heterogeneous environments. This paper proposes an adaptive cross-platform scheduling strategy tailored to the characteristics of NWP models. This approach harmonizes the advantages of traditional and cloud-based parallel computing, integrating two distinct parallel programming methodologies and reconfiguring the parallel programming framework of the forecasting models. Experimental results show that the framework effectively improves adaptability and resource utilization, significantly improves computational efficiency and reduces operational overhead in hybrid cloud deployments.

Publisher

Springer Science and Business Media LLC

Reference52 articles.

1. Shapkalijevski. Perspectives toward Stochastic and Learned-by-Data Turbulence in Numerical Weather Prediction [J];Metodija M;Weather Forecast,2024

2. Sung Goon Park. Assessing the Reliability and Optimizing Input Parameters of the NWP-CFD Downscaling Method for Generating Onshore Wind Energy Resource Maps of South Korea [J];Kim J;Energies,2024

3. Wolfgang Blochinger. Equilibrium: an elasticity controller for parallel tree search in the cloud [J];Kehrer S;J Supercomputing,2020

4. On producing reliable and affordable numerical weather forecasts on public cloud-computing infrastructure [J];Tcy Chui D;J Atmos Ocean Technol,2019

5. Coffrin C, Arnold J, Eidenbenz S, Aberle D (2019) J. Woodring. The ISTI Rapid Response on Exploring Cloud Computing 2018 [J]

同舟云学术

1.学者识别学者识别

2.学术分析学术分析

3.人才评估人才评估

"同舟云学术"是以全球学者为主线,采集、加工和组织学术论文而形成的新型学术文献查询和分析系统,可以对全球学者进行文献检索和人才价值评估。用户可以通过关注某些学科领域的顶尖人物而持续追踪该领域的学科进展和研究前沿。经过近期的数据扩容,当前同舟云学术共收录了国内外主流学术期刊6万余种,收集的期刊论文及会议论文总量共计约1.5亿篇,并以每天添加12000余篇中外论文的速度递增。我们也可以为用户提供个性化、定制化的学者数据。欢迎来电咨询!咨询电话:010-8811{复制后删除}0370

www.globalauthorid.com

TOP

Copyright © 2019-2024 北京同舟云网络信息技术有限公司
京公网安备11010802033243号  京ICP备18003416号-3