The VLab Framework: An Orchestrator Component to Support Data to Knowledge Transition

Abstract

Over the last decades, to better proceed towards global and local policy goals, there was an increasing demand for the scientific community to support decision-makers with the best available knowledge. Scientific modeling is key to enable the transition from data to knowledge, often requiring to process big datasets through complex physical or empirical (learning-based AI) models. Although cloud technologies provide valuable solutions for addressing several of the Big Earth Data challenges, model sharing is still a complex task. The usual approach of sharing models as services requires maintaining a scalable infrastructure which is often a very high barrier for potential model providers. This paper describes the Virtual Earth Laboratory (VLab), a software framework orchestrating data and model access to implement scientific processes for knowledge generation. The VLab lowers the entry barriers for both developers and users. It adopts mature containerization technologies to access models as source code and to rebuild the required software environment to run them on any supported cloud. This makes VLab fitting in the multi-cloud landscape, which is going to characterize the Big Earth Data analytics domain in the next years. The VLab functionalities are accessible through APIs, enabling developers to create new applications tailored to end-users.