For almost 5 decades, ECMWF, the European Centre for Medium-range Weather Forecast, has been producing numerical weather forecasts and maintained one of the largest archives of meteorological data. Recently the European Commission has entrusted ECMWF with the implementation of the Copernicus Climate Change Service (C3S), and Atmosphere Monitoring Service (CAMS), opening up the access to a huge amount of environmental data.
ECMWF has a long experience to decode, manipulate and visualise GRIB and NetCDF data. This talk will present how with the help of community open source tools users can more easily explore these large datasets.
But even if users have the right tools and knowledge to manipulate the data, the amount of data to transfer is still a bottleneck. It is why ECMWF embarked in various projects to build and use cloud environments. One of these is the EU-funded Horizon 2020 HiDALGO project, which explores the building of workflows using these tools over various HPC and cloud environments. Open source plays again a vital role to make this happen. First findings of this exciting new work will be presented in this talk.