GFS & ECMWF processing scripts

Software WxSim
, ,
1

I’ve started some work to rationalise the structure of the scripts used to download and process the GFS and ECMWF data that’s downloaded by WxSimate. I began this about a year ago and life got in the way.

The work doesn’t (shouldn’t!) affect the data you download. This is all about putting scripts and data into consistent locations to make it easier to support/manage. As such, it shouldn’t affect your downloads, but I’m putting this note here just in case something goes wrong. I have development and production environments and databases and all work is completed on the development scripts/directories before doing the production work, so that shoudl tease out any issues.

After completing this work things will be in a much better place to make improvements in the scripts and the way they work.

1 Like
2

Thanks Cris.
Greatly Appreciated.
Kindest Regards,
Tony

1 Like
3

Progress report -

  1. The Dev and Prod scripts for ECMWF are now in their new directory structure. GFS was done some time ago and Dev ECMWF testing went well so I also moved the Prod ECMWF. Both Dev and Prod ECMWF have run through two cycles from the new directories without any problems being seen.

  2. I’ve identified a way that could make some ECMWF data runs complete sooner. When a run ends prematurely for some reason (there are various reasons why this can happen) it could sometimes take up to 30 minutes for the next run to start. Dev ECMWF has been modified to reduce the restart delay to either 5 or 10 minutes depending on the failure mode. I suspect that 5 minutes delay is likely to be the most common restart time. After some more testing I’ll apply this to the Prod ECMWF and then decide whether the same technique can be applied to GFS. I don’t know whether this modification will make much difference though because I’ve no easy way to find out how many runs terminate prematurely.