McMahon - Last Good ECMWF was ( 2024112812 z ) and Last Should Be ( 2024112818 z )
Note: this was Checked at 2024-11-29 02:00 UTC.
It is possible that the Data was running Late and may now be available.
Note: this is an automated Post, check data to ensure it is correct!
Kindest Regards,
Tony
Looking at the data page looks like it has stopped it’s not done the 00z either
I’m investigating this now. The scripts seem to be running but not finding any new data to download for some reason.
It’s possible that there’s a script update needed to access the ECMWF data. I’ve done a test run with the new script which worked as expected. I now have to rebuild the container image and upload it so that I can download it again. Sounds stupid but I know what I mean!
I’m hoping that when it’s ready the new image will allow the downloads to work again.
It wasn’t that. The debug info is pointing to a script that’s been unchanged for months suddenly doing odd things for no obvious reason. I’m currently debugging but don’t know how long this will take to fix.
1 Like
I’ve not given up on this!
It appears that my attempts to update the container image with the latest ECMWF scripts has failed. Although, not exactly failed…I think the image has been rebuilt and has been uploaded to my container registry but my docker environment doesn’t seem to be able to access the new image. Don’t worry if this doesn’t make sense…I think I know what I’m talking about.
1 Like
I think there something more going on, when I just checked my wxsim I had 2 messages that the gfs/ECMWF were 24 hours old and found the gfs was last updated at 21:00 yesterday I normally have download if new data ticked so I appear not have got the files that indicate newer data
I unticked the download if new data and ran wxsimate again and it got the 29 12z gfs so should stop the messages
I wonder if WxSimate checks for both new GFS and ECMWF data. If it finds one is missing it doesn’t do the other?
I’ve made some progress…after many attempts I’ve got the container image update with the latest software. Unfortunately that hasn’t fixed the problem with the ECMWF downloads, but at least it’s given me a up-to-date platform to continue investigations with.
I think I have successfully restarted the ECMWF downloads. We’ll know for sure in about 30 minutes.
What was wrong…well I’m pretty sure I know what was wrong but I haven’t a clue how it’s happened.
The scripts start by doing this…
- Figure out what the last run was
- Run a script to find out what the latest available data is
- Compare (1) and (2) and if newer data is available run another script to download the data.
Script (2) has no parameters…it’s just querying the ECMWF servers to find the latest data. Script (3) has parameters that say ‘download this data’, e.g. 20241129 12.
The error I eventually got to see was that script (2) was complaining that it had didn’t have parameter 2 when it shouldn’t have been looking for any.
Looking at the scripts I found that script (2) was actually a copy of script (3). I can’t understand how that would have happened. The 12z run finished successfully at 20:20 last night. So I assume script (2) was correct at that time. The 18z run failed…at 02:00-ish when I was fast asleep. So how did the script change when I wasn’t using the system? Nobody else has access to the servers and even if someone did, copying one script to another deep in the directory structure seems like a very odd thing to do.
Once I knew what the problem, it was a quick job to grab a copy of the script from my development environment to copy over the rogue copy and restart the download process.
As I was typing the 12z run for 29th Nov has completed so we’re back to normal again.