I have just found that WXSimate is failing to download any ECMWF data at all it is getting a server error 500 on each attempt.
Stuart
That’s weird. 500 is usually a PHP code error but I’ve not changed any of the PHP code or the underlying data. I’ll investigate once I’m properly awake.
Thanks Chris. I did try more than once but it failed each time. Just tried again and it’s still happening.
Stuart
Just tried manual download and brings up these errors
These do not show on the auto run only manual download
It looks like the scripts run correctly when run manually (in the foreground) but are failing when run from crontab in the background. That probably means that an environment variable isn’t set correctly or I’ve assumed the script is running in a directory that it’s not in when in the background. That’s not easy to debug but I’m working at it now. I only run the DEV scripts on-demand rather than in crontab which is why I’ve missed this.
I’ve got a manual run of 00Z running in another terminal session so that will be available as soon as it’s finished.
I’ve not found the reason why the background jobs to process ECMWF data are failing. It’s not a quick/easy rollback to the previous version so we have to stick with the ‘new’ version for now.
Unfortunately I have to go out in about 15 minutes and won’t be able to get back on my laptop until this evening. So the 00Z data will be available once the manual run finishes but the 06Z run will almost certainly fail this afternoon. I’ve set the background job to log output so I’ll hopefully be able to see the error when I return. I’ll be able to run the 06Z job manually but the 12Z will be due soon after I return home so I might put my efforts into trying to fix the problem so that the 12Z run works OK.
Final comment: The first manual run has failed, but I understand why (the background job kicked off whilst the manual one was running). I’ve fixed the cause of that now, so the foreground job will run without interference from the background job, but it means that the 00Z data won’t be available for another 25 minutes. Sorry 
I have now been able to download both sets of ECMWF data OK so that manual run seems to have completed OK
Stuart
I found 15 minutes to get back on the laptop and I now know which part of the script is failing. I’m stumped why it’s failing at the moment though. I’ve paused the ECMWF scripts for now which means you won’t see broken data, just the 00Z data until I can figure out what I’ve got wrong.
I think I’ve fixed it. 06Z running now but I can’t stay until it’s finished so I won’t know if it’s worked until I get home again.
Looks to me like it worked…
Stuart
My 3pm run seems to have got the 06z ok
The issue was the Linux path.
The main scripts are written in Perl, but there are some things that would either be complicated or slow to do in Perl, so the scripts run some external shell scripts and Python code.
There are a few different ways of running external code and I’d changed the way that one particular shell script is called to a more secure method. The old way carries the path forward from the calling script but the new way doesn’t so a very important executable couldn’t be found which made all the unpacked data files empty. Ooops!
I have a check for an empty file but that didn’t stop the script for some reason, although I did see a warning message saying there was a problem. So that’s something I need to fix.
Anyway, I think we’re back to a stable position and the 12Z run in a couple of hours should be OK.
Thank Chris much appreciated.
Stuart
1 Like