McMahon ECMWF 2025013118 z Missing - Last was 2025013112 z

McMahon - Last Good ECMWF was ( 2025013112 z ) and Last Should Be ( 2025013118 z )

Note: this was Checked at 2025-02-01 02:00 UTC.

It is possible that the Data was running Late and may now be available.

Note: this is an automated Post, check data to ensure it is correct!

Kindest Regards,

Tony

Data was available late.

It looks like there were problems downloading the data with it taking 1h10m when it’s normally less than 10 minutes.

I notice the 00z just finished has taken 1hr 20 min to process after download complete in stead of the usual 20 min

It did, but the download took about 45 minutes rather than the usual 6. I suspect that the processing delay was also caused by download problems. The download/processing process goes something like this…

  1. Is it time to start downloading? Yes, goto 2, otherwise wait 5 minutes.
  2. Loop from 1 to ‘x’ (x is variable for ECMWF because alternate runs have 90 hours or 144 hours of data)
  3. Try to download file ‘x’.
  4. If the download fails 3 times abort the run and goto 1
  5. If the file downloads, check that it’s about the right size (they’re compressed so the size varies a little). If it’s the wrong size goto 1.
  6. When all files are downloaded start processing
  7. Unpack all the files and check the unpacked size (they’re now all a standard size). If any file is too small goto 1.
  8. Process the contents

The stats that you see are, for simplicity,

Data Source ready - the time that (2) first starts
Download Complete - the time that (6) is first hit
Processing Complete - the time when (8) finishes

So if there are problems in (4) and (5) that means the Download Complete time is later than the average. If there’s a problem in (7) then even though it looks like the data is downloaded there can still be one (or more) re-download cycles to try to get valid data. There are other reasons why downloads may restart, e.g. the download process crashes, but based on other info I capture elsewhere I don’t believe that’s a common cause.

At some point I intend modifying the stats process to add an ‘attempted downloads’ count for each cycle. Unfortunately the way the stats processing currently works doesn’t make that easy to retro-fit.