McMahon GFS 2023121218 z Missing - Last was 2023121212 z

McMahon - Last Good GFS was ( 2023121212 z ) and Last Should Be ( 2023121218 z )

Note: this was Checked at 2023-12-13 00:06 UTC.

It is possible that the Data was running Late and may now be available.

Note: this is an automated Post, check data to ensure it is correct!

Kindest Regards,

Tony

I am just stating these as Resolved for now. As the issue Resolves if that makes sense.

I think things are back to normal. The 00z GFS run seems to have finished in line with the average finish time. I’ll take a better look when I get on my laptop later on .

1 Like

Having done sleep processing of my memories I know what’s happened.

A few weeks ago I applied some server O/S and utility updates, as I do on a regular basis. These updates had a kernel update included which needs a reboot, which I did at a quiet time (nobody logged in). The reboot didn’t work properly though. Usually the server comes back online after about 5 minutes. This time it wasn’t back to after 30 minutes so I assumed the server had locked up for some reason.

The server is in Finland and I have no keyboard/screen access to see what it’s doing so my only option was to power cycle it in the hope that would resolve the problem. After the power cycle the server came back up as expected and after a few checks it appeared to be running normally.

I strongly suspect that the power cycle interrupted a disk write which left the RAID array in a “check needed” state. The array check is run once a month, on the 12th starting at 01:16, and that dear reader is when things went horribly slow yesterday.