McMahon - Last Good GFS was ( 2025031912 z ) and Last Should Be ( 2025032000 z )
Note: this was Checked at 2025-03-20 06:06 UTC.
It is possible that the Data was running Late and may now be available.
Note: this is an automated Post, check data to ensure it is correct!
Kindest Regards,
Tony
Not sure why this happens minutes after I shut the laptop down for the night. I’m also not sure what’s gone wrong. The database is up and accessible on the databae servers, but the application servers have lost contact with the database.
Im taking the opportunity to apply updates to all servers and reboot them which will hopefully allow things to connect up after they’re all back online. The database servers are all restarted, but the application servers are much bigger and take longer to restart so I won’t know if this has been successful for a little while.
The reboot hasn’t helped. I have something else I need to do now but I’ll be back to look at this ASAP.
I’m back. I know what’s happening now (database load balancer not connecting to the database). I just need to figure out why now!
1 Like
I think things are on the way back up now. I ended up with two network subnets overlapping. The dodgy one was masking out the real one which meant that nothing on one of the servers could talk to any other servers. This broke lots of things that need inter-server communications to work.
I know how the overlapping subnet arrived - an over-zealous install script hijacked the network addresses without checking. I won’t be testing that software any further!
The broken network broke some other stuff too, i.e. distributed file storage. I’m currently fixing that.
I think things are back to normal. Both GFS and ECMWF downloads are running seemingly without errors. I’m going to have some lunch now and check back afterwards.
1 Like
06z GFS and 00z ECMWF data is now available.
2 Likes