I’m investigating a database problem and that’s probably going to mean that the GFS and ECMWF data will be unavailable until I resolve the issue.
1 Like
For some reason the database user table has deleted itself. That means the root and system user accounts have no access to any data. I think I’m making progress, but it’s been slow because the method has changed in the version of the database I’m using so there are lots of online references to the ‘old’ process and very few about the new process.
As IT always say “Have you turned it off and on” 
I worked in IT for nearly 40 years so turn it off/on again is well ingrained in my fingers.
The first two nodes are back up and running. The third node is currently rebuilding.
Once that’s finished I can go back to try to resolve the original problem. It’s only visible to two people so you won’t notice it.
Same as me. Worked it It for many years like you Chris… 
It appears that the third server database rebuild inadvertently fixed the original problem that was being experienced. I don’t understand why but I’m not going to let that worry me at the moment!
1 Like
Hi Chris,
Seems not to be working at the moment. Thanks in advance for anything you can do.
Tom
Hi Tom
I’m working on this - see McMahon ECMWF 2024092206 z Missing - Last was 2024092200 z - #3 by administrator
My brain isn’t at full speed…I have a bad cold…so I haven’t been paying much attention to the servers for a few days.
Hi Chris,
Actually, I just tried it again, and it seems to be back on! Did you just change something? Sorry about your cold! And thanks!
Tom
The restart of the database on the third node has helped things do what should happen. The database cluster software now knows that one node isn’t well so it’s running off the two working nodes. I just checked GFS and it’s downloading the latest data so that should become visible in the nex t20 minutes or so. ECMWF is also running and I think the latest data might come available at the same time. I can’t be 100% sure how quick they’ll be because they don’t usually run in parallel and I expect they will each slow the other one down a little.
12z GFS data is now available
12z ECMWF data is now also available