Traceroute is showing packet loss somewhere in the US, but there are are only IP addresses listed so it’s not easy to tell exactly where. That would make sense though…it’s probably close to NOAA which is why both data sources are suffering.
The “solution” i came up with was to dump the partly download (get_gfs.pl) i used before and download whole files with wget instead, and then check the size of them and re-download them after couple of minutes if needed. That seems to work at least partly but not allways.
I was earlier in touch with NOAA’s helpdesk and they said whole thing started with a crash of the dataservers and they was forced to migrate to new servers what was/are not fully ready yet. When i asked how it will take before they are fully configured was answer “it will take months”…
// Henkka
From NOAA’s nomads website:
7/17/2013 12Z - Some users are reporting that OPeNDAP downloads sometimes don't complete. We believe this is due to the application server timing out because of being CPU bound. The long term solution is to increase the CPU of these systems, but that is a few months away. Until then, if your downloads are randomly incomplete we recommend pulling data down in smaller chunks.7/25/2013 - NCEP switched to feed from 2 brand new supercomputers. The location and network paths of these supercomputers differs from the previous. We are still fine tuning these paths to ensure that data arrives as expected.
8/7/2013 - NCEP is actively working on an issue that started on 7/31/2013 of data not arriving timely. This problem is causing data to arrive out of order and sometimes specific forecast hours will be delayed.
I was just about to post that! 
I’m going to try to solve this in one or both of two ways:
(1) Get Sam (Bohler) to look into the timing of the downloads and
(2) See if I can get WXSIM to handle missing data
In any case, I think this explains some errors people have been seeing, maybe even before today!
Tom
It’s not just timing of the downloads…I’m seeing partial downloads, i.e. corrupt data files. I’ve also modded my code to try resuming failed downloads. Even then I can’t download a single file…with 10 resuming retries and a timeout of 10 minutes. Even if I could get full files, at that speed it would take 10 hours to download 6 hours worth of data! The link is badly broken 
In addition to the small snippet I receive from WXSIMATE, I also download GFS data from NOAA to generate forecast charts. Yep… data is very late and very very slow to download and some times I just get half a file.
Yes, I was just working with Sam some on this and he’s also suspecting corrupt files, and will be watching the 18Z download. Surely NOAA will get this fixed at some point, but meanwhile I’m trying to figure the best way to manage the problem. Sam is looking at some ideas for checking the files as they (slowly) come in. I’m also wondering if we’d be better off downloading a bit late, to avoid the “rush” on the data from all the users who might be “clamoring” for it.
Meanwhile, I will try to make WXSIM as “tolerant” as possible of data with gaps in it!
Tom
I just checked and found that even NOAA’s own Air Resources Laboratory (source of the READY data most WXSIM users downloaded before Chris created his GFS site) doesn’t have the 12Z data posted! If they’re having trouble getting their own data, I guess that at least means they’ll be working hard to fix it!
Meanwhile, hang in there, and we’ll try to make do with the trickle of data we still seem to have. For the moment, try the Bohler source, and use this very recent update of WXSIM
Tom
May I suggest we stick to Toms thread “Quick fix for GFS data problem” from now on, I see no point in having multiple threads.
Can a mod please lock this thread now…
Stuart
Hi Jean-Robert,
It is a coincidence that a problem with the NOAA servers (where the GFS data originates) got much worse at about the same time I released the new version! See the posts above.
I am pasting in below a sort of “form letter” about this, describing a fix for the problem.
Tom
(This is kind of a mass email, as many people are asking the same question today)
Yes, there is a problem at the very source of the GFS data. The NOAA server supplying this data to everybody (MUCH more than just WXSIM users) crashed on July 5, and they tried to fix it. The service was slow, but kept working for a while. Then, a couple days ago, it got worse, with data often incomplete and files coming in out of order. At that point, Chris McMahon’s system, which will not produce a new file (for WXSIM) with any corrupt data, basically quit working.
There is good news, though!
First, NOAA will surely get this fixed. Second, Chris is working on a work-around. Third, the new, alternative system by Sam Bohler is a bit different, and is producing data files, which under these circumstances generally have anywhere from 2 to 4 corrupt lines (out of 61). I quickly made changes to WXSIM yesterday, which allow it to tolerate individual lines of bad data by detecting them and replacing them with an average of the surrounding lines (two bad lines in a row might still cause a problem, but in the last four runs I have not seen this happen, thankfully).
So, here’s what to do:
(1) Upgrade to the latest official version using the UPGRADE installer (wxsimupg.exe) for version 12.10.1 at www.wxsim.com/dnld.
(2) Get the “emergency” patched version of wxsim.exe at www.wxsim.com/wxsim.exe (download it and use it to replace the wxsim.exe from version 12.10.1). It will say version 12.10.1d.
(3) Try that, and hopefully all will be well! Also, though, please follow the situation on the forum at http://discourse.weather-watch.com/c/29.
Best Regards,
Tom
Thread now closed so please refer to this thread for further details: "Quick fix" for GFS data problem