Since I now run WXSim automatically I usually take a peek at the data downloaded only sometimes to check its still OK. Well the past two days I’ve been checking more as I found I was getting next to no SYNOP data being downloaded, no errors just anything from 1 to 10 lines. So far I’ve only been around first thing in the morning to check so I’m not sure what happens later in the day. I have found that if I change site I can find it all. I left the site selected where I found all the data but this morning again from this site I was only getting a few lines again. Again changing site found all the data.
Now I’m wondering if this is caused by the timing of my downloading the data which is about 07:10 BST (06:10 GMT) which could perhaps be when the data is being updated in the USA. Its interesting that now at 07:40 BST I have just gone back to my first SYNOP site and gotten all the data as usual, so it is starting to look like it might be a timing problem.
The other thing I’m not sure about is how missing SYNOP data might affect my forecasts?
Stuart
Hi Stuart,
That is an interesting observation, as I had come to a similar conclusion myself, about getting the timing of the download optimised. I had moved (delayed) my morning download by an hour (to after 0800 BST) to seemingly get better forecasts. However, I had not identified what was the cause, but think you may be onto something with the SYNOP being possibly abbreviated at certain times…mmmm :?
I am wondering if Tom might know anything about the schedules for loading this data. I currently run 3 forecasts a day, 7am 1pm and 7pm, now I need to see if this is happening at these other times.
Stuart
I think Tom has commented on this before and he said it’s best to wait at least 12 or 14 minutes past the hour before attempting to grab the data. I have been trying to run my forecasts 45 minutes past the hour to ensure that all the stations have reported and the data is logged. I don’t know if it helps but I haven’t had any issues since doing it this way. I run all my forecasts manually but it shouldn’t make a difference from auto mode.
Hi Dan,
From my experience it is not just the minutes past the hour (I use 25 mins past), but the actual hour selected that seems to have a such a marked effect… :? :? However, if you are using 45 mins past the hr satisfactorily, perhaps I’ll have a go with that to see how things go… 
…Perhaps though, Tom with his experience of downloads, will be able to advise us as to when it is best to download/extract the data from the servers concerned…
I just looked at the 13:05 download and compared it with what I got doing it now at 17:50 and at a rough guess there was 40% more SYNOP data than at 13:05. The 13:05 run had probably 3 times as much SYNOP data when compared to the 07:05 run. So yes I think the hour chosen probably matters a lot especially if only doing 3 or 4 forecasts per day like me. I’d like to know when the SYNOP is updated, is it every hour, every 4 hours or what exactly?
Stuart
Hi Everybody,
Being in the U.s., where METAR data is more abundant than SYNOP, I don’t really look at SYNOP files too regularly. However, I have had the impression in the past that the data can be rather slow coming in. I wouldn’t be surprised if much of it had yet to arrive 20 minutes, or even 30 minutes after the hour.
As for the frequency of reports, it’s at best 3 hours (00Z, 03Z, 06Z, etc.), with most stations every 6 hours, and some only every 12. WXSIMATE already selects the last multiple of 3, 6, or 12 hours (your choice), which means you should generally get a pretty populated file if you’re at least 20 minutes past those hours.
Important point!: if you run the programs at, say, 08Z, you will be getting two-hour old SYNOP data. In itself, that’s not a problem, but it becomes a problem if you then import local data from your weather station, which will be about 2 hours out of sync with the SYNOP data, and will likely produce somewhat poor advection profiles and forecast innaccuracies. If you are using SYNOP and home station import together, your run times should be less than an hour after the synop release times. For example, you could run at about 0630Z, and get a good batch of SYNOP for advection, and import local data to get conditions at the “top” of the hour (0600Z). If you are running WXSIM manually and typing in your conditions, you should use those at about the same time as most of the SYNOP or METAR data.
Hope that helps! 
Tom
Thanks Tom, I’ll try running at 30 minutes past 6, 12 and 18 hours and see what happens.
Stuart
Another factor to consider in deciding auto run times is the timing of the GFS data. I haven’t checked too closely, but I think it’s coming out about 6 hours after the initialization time. If that’s the case, then 30 minutes past 00, 06, 12, and 18Z would be good run times because they would just catch the latest GFS data. You might want to check and make sure that’s the case.
You also may want to see which has better coverage, SYNOP or METAR. I think in Europe they’re both fairly abundant, with the only disadvantage for SYNOP being the low frequency (3 or 6 hours) of reporting.
Tom
Well I checked this morning and the SYNOP data was there completely after 06:30. I also remembered that I had WXSimate archiving so I went back to look at the saved data and found that I had been getting less than 50% of the normal amount of data downloaded from the day I changed the timing of the runs to 07:05, they used to about 06:15. I seem to remember changing it because I was missing the latest GFS data with a d/l just after 06,12 and 18 hours.
So from this check I will stay with running WXSimate collection at 30 minutes past 06, 12 and 18 hours. I think this is likely to catch Chris’ GFS data as well, at least it did today. I’ll keep an eye on it for a couple of days. At least now I remember that WXSimate is archiving I can go back and check at any time.
Tom I d/l both SYNOP and METAR, I hope that WXSim will sort out the best of each to use. Is that OK?
Stuart
I have adjusted all my times to a more suitable 43 mins past the hour for WxSim to run the forecast (at 0643z, 1243z and 1843z)…downloads for WxSimate to take place at the corresponding 40 mins past the hr. However, I have noted that the effective time for the local wx stn Wx is always taken as being at 55 mins past the preceeding hr. It will be interesting to see what transpires with the forecast accuracies, particularly with regards the GFS wx element of the inputs. If I remember correctly, this is why I had delayed the forecast by the additional hr, to take into account the processing of GFS onto the server. ie. For the latest GFS to be ready for consequent download.
Tom I just realised that I was talking about GMT+1 timings and you are talking Zulu or GMT! So I am still 1 hour out of sync. I guess I should be running at 07:30 13:30 and 19:30 BST rather than my new times of 06:30 12:30 and 18:30 because my SYNOP is likely old.
As a matter of interest how can I tell what the timings are for the METAR and SYNOP data in the 'data.txt file created by WXSimate?
Stuart
A couple of comments …
(1) With regard to local station data being 55 minutes past the hour, this is deliberate, in order to synchronize well with most METAR or synoptic obs. In the U.S., there’s a tradition of taking readings just (about 4 to 8 minutes) before the hour for METAR. In Canada, it’s right on the hour. In much of Europe, METAR shows up twice per hour, at 10 minutes before and 20 minutes after in many cases. I suppose SYNOP is at the hour, but I suspect this varies. All in all, a few minutes before the hour affords the best synchronization, on average. Differences of up to half an hour may not be too serious, but I really prefer no more than 10 minutes difference for best results. This means you have to be careful about using local data for the hours in between the infrequent synoptic reports.
(2) WXSIM uses METAR, or SYNOP, but not both at the same time. You choose on the Import page which you want to use, and it keeps that setting until you change it. It would probably be possible to combine them, but it does raise issues, including the fact that much of the data would be redundant, and also lack of synchronization on in-between-SYNOP-report hours.
Hope that helps!
Tom
Tom thanks for the update. I now understand a bit more about this. Having checked my settings I can see that having incomplete SYNOP data would not have been a problem as I use METAR data in the Import panel currently. I guess my next question is which would you suggest I should use here for data import METAR or SYNOP. I have run a test to compare a forecast done with METAR against one done with SYNOP and the differences can be significant, for example wind speed and wind direction can be completely different, temps look to be mainly the same and cloud cver is pretty much the same in both.
Stuart
Thanks Tom,
That all makes sense. Nice to have confirmation though. Forecasts ran OK yesterday evening and this am and seemed very sensible considering the very changeable weather at the moment.
Best regards and thanks again for the support,
Hi Stuart,
You may need to take another look at the METAR versus SYNOP issue, at least assuming you are using data from your own weather station. METAR versus SYNOP make no difference in forecast wind speed and direction, as these are pretty much straight from GFS (with slight modifications which have nothing to do with imported surface data). The only difference that could occur would be if you are not using your home station data, but are simply starting out with “home” data from your METAR or SYNOP site. For some jobs, I deliberately give different sites for these, to help cover in case one is not reporting. Also, the latest SYNOP and METAR may be at different times (as discussed above). The important point is that these play no role in the subsequent forecast of wind speed or direction (or cloud cover, for that matter).
What surface datado affect is advection, which affects temperature and dew point in the forecast. Still there should be hardly any difference, as it’s still the same data, just in a different format and perhaps for a somewhat different group of stations. If the wind forecast was different, it had another cause (like a different set of GFS data).
METAR gives you the flexibility of choosing any forecast hour you like, while with SYNOP there are only about four good times per day (shortly after 00, 06, 12, and 18Z). SYNOP has the very slight advantage of temperatures and dew points reported to the tenth of a degree, instead of whole degree (in the U.S., many METAR reports contain tenths to the extent of being able to recover the original whole Fahrenheit degrees).
Hope that helps!
Tom