WD is corrupting data from WMR200 data logger

Archive Archived Weather Display Problems
1

WD is corrupting data from WMR200 data logger
I was unfortunately hit by lightning in my office and my computer was down for 8 days.
I’ve had my computer fixed and fortunately I could recover my disk.
Date and time of lightning strike : Fri, 12 Feb 2010 @ 17h24 Date and time of WMR200 data recovery: Sat, 20 Feb 2010 @ 16h00
I had to run WD 8 times in total to get all the data extracted from the WMR200 data logger.
It took me about 3 hours for the data recovery from the WMR200 data logger.

At each step I made a backup of the following:

  1. wmr200history.txt & programerrorlog.txt etc.
  2. ./datafiles folder
  3. ./logfiles folder

If I have a look at the wmr200history.txt files I can see that the WMR200 did send all the data for the 8 days correctly.
It looks like WD history extraction is not joining the data correctly, although correctly send by the WMR200.
Specifically for the data extraction on the 15th Feb at 05:16:00 things started going horribly wrong (see attached history file in folder [2010-02-20 @ 16h40]).

I have attached following files for all 8 runs:

  1. wmr200history.txt
  2. programerrorlog.txt

Attached for the last run is:

  1. logfiles/22010extralog.csv
  2. logfiles/22010lg.txt

Zip file here: [color=blue][b]http://www.bashewa.com/data/2010-02-20.zip[/b][/color]
If other files are needed please let me know.

I’m also missing the climate data for the whole month of February (see http://www.bashewa.com/wxclimatedata.php).

Brian can you have a look at this problem and possibly recover the data for me please?

Here is a screenshot of my current WD screen.
There’s a notice with an error overlaid on the screen and the icon is also missing.

If you are a WMR200 owner then please add your voice to this thread … the more voices the better.

Edit: I have posted possible solutions over in this post: [color=blue][b]Proposal how to fix this problem[/b][/color]

WD - Problems2.gif
WD - Problems2.gif818×602 112 KB

2

Here is a screen shot of the spiky graph data just after the data logger extraction / recovery.

WD - Problems.gif
WD - Problems.gif818×602 110 KB

3

Hi Brian,

I’ve had some time to analyze the 8 history files and can see that the WMR200 is sending all the data minute-for-minute to the history extractor for all 8 days logged.
My WMR200 is set up for 1 minute logging.

Here are all the log files after the 8th run: [color=blue][b]http://www.bashewa.com/data/2010-02-20-logfiles.zip[/b][/color] 653kb

I have also analyzed the above log files and have come across duplicate records, out of sequence records and missing records.

I have noticed the following which is causing potential problems for the WMR200 History Extractor:

1) Packets don’t always seem to arrive in date/time sequence.
This happened during the 2nd to 8th run. There were about 5-10 minutes where I was backing up the
previous run while WD was inactive. During this time the WMR200 was logging those off-line minutes.
This causes WD to treat the older data arriving at a later stage as rubbish and then discards them.
This in turn leads to missing data in the log files.

2) Current packets i.e. D3, D4, D5, D6, D7 & D9 might arrive in between the history packets.
This causes problems in lost data after the history extraction. The 8 data extraction runs
for the 8 days took me about 3 hours to extract. These 3 hours have been lost in the log files
due to the History Extractor dumping them.

3) History data (D2) might be split across 2 packets.
The WMR200 History Extractor does not seem to be able to join these properly which results in
invalid dates, spiky data and also lost data.
See line 1187 in the first history log ./2010-02-20 @ 16h00/wmr200history.txt
`
DATA RECEIVED BY WD:


**Current actual data D7 10 04 10 14 02 0A 00 13 01 27 82 00 51 29 02
No more missed data count 1
**Current actual data D2 31 1F 12 0C 02 0A 1F 00 00 00 6C
D2 31 1F 12 0C 02 0A 1F 00 00 00 6C
**joined lines together D2 31 1F 12 0C 02 0A 1F 00 00 00 6C  ** 00 03 19 00 0C 01 01 07 08 0C 00 00 00 00 20 FF 4D 13 FE 33 01 90 1A 01 28 8C 00 52 11 E5 00 38 8C 00 00 38 08 D2 31 20 12 0C 02 0A 1F 00 00 00 6C 00 03 19 00 0C 01 01 07 08 0C 00 00 00 00 20 FF 4D 13 FE 33 01 90 1A 01 28 8C 00 52 11 E4 00 38 82 00 00 2E 08
**Current actual data  D2 31 1F 12 0C 02 0A 1F 00 00 00 6C 00 03 19 00 0C 01 01 07 08 0C 00 00 00 00 20 FF 4D 13 FE 33 01 90 1A 01 28 8C 00 52 11 E5 00 38 8C 00 00 38 08 D2 31 20 12 0C 02 0A 1F 00 00 00 6C 00 03 19 00 0C 01 01 07 08 0C 00 00 00 00 20 FF 4D 13 FE 33 01 90 1A 01 28 8C 00 52 11 E4 00 38 82 00 00 2E 08
D2 31 1F 12 0C 02 0A 1F 00 00 00 6C 00 03 19 00 0C 01 01 07 08 0C 00 00 00 00 20 FF 4D 13 FE 33 01 90 1A 01 28 8C 00 52 11 E5 00 38 8C 00 00 38 08 D2 31 20 12 0C 02 0A 1F 00 00 00 6C 00 03 19 00 0C 01 01 07 08 0C 00 00 00 00 20 FF 4D 13 FE 33 01 90 1A 01 28 8C 00 52 11 E4 00 38 82 00 00 2E 08

As you can see there is also a current indoor temperature packet (D7) arriving together with the history data (D2).

DECODED DATA BY WD:


Decoding missed data now
Time from decode 01/12/2001 00:25:00
baro check 1474.0
indoor temp check 204.8
indoor hum check 8
found extra D2 18 32 12 2 10  ** 20 12 0C 02 0A
checking to use extra found  D2
time/date in extra found D2 02/12/2010 18:32:00
passed to use extra D2
Time from decode 02/12/2010 18:32:00
baro check 1022.0
indoor temp check 28.2
indoor hum check 40
missed data time: 18:32 date: 12/2/10
Missed data Acumulate Rain= 0.77
time from data 06:32:00 PM looking from 11:55:00 PM current time 03:53:12 PM
Difference in rain =0.0
number of sensor = 1
Current Channel # =1
missed outdoor temp =22.8
missed outdoor hum =56
*****updating data , skip time 06:31:00 PM 12/02/10 amount skipped 2

As you can see there is a Baro reading of 1474.0 hPa, indoor temperature of [color=red][b]204.8

4

Looks like I’m having the same sort of problem, the WMR200 data is not being extracted / interpreted correctly.

I often have only 5 days or less of graphing data showing then it can jump to 23 days graphing and remain there. I have never managed to get a full 31 days graphing data showing in WDL or WD correctly.

So Brian, another plea, please would you have a loo at this problem.

Thanking you… Colin

5

Ok, I’ve just backtracked into the past a little bit.
This problem was reported way back on the 22nd Dec 2007 and is seems it is still with us:
[color=blue][b]http://discourse.weather-watch.com/p/215009[/b][/color]

6

Hi!

I have also overnight boarded by the data.
Climatologie The database is good. (http://idojaras.mobilbankar.net/wxclima.php)
But the OFC and weekly graphs do not.
I do not know what exactly causes the error.
Climatologie not know the place to set your database?
Lg.txt tenderloin, in principle, could be back

7

Unfortunately I’ve had another power failure here today. I do have a UPS but it only lasts for about 5 hours. Our power is sometimes off for much longer than that. Today it’s been off for 13 hours after they stole the underground copper cable that feeds my premises, again last night (3rd time in 4 weeks).

I’m absolutely dependent on the WMR200 data logger keeping my history … and unfortunately I had the same problems again as described above.
History, data and log files are available if needed.

Edit:
This happened after having installed the latest version of WD (build 21) from the full install.
I thought that I might have had an old version of the Data Gatherer but I see it’s still the old version 4.7 from the 27th August 2008.

8

I am also having problems extracting data, I started again at the start of January with a little success, most of the data was being downloaded correctly with the odd strange data, I suddenly got some 50C data out of nowhere, but the biggest error came last week when I came to download about a weeks worth of data it just vanished.

I have another question, I have the wmr200a so obviously can’t get my time by radio signal, many have said your wmr200 time has to be the same as your pc, but my wmr200 does seem to gain time, when would be the best time to sync it again when it’s out.

9

Climatologie nooa and on the fillets, the WD-logfile-Convert-action brought with location.
The graphs of the WD was messed up but the place was Climatologie.
But it is also so that the tab frisitettem again wiew-awerages/extrem

10

Hi Matt,
My clock is also fast.
It should not be a problem except that you might find a gap, i.e. missing records for the time difference, after a restart.
However after a recovery you now also end up with future data in the logs.
I don’t know how Brian handles that. He probably just overwrites it I guess.

It’s best to synchronize your clock at least once a month.

11

I did some brainstorming :smiley:

The problem with the Console date/time being out of sync with the PC can be solved as follows:

Lets say your console clock is 10 minutes faster than your PC.
Now all the records would normally be logged 10 minutes into the future in the logs.
However during the data recovery there is current data arriving as well with a timestamp on it.
If WD receives a packet like this it can work out the time difference between the PC and the console
and make the necessary time adjustment i.e subtract 10 minutes from the incoming history data.

Problem solved.

12

Are you sure the current data has a time stamp? That’s not true for some other stations.

13

Yes, all the records have a timestamp.
Here’s the header of all the records:`

BYTE DESC VALUE

00: Msg Id [ D2 | D3 | D4 | D5 | D6 | D7 | D8 | D9 ]
01: Length Length of record
02: Minute |
03: Hour |
04: Day | time and date
05: Month |
06: Year |
07: Data data follows from here onwards.
…
nn

`

14

OK, just a thought :slight_smile:

15

I had same problems with my WMR 200.

Most of them were solved with a “downgrade”: http://discourse.weather-watch.com/p/340570

In the meantime I use meteohub on a sheevaplug to store the data and it is absolutely reliable.
In WD the data can be imported from meteohub and there are hardly any errors in the data
(Exception: the rain sum is not resetted at the end of the month).

I capitulated to get any help here regarding the data logger of the WMR 200.
The only answers I get are “It should work” or “try latest update” which are not really helpful.
I didn’t get any response to my post in the link above.

WD works fine when it runs 24/7.
Nobody does acknowledge it, but you must see:

WD doesn’t really support the data logger of the WMR 200.

16

Me too!!! I’ve previously raised these exact issues a year ago and just gave up trying to fix it. The WMR200 and WD work fine 95% of the time (the occiasional spike aside) unless you are extracting the data from the data logger, whe it all goes haywire. Ive just become accepting of the fact that if if have to use the datalogger, then I’m going to get huge holes in the data.

17

Me, too! I turn off my computer at night, and I would really like to have the overnight data loaded when I start WD in the morning.

Lane

18

Bump !!!

Windy are you there?
Have you had some time to look into this problem yet?

I did some further investigation and see that it’s not only a Windows user problem.
I have come across Mac and Linux users with WMR200’s having similar problems.
There’s a post just above this one from a Linux user.

I have come to the conclusion that having a separate data gatherer at startup is not ideal.
If not all the data gets extracted from the WMR200 data-logger at startup there’s bound to be problems.
It seems that when the real-time data gatherer starts receiving history data, that this data is then treated as junk.

Wouldn’t it be better to have the history data managed together with the real-time data?

Last time the history data extraction module got some attention was way back in August 2008.
Pleeeeeeeeease Brian, I beg you, can you please have a look at this?

19

This sounds alot like the problem I am having though not just on startup.

I see the data appearing in the data gatherer from the WMR200 but nothing appears on the console. I asked questions about this a few weeks ago and never really got anywhere. All my extreme conditions seem to be stuck at the daily reset values, -148 for the max temp, 212 for the minimum temp. WD seems to think it is not getting the data but yet I watch it scroll by in a continual stream. I guess it is not just me.

Brian please help…

Doug

20

Exact same issues I had and following no response from emails to Windy I have to say that I am now using VWS which has run perfectly from install iin November. Sorry to post this but unless people vote with their feet their concerns will not be taken seriosly