i’ve had a lot of data logging problems, including ones from history extraction crashing. loses data. adjusted to retrieve from time of crash. still gaps in graphs, converted log files etc. temp wind rainfall graphs display random days. 24 08 26 26 0-1 etc. are there specific log files to convert for correcting specific data?
also i’ve noticed in the log files that after 12.58 am (0 58) it then goes to 11.59pm (23 59) then to 1 00 am:
0 58
23 59
1 00
1 01 etc.
also at 9 am the date shifts one day forward ie tommorows date, then resumes to current date at 9 01am.
i also seem to get random wrong dates through the log data (maybe alot to do with converting files to try and correct other problems.
is there a straight forward way to correct data, especially UV and barometer data?
correcting the log files, then converting them and updating the graph data doesn’t semm to correct the graph information…
If you want help analysing possible reasons for failure with logging and such like it is really useful to attach a copy of wmr200history.txt (it is in the wdisplay folder, and only relates to the most recent start up of WD).
You’ll probably need to zip it up as it can be quite large - or the most important bits are usually the first 100 lines or so, the again from just before “Station Ready” (starting 10-20 lines before the nearest “No more missed data count 1” above “Station Ready”).
Do you use the “Re create the last 31 days graph data”?
i exited the program edited the incorrect data (eg, 32009log.txt). re opened the program, converted the log file to data file. then recreated 31 day graph history. i could be wrong (i’m getting i bit mixed up now if things were as i remember them), but the current graph data was missing, like it scrubbed everything and started fresh. looking at it today, it’s all there except about an hour of zero data (not missing, just lines drop off and pick up again). although UV data has not come back.
i’m guessing current graph data is a seperate inf file to graph history and that’s why it returned later, but not when i recreated it? also am i right in thinking the log files are a means for us to view edit and convert to correct data files, but the data files are what counts as far as correct information being displayed? i’ve tried to get rid of ridiculous UV solar readings and can’t. i’m going to wait for next month to start (now i’ve sorted most problems) to see if it settles then. if not i’ll attach a copy to get some more help. i was just a bit concerned also with 12.59am and 9am things as well, thought it might be throwing everything out of whack. and thanks for the reply.
As usual (every day), after my WMR200 data was extracted, I saw on the graph that the temp., RH, And DP lines had dipped sharply in the minute after the history extraction was complete, so I corrected the 32009lg.txt file, converted to graph data in WD, and re-created the last 31 days of data.
Every time I have done this prior to today, the graph lines smooth out and all is fine.
Today however, all the graph data since around 2300 yesterday was erased. I frantically searched the 32009lg.txt file but all was fine here, so I looked at the graph file (.inf) to find that the data was indeed missing. I tried repeating the convert log to graph and re-create procedure, only to find that once again this deleted all recent graph data since ~2300 (thankfully only about 10 mins this time since most of it was already gone forever).
The attached screenshot should aid this explanation (note time change from 22 to 217 to 18 and double grey vertical line on wind graphs where the data discontinuity occurs).
Any ideas on what is going wrong would be much appreciated.
Timmead
It is getting near the end of the month - check the log file to make sure you do not have more than 44640 lines of data. Anything more than this will not be processed by WD. If this is the case you will find the log file has a lot of duplicate lines that will need to be removed. I use the sort and filter capabilities of excel for this purpose. But the WD logfile bug finder/checker program that can be found through the forum may also be able to do this?
If this was your issue, once the excess lines are removed convert the file as normal and all should be fine.
The duplicate lines seem to occur if you download from the WMR200 console memory a lot rather then record live data and if there console time, date and timezones are not kept exactly the same.
Well I checked and I think you’ve identified the issue, so thank you.
32009lg.txt has 46055 lines as of 2142 today,
The graph file is on 44503.
So by the end of tomorrow there’ll be around 3000 lines too many in the lg.txt file.
You say you use excel to remove duplicates, but doesn’t WD require .txt files for conversions?
Also, as it could be a very lengthly process removing randomly spaced duplicates, how does WD cope with huge gaps in the data if I decide to simply remove a couple of hours worth of data from each day?
i am in AUS, reset time of nine am. i have under reset times 0,0,0, (1 for yearly) check boxes AV/Ext reset time of midnight and use AUS day of 9am option selected. my log files for march shouldn’t have too many lines as i have repeatedly gone through the log files and converted the files, i mainly have missing entries, not duplicates.
do the time entries have to match between different data files for correct data alignment? the date forward at 9am sort of weird to go day forward for one line and then back again, would expect to see a changeover date at 9am and remain sequential. and having a 23 59 after 0 58 and then another 23 hours later (2 x 23 59 entries for on day) makes no sense at all to me.
under graph setup - reset/clear the 12 hour graph. does this lose data or is that retained in the 32009.inf file and restored when it becomes older than 12 hours?
in the temperature/wind/ rain chart, can’t get this one right, currently shows data at bottom as 27 28 0-1 31
the temp for one day did say -30, but now is correct? sunshine hours dates are showing 1 24 08 24 25 26 26. no max solar or max uv readings are displayed. i’m sort of concluding that waiting till the month is over, then correcting it as previous month data is best option but want to make sure i don’t start a new month with same probs all over again.
Normal daily resetting wind/temperature daily value , day: 30
Normal daily resetting rain daily value , day: 30 rain today 0.0 mm
Current coldest night value (oC) 19.9 6:00:01 AM 30/03/09
Resetting warmest day values 6:00:31 AM 30/03/09
Resetting warmest day values 6:01:31 AM 30/03/09
Current warmest day value (oC) 24.2 6:00:01 PM 30/03/09
Resetting coldest night values 6:00:31 PM 30/03/09
Resetting coldest night values 6:01:31 PM 30/03/09
Reset overnight minimum temp 9:00:01 PM 30/03/09
Reset overnight minimum temp 9:01:01 PM 30/03/09
Normal daily resetting wind/temperature daily value , day: 31
Normal daily resetting rain daily value , day: 31 rain today 3.0 mm
Current coldest night value (oC) 19.8 6:00:01 AM 31/03/09
Resetting warmest day values 6:00:31 AM 31/03/09
Resetting warmest day values 6:01:31 AM 31/03/09
ERROR: Failed to get data for ‘alive’ at time/date 10:21:05 AM 31/03/09
ERROR: Failed to get data for ‘alive’ at time/date 10:22:44 AM 31/03/09
ERROR: Failed to get data for ‘alive’ at time/date 10:24:56 AM 31/03/09
ERROR: Failed to get data for ‘alive’ at time/date 10:48:02 AM 31/03/09
ERROR: Failed to get data for ‘alive’ at time/date 10:54:05 AM 31/03/09
ERROR: Failed to get data for ‘alive’ at time/date 10:57:23 AM 31/03/09
ERROR: Failed to get data for ‘alive’ at time/date 11:00:08 AM 31/03/09
ERROR: Failed to get data for ‘alive’ at time/date 11:10:35 AM 31/03/09
ERROR: Failed to get data for ‘alive’ at time/date 11:11:08 AM 31/03/09
ERROR: Failed to get data for ‘alive’ at time/date 11:13:20 AM 31/03/09
ERROR: Failed to get data for ‘alive’ at time/date 11:13:53 AM 31/03/09
ERROR: Failed to get data for ‘alive’ at time/date 11:16:05 AM 31/03/09
ERROR: Failed to get data for ‘alive’ at time/date 11:27:05 AM 31/03/09
Current warmest day value (oC) 19.4 6:00:01 PM 31/03/09
Resetting coldest night values 6:00:31 PM 31/03/09
Resetting coldest night values 6:01:31 PM 31/03/09
You say you use excel to remove duplicates, but doesn't WD require .txt files for conversions?
I import the txt file into excel, make changes and then export it again as a tab delimited txt file. Not staightforward but effective. Make sure you have backed up the file before playing with it.
Also, as it could be a very lengthly process removing randomly spaced duplicates, how does WD cope with huge gaps in the data if I decide to simply remove a couple of hours worth of data from each day?
I use the filter function of excel to automatically find any totally identical lines. This does not get all the lines but so far enough to get below the magic 44640 number. WD generally works fine with data missing. But note that you could have problems if the missing data includes your daily reset times (i.e. around midnight fr most and 9 am for the ANZACS)
Bretto
the date forward at 9am sort of weird to go day forward for one line and then back again, would expect to see a changeover date at 9am and remain sequential.
I think the the change in date for the single 9am log file line is Brian's magic trick to allow us Aussies to have WD record rainfall in accordance with the BOM, which is day earlier than NZ. This feature is very new and so far I have only used this for my back-up station to see how it all goes.
and having a 23 59 after 0 58 and then another 23 hours later (2 x 23 59 entries for on day) makes no sense at all to me
This data is not correct. I have seen this occur with the computer and console times do not match. A deep reset of the console may also be worthwhile to clear any ghost data remaining in the memory
do the time entries have to match between different data files for correct data alignment?
Not sure what your asking.
Hopefully someone else can help you with the rest of your queries and what the errors in the program log file mean.
well i think i’ve undone his magic trick when correcting bad data.
with the alignment of data i meant if, for example, i’m missing 2 hours from the vantagelog file but not the 32009 file the info gets jumbled. if i have missing days, do i need to at least fill those gaps with at least bogus data? sorry i can’t make much sense to anyone else if i can’t make sense to myself. i think i’ve tried to take in too much at once and it’s just mashed up in my head. i’d really like to know where the trend data is getting messed up.
i also set the check box to sync the time last night. waiting to see if that helps. i also read through the other thread about the checker program, but don’t want to introduce myself to more trouble before i’m on top of what i’ve got.
i also set the check box to sync the time last night. waiting to see if that helps.
Happy to be corrected, but if this does anything at all I think this tries to set your computer clock to match the console, not the other way around. Not much use if your computer time is also synced to an internet server. I have to manually change the console time about once a week to keep within less than 1 minute difference.
yeah, i thought i read that somewhere, but my console isn’t a very accurate clock, i noticed i set it and a few days later it’s out again. so now i’m going to have the console drag my pc time out as well. also you are right about the 9am thing. i read it in the suggestions section. sorry i try to read up before posting, but can’t read it all.
i was reading about the bug checker, but trying not to go too fast too quick. i’ll work out where i’m at before i know where i’m going.
I did all this and the excel filter trick worked perfectly, but when I loaded up WD and converted the new log file (tab delimited) to graph data the graphs went haywire. As WD started logging, I could see from the log file that its entries had a different set up from the entries produced by the excel conversion, which is probably the problem [see attached screen shots].
How do you get around this problem?
Timmead
EDIT: Now WD has decided not to do the monthly rain reset on the logs, and is not even converting my new April log to April graph file, I really need to know how to get this to work!
