This weekend, I want to take some time and try some server side charting using a package I’ve found. To do this, I will need access to logfiles and a description of their format etc…
My main goal is the generate custom graphs that match my website look and feel rather than use the charts that come out of WD which are very different from chart to chart.
Seems there is a number of different ways this could be done.
In the logfiles directory, there are a number of logfiles which contain data that could be used. Having those sent to the remote server would enable me to snarf the info from the datafile and generate the charts that I want.
Is anyone doing this type of thing?
Also, MySQL supported by the system might be a better option, but I’m not really sure about how well that is working. It appears that this will start another transfer process which will now mean I have three, regular FTP, FTP of the client*.txt files and now a MySQL feed.
I’m curious who else has been doing stuff like this…
I take the data from the xml file and put it into a SQL database every minute. Every day the data is split into tables of data which I want to keep on a minute by minute basis, data of daily highs and lows, and monthly highs and lows. I use this for my website and I’ve recently been playing about with a free charting component for .net so I can do just what you describe. It’s looking very promising but I need to spend some more time with it - some hope!
there is people have charts from the mysql working
but you can get wd to create a custom log file, too , (see under setup, logfile …)and have it appended,…but i guess an option to have it only so many records would be good
What…only three updates? I’ve just done a quick count up (which has probably missed some things) and my WD machine is downloading data from at least 6 sources and doing FTP uploads using four different programs, plus two MySQL updates (to different databases). As I get time I’ll be increasing the number of incoming feeds, although I’m not envisaging added any more uploads at this stage.
MySQL uploads seem to be working fine. I’ve got 780,000 records from WD stored in one table now (with updates every minute) and another table is being populated with some different data every 5 minutes.
I must get round to adding some graphs using the data soon though!
I was only counting the feeds to the same server. I do have APRS and WUnderground etc… too.
Good to hear about the MySQL uploads though. I might give that a shot.
What happens if you lose access on one of the MySQL uploads. Does that data end up lost? While not often, COX cable has really screwed up the connection from time to time making any updates next to impossible. With the normal data that WD sends, its not that big of a deal since it will still get there, but I’m not sure about the MySQL data.
I’d forgotten APRS, Wunderground as well (and I’m sure some more too).
With MySQL, if you lose the link then WD doesn’t catch up on the missing data, but you can do it manually. I’ve been uploading the complete previous month’s data from WD just to make sure any gaps are filled. However, I do get some gaps when my WD PC eventually keels over and dies and as I don’t have a datalogger these are lost for all time.
That is a good idea the catching up part. That would be easy since I’m sure duplicate records would not be accepted. Setting up the system to do an update once a week to catch missing pieces would most likley do the job.
Is there a current updated description of the clientraw*.txt files.
There is a description file in the WD directory but it does not mention the clientrawdaily.txt file. I’m not sure that the other two descriptions are up to date.
Where are you getting the creation of the XML file from?
It's from within WD - you can create a file called wdfulldata.xml based upon the wdfulldata.html template. Make sure you go to Setup - Control Panel - FTP / Internet Setup - Custom web page - and tick 'upload a XML data page'.
