rrd graphs with wdconsole

Ask a Question
,
1

Using rrdtool to collect and graph WDCOnsole data.

I wrote this up for a different site, but thought I’d add it here. Here’s hoping this time, as I had some trouble the last time I tried to upload this.

I used to have an old weather station which I used with the excellent open2300 scripts from Kenneth Lavrsen in Denmark to store data and graph it. When I left my previous company, my colleagues were so relieved they gave me a TFA weather station, which is a rebadged IROX USB system. This doesn’t work with open2300, and the software it comes with is Windows which I don’t run. Anyway, I didn’t want a snazzy app, I wanted to monitor the weather. Thanks to Brian, WDConsole supports this weather station pretty well. This worked with some AJAX scripts someone elsewhere on the web wrote, and I also found a utility that could read and graph mysql data which WDConsole can write to. But nothing with the simplicty of open2300.

Then I stumbled across a website talking about using rrd, the round robin database. What an amazing bit of work. WDConsole spits out a file called clientraw.txt every minute. NameBright - Domain Expired has a PHP Parser for this file so that you could easily read the field names - over 160 of them, though I was only interested in 12 or so.

So I created an rrd database using this script:-

rrdtool create TNCWeather.rrd
DS:tempin:GAUGE:600:-40:100
DS:tempout:GAUGE:600:-40:100
DS:windchill:GAUGE:600:-60:100
DS:humout:GAUGE:600:0:100
DS:humin:GAUGE:600:0:100
DS:windspeed:GAUGE:600:0:150
DS:gustspeed:GAUGE:600:0:150
DS:winddir:GAUGE:600:0:359
DS:baro:GAUGE:600:0:1200
DS:drain:GAUGE:600:0:200
DS:cloudheight:GAUGE:600:0:25000
DS:rrate:GAUGE:600:0:200
RRA:AVERAGE:0.5:1:288
RRA:AVERAGE:0.5:3:672
RRA:AVERAGE:0.5:12:2916
RRA:AVERAGE:0.5:288:365

I then found an awk incantation which would read the relevant bits out of
the clientraw.txt file straight into the rrd. like this and which could be put into a script:-

#!/bin/bash
###Populate database with latest data
rrdtool update $DBLOC cat $LOC/clientraw.txt | \ awk 'BEGIN {FS=" "} {print "N:" $13 ":" $5 \ ":" $45 ":" $6 ":" $14 ":" $2 ":" $3 ":" $4 \ ":" $7 ":" $8 ":" $74 ":" $11}'

This fires off every 5 minutes via a cron entry. A script I called weather-to-rrd.sh calls the graphing script with appropriate parameters. If the forum software allows, I’ll add a zip file containing all of this.

Then, a script generates the graphs. I managed to use parameters for this so that one script generates hourly, daily, weekly etc graphs. It’s longish, so I’ll add it as an attachment.

Finally, a horrible bit of html displays the various graphs at 3-hourly, daily, weekly and yearly intervals, with varying amounts of averaging of the data, as shown by the rrd creation script.

I use this data internally and via a password protected area on a website, because my bandwidth is limited.

Hope this helps someone. I know the scripts are horrible, but I’m not a programmer. Please feel free to improve them, and hopefully post up your improvements.

Stevan

wdgraphing.zip (6.01 KB)

2

you could though also look at WDL
which uses the clientraw files to graph the data (i.e uses the clientrawhour.txt, clientrawextra.txt, and clientrawdaily.txt)

3

Yes, you’re right, and that would be ideal. but I run this on a relatively low powered, headless server, and really just want the graphs rather than the additional details. WDL is just too good for me, Brian! But WDConsole fits this requirement perfectly, and in my view also complements the unix way of using building blocks to build up what you’re after.

S

4

Thanks for sharing, running on NSLU2 :smiley:

rrdtool.jpg
rrdtool.jpg703×320 87.9 KB

5

will be releasing a new consolewd version this week
with new custom tags and a bit more control over timing of ftp, etc
and some fixes and improvements (thanks to Yves!)

6

Great! Sorry about the poor quality of my scripts.

S

7

:?:

good quality, and easy to install :smiley:

8

My weather station has been out of commission since mid-December 2008 (I don’t recommend Oregon Scientific by the way!) so my weather website is stale. You can get a feel for some of the RRD graphs I have - see the monthly values toward the end, the others are much too stale: http://weather.hillsandlakes.com/wxgraphs.php

Once the latest console gets released, I’ll upgrade as well. At some point, I’ll look at making what I’ve written available to anyone who wants it.

Cheers

9

These look great, I use RRD graphs to monitor my router using routers2.cgi as a front end http://vk2him.theshackbythebeach.com/index.cgi

I’d be interested in getting your rrd graphs running on my site
Cheers

10

Thank you for the kind words. I want to add a rose plot and a bit snazzier RRD’ing. Unfortunately, it may take me several months. Once it’s packaged, I’ll post a link on this forum.

Cheers!

11

Finally! after four months, I have my weather station back from repair (Oregon Scientific doesn’t get a gold star for support). My RRD graphs are being updated with data. I might re-think how I move the data from my laptop to my desktop (hosting the apache server) … but that’s later. Here’s the link if you’d like to see the graphs: http://weather.hillsandlakes.com/

12

the graphs look good Pablo, have you tried installing the latest version of rrdtool as the new graphs have a mouse-over feature that displays the values under the cursor? You can see that in operation on my site via teh link above?

Ian

13

Hi Ian,

Thank you for the tip (and the kind words) … I just downloaded 1.3.7 and I’m running `configure’ now … let’s see if it compiles fine… it should (I believe) since I compiled 1.3.4

I’ll post an update if/when I get it up 'n running.

14

Hi Ian,

I tried to use the latest RRD however there seems to be an issue with openSUSE 11.0 as no date is UPDATE’d. I strace’d and found MMAP is used to write to the file(s). I recompiled with MMAP disabled and still, it’s not working. For now though, I want to implement http://haroon.sis.utoronto.ca/perl/rrd.cgi/localhost_stats/privoxy.html#Historical If I get ahead on my work, I may be able to do it this week. I still want to get the rose plot working too. Once these two items are done, I’ll work on packaging and releasing. Phew! :slight_smile:

15

Howdy,

Major re-work to simplify my setup with the grand plan of releasing the bits to the general public. I plan on providing a HOWTO to get folks going.

Where I used to have a laptop connected to my WMR-100 and pushing the bits to my desktop machine, and Apaching running on it, I now have Apache running on the klunker laptop too. I’ve been able to simplify some of the scripts and all in all, the laptop barely sweats. The graphs are still being generated whether there’s demand or not. The plan is to move that over to a .cgi.

I still plan on implementing a rose plot for wind and dynamic inquiry via `rrd.cgi’ (as mentioned previously).

Once I do the above, I’ll package.

btw, I added a black line above the wind plots. Cosmetic but I wanted it to look tidy.

Also, my code will be open. You’ll be able to do what you like with it.

Cheers!