Stuart - the phone app is interesting but either I counted seconds wrong last night or the coverage in my area isn’t good. We had a smaller storm at about 23:00 and I counted 12-14 seconds-ish for a few lightning to thunder instances which puts those strokes at 4-5km away. The nearest strikes shown at that time by the app were over 40km away. There’s no evidence of anything nearer than 20km all night. So maybe we do have the wrong kind of lightning in this area…neither my sensor nor Blitzorung can detect them :lol:
Unfortunately my detector wasn’t online last night. Something (I don’t think it was me) messed up my router so I shut things down overnight until I had a chance to investigate. I’m back online again and waiting for the next storm.
I monitor the stats from my router and can usually tell from the errors I see that thunder is near, well nearish as a storm over in France will cause errors. I guess my phone cable from the telegraph pole is a good aerial!
I think my problems were DHCP based rather than losing Internet connectivity. Also FTTC and then underground to the house wall means far less exposed to local lightning issues.
However, the other 2 detectors in the same place are currently at 2192 and 2709 (both connected to Arduino and then RPi).
Another detector 40 km away is currently counting 3800 with current rate 6/min (maximum was 35/min).
We got a thunderstorm nearby.
The first stroke detection was at 17 km and slowly it came closer 12, 7, 6 and 5 km and now it is fading away at 6 km.
150 strokes in the last three hours and that is almost what Blitzortung is showing.
Energy between 3000 and 230000.
Now all i have to do is to write a php script to present the data on my website.
This is where’s it’s difficult to debug. Is my code missing interrupts or is something else going wrong? I think the code is good enough for at least a few hundred ‘events’ per minute because I’ve seen that many disturbers with the sensor in a noisy place, so 35/min should be easy by comparison. I’m looking for ways to speed the code up, maybe using PyPy and also to optimise the interrupt code still further to see if that will help.
Improving performance can be a double-edged sword. I’ve made some significant improvements to the way that the sensor is read but now the Python code can’t handle the number of disturbers it’s seeing. It died somewhere over 500 disturbers per minute which is a good number, but I need to find way to stop it crashing after that point. I might just turn on the ‘mask disturbers’ option when I start to see more than, say, 250 disturbers per minute and just show ‘> 250’ in the stats display.
I’m sorry for making a mess… the detector that read 3800+ strikes the other day is in way better location (flat area) that the other 3 (surrounded by mountains). The 3 detectors are in same place (1 outside, 2 inside including the one connected to a RPi). The outside detector read ~2700, the inside one about 2100 I think. The RPi (the new one) was at ~1400. The relevant comparison is between the “2100” and “1400”.
A reason for the different performance surely comes from the settings.
The “2100” settings:
wdth=2
srej=1
noise =0
gain indoor
The “1400” settings
wdth=3
srej=3
noise=0
gain outdoor
I don’t know the calibration value for the “1400” and the script always calculated different setting. That’s why I set it to 80 pF. If I had the gain set to indoor, it started producing 4 disturbers/sec. When set to outdoors, it’s relatively quiet. Perhaps the sensor itself (the breakout board, Pimoroni UK) is a bit different from my others (Playingwithfusion US).
I didn’t want to complain about the readout performance - I just wanted to report back that the system worked good!
Ahh OK! I wanted to do some performance improvements anyway because I felt sure it was possible to improve things. I’ll work out the best way of removing the next performance obstacle…and probably move it somewhere else! I’m not sure what I’ve got wrong at the moment because after about a minute with 500+ disturbers the sensor stops sending interrupts or my code stops receiving them. That’s the next thing to fix.
I wouldn’t worry about the tuning capacitor value moving around a little. As long as it’s less then 3.5% then it’s good enough for the sensor so I still use AUTO runs even though it sometimes gives a value 1 or 2 either side of what seems to be my average (which like you is also 80pF).
There isn’t one. It’s just a number between 0 and approx 2 million that the sensor assigns. It’s only useful as a relative value to compare one stroke to another.
Progress on v3.3 is slow. I’m using the lightning sensor work as an opportunity to learn more about Python so when I want to do something new I have to research the options and try to decide which is the best one, then try it, find it wasn’t and then try the next option. It’s a slow process, but I’m learning fast and I’m hoping that v3.3 will be considerably better than v3.2…maybe not in functionality but definitely in terms of how it achieves the functionality.
I’m finally making progress again. I’ve spent the last 10 days chasing a weird bug that stopped interrupts from being handled. I’ve tried all sorts of things to try to fix it without success. What’s more annoying is that having moved some code around today to try to narrow down what was causing the problem, the problem has gone away. I don’t understand what I did that fixed it but I’m just accepting that the IT Gremlins like the new code structure better than the old so they’re allowing it to work. The code is working well now, happily handling 3-400 events per minute which I think should be enough for most people.
Now I can get back on track making all the other changes that are part complete or not started.
I’ve been sidelined for a couple of weeks. My Davis VP2 choked up with debris in the rain bucket so I decided to do maintenance on it.
I wound up replacing the rain tipper with the new single spoon, the FARS fan, temperature/humidity sensor, super cap and the new style rain bucket. Plus cleaning all the plastic which was growing mold and green gunk.
I would have had it done in a day, but one of the threaded rods broke. I ordered three as a set and chose First Class mail instead of Priority. They took a week getting here from Florida to Pennsylvania. When they got here I discovered I order the wrong ones.
But It’s back on line and now I can get the outside Lightening Pi installed.
Sounds like you had a major overhaul. At least you should be good for a few problem free years now.
I left my system running overnight with the settings allowing a lot of disturbers and it’s been processing an average of about 450 events per minute for about 12 hours. The CPU is running at about 69C which is fairly high but I’ve not made any attempt to cool it (no fans and the Pi isn’t really best oriented to allow the heat to escape passively. I think that’s a pretty good soak test so today I can go back to the remaining v3.3 mods.
I’m about to start on a major modification which, if it works, will add a useful new feature and will also completely replace another part of the existing system. I’m not going to say what the changes are just in case they come crashing down round my ears, but I should hopefully know in a few days if it’s looking like this will work. It actually moves the system in a direction I wanted to go in to help with future projects and it should fix a problem I’ve been experiencing with the current code as well as simplifying the overall code footprint.
My proof of concept testing has gone much better than I expected so I can now let you know what my plans are. I’m making two significant changes…
Replacing the Python web server, which was specifically written to display AS3935 data, with the commonly used Apache web server. This will make it far easier to modify pages and for you to add your own pages if you wanted to. It also means that I can easily add pages for other IoT projects without having to either have multiple Python web servers or one big Python web server that has to support all of my projects. The pages are a combination of HTML and JavaScript, much like the current pages, but now used directly from Apache.
The AS3935 monitor process talked to the Python web using Socket.IO which I’ve been having problems with. I’m sure the problems are due to my way of using it rather than the library having bugs. However, I seemed to be taking one step forwards followed by another backwards, so I decided to drop Socket.IO in favour of MQTT. The Mosquitto MQTT broker server will be installed on the Pi. The AS3935 monitor will publish data to it and the web pages will subscribe to the data topics they need to build the page view. Doing it this way isolates the monitor from the web server. As long as the MQTT broker is running then the monitor can publish to it and the web pages can talk to it. This approach also means that people can collect AS3935 data directly using their own MQTT client and not use the web pages at all if they wanted to. Whilst I’m running Mosquitto on the Pi the monitor code is configurable to allow another MQTT broker, including a public broker, to be used if you already have/use one. Using MQTT also means that the next projects I build can publish data into MQTT as another topic, with new web pages for the other project just subscribing to the topics that are needed for those pages. It also means that if someone wanted to create a web page using data from multiple projects that would also be relatively straightforward to do.
At the moment, I have a modified version of the monitor publishing all the data into Mosquitto, Apache2 installed and running with a test web page subscribed to one of the topics. The web page simply dumps the topic’s data elements out at the moment. I now need to replicate the original web pages which will take a little time, but I know what I need to do, and also improve the use of MQTT in the monitor, e.g. by adding QoS and some more error trapping.
This approach has shrunk the monitor source code by about 15% and also leaves just one config file which is also a lot smaller and simpler.
I’m making good progress again. The MQTT implementation is working well and I now have the web pages re-created as well as JSON pages to display the latest statistics and current settings. Nowcast data is now also available in JSON format although I’m not sure if that’s too useful…it was so easy to do I just did it!
The new way of doing things seems to be good to a sustained rate of about 510-520 events per minute but I’m not sure if that limitation is in my code or just that’s the typical rate of events produced by the sensor in my electrically noisy test location. The web page also now populates with data very quickly after loading unlike previously where there could be a 15-20 delay. Removing the delay came with zero development effort with the new MQTT method which is great because I was getting stuck trying to improve page load speed with the old Socket.IO solution.