Raspberry Pi Lightning Detector v4.0

This is version 4.0 of this project. It is likely to be the last major release for a while. It is a very different program than the previous v1/2/3 releases and includes different technologies and solutions than the previous versions.

You shouldn’t need to rebuild your Pi if you’ve already used version 1, 2 or 3, but I would recommend creating a backup of your existing ‘Sensors’ directory before staring the installation. You can then create a new empty ‘Sensors’ directory before installing version 4. This version will install and run over the version 3, 3.1 or 3.2, but if you want to be able to roll-back to the previous version you should ensure you have a copy of the previous install before you proceed. If you are currently running v3, 3.1 or 3.2 you can copy the as3935_monitor.config file and the .log files in the current Sensors directory to the new directory before starting the installation.

I’m using an ams AS3935 lightning detection sensor IC with a Raspberry Pi to detect lightning strikes within approx. 40km. To quote from the manufacturer about the AS3935…

AS3935 Franklin Lightning Sensorâ„¢ IC is a programmable fully integrated lightning sensor that detects the presence and approach of potentially hazardous lightning activity in the vicinity and provides an estimation on the distance to the head of the storm. The embedded lightning algorithm checks the incoming signal pattern to reject the potential man-made disturbers. The AS3935 can also provide information on the noise level and inform the external unit (e.g. microcontroller) in case of high noise conditions, with the noise floor generator and noise floor evaluation blocks.
[ul] [li]Lightning sensor warns of lightning storm activity within a radius of 40km[/li] [li]Distance estimation to the head of the storm down to 1km in 14 steps[/li] [li]Detects both cloud-to-ground and intra-cloud (cloud-to-cloud) flashes[/li] [li]Embedded man-made disturber rejection algorithm[/li] [li]Programmable detection levels enable threshold setting for optimal controls[/li] [li]SPI and I²C interface is used for control and register reading[/li] [li]Antenna Tuning to compensate variations of the external components[/li] [li]Supply voltage range: 2.4V to 5.5V[/li] [li]Power-down, listening, and active mode[/li] [li]Package: 16LD MLPQ (4x4mm)[/li] [/ul]

The project is currently using module parts, which means that it’s more or less plug-and-play. That does mean it’s probably more expensive too. However, if you have the skills/knowledge you can build it from individual parts/modules which will probably be cheaper. If you can solder (7 wires) to a small-ish board then you can save some money by using the second approach. I have a different AS3935 board to test but haven’t had time to do that yet. I’ll document it when I’ve tested it.

Hardware

  • 1*Raspberry Pi - I’m using a 4GB Model 4B but it also works on Model 3 devices with less RAM and may work on Model 2’s too. The GPIO interface on Model 1’s isn’t compatible though, so it won’t work on Model 1’s, nor on the Rasbperry Pi Zero which is underpowered to run all the required components. Raspberry Pi’s are available from many sources so I’m treating them as commodity devices and I won’t discuss them too much within the project. My Pi has the ‘Full’ version of Raspberry Pi OS installed using the Pi Imager tool which is downloadable from the Raspberry Pi site, but it should also work on other versions on the OS. These instructions assume that your Pi already has the OS installed, is set up to work on your network, that you’re able to log into it and that it’s updated with the latest Pi OS.
  • 1*Mikroe Pi 3 Click Shield - This is a Pi addon board designed to plug into the 40-pin GPIO connector on a Raspberry Pi. It provides two mikroBUS sockets which allow you to plug a wide variety of ‘click boards’ into the Pi. This is a simple way to add sensors onto a Pi, but be aware that Pi software isn’t available for all of the sensor boards so you may have to write your own/adapt other code. You can buy the Pi 3 Click Shield direct from Mikroe Pi 3 Click Shield in the US, and also some electronics suppliers in other countries, e.g. RS Components or Farnell in the UK. Note that whilst this is called a ‘Pi 3’ shield it is also compatible with the Pi 4 (and I think the Pi 2). I believe there is also a single board Click Shield available which should be a little cheaper.
  • 1*Mikroe Thunder Click board - This is a mikroBUS module that plugs into the Pi 3 Click Shield. It has an AS3935 lightning detector chip and a MA5532 coil antenna built onto the module with supporting components. You can buy the Thunder Click direct from Mikroe Thunder Click in the US, and also some electronics suppliers in other countries, e.g. RS Components or Farnell in the UK.

Assembly

  1. Before starting, turn your Pi off and detach any cables that are plugged into it.
  2. Plug the Pi 3 Click Shield onto the 40-pin GPIO connector on the Pi. Follow the instructions for the Click Shield for how to do this.
  3. Plug the Thunder Click board into one of the sockets on the Click Shield. It will work in either socket. Mine is plugged into socket 2 because I have another module plugged into socket 1. You will need to use the socket number (1 or 2) in the installation below. Ensure that the small GPIO/ADC switch on the Click Shield for the socket you choose is set to GPIO. This should be the default out of the box setting.
  4. Plug the cables back into your Pi and turn it on. After the Pi has started to boot up you should see the green LED on the Thunder Click board light up.

That’s it for assembly. The rest is down to software.

Software

As mentioned above, not all Click Boards have software available for them on the Raspberry Pi. This is true in the case of the Thunder Click board. There is some software available to use an AS3935 but much of this is for using the chip on its I2C interface. The Thunder Click is configured to use the SPI (Serial Peripheral Interface) bus which meant that I’ve had re-work some existing code to make it work. I understand that the AS3935 SPI interface is more reliable than the I2C interface so that’s good for this project.

The project now only has a single installation option. This installs Apache2 web server, the Mosquitto MQTT Broker, pigpio to control the SPI bus, various Python modules to assist with controlling the AS3935 and the software I have developed/modified. An AS3935 monitor program runs in the background, managing the AS3935 and watching for events, e.g. lightning strokes. It passes event and other information to the MQTT broker. Web pages in Apache2 and CGI programs access the MQTT broker to read data for output through a browser. The data can also be accessed via an MQTT connection, either MQTT protocol on port 1883 or using MQTT over WebSockets on port 9001. Data is presented in a variety of HTML/JavaScript, JSON and text formats.

There are a number of steps required to set the software up…

  1. Enable the SPI bus on the Raspberry Pi. See https://www.raspberrypi-spy.co.uk/2014/08/enabling-the-spi-interface-on-the-raspberry-pi/
  2. The software uses Python 3 to access the AS3935 and pip3 during the installation process. If you’re running Raspberry Pi OS (previously called Raspbian OS) then you will probably have both Python 3 and pip3 installed by default. If you’re using another OS or installed a version of the Raspberry Pi OS without Python 3/pip3 included please install them using the appropriate method for your OS. You can check if you have them installed by typing ‘python3’ in a terminal window. If you get a ‘>>>’ prompt then Python3 is installed (press Ctrl+D to exit Python). To check if you have pip3 installed, type ‘pip3’ in a terminal window. If you see help text displayed then pip3 is installed.
  3. Create a directory “/home/pi/Sensors” for the AS3935 software. At this time, the software MUST be installed into this directory. If you want to have the software in a different directory you will need to modify some files manually to enable this, e.g. the monitor service unit file.

The next steps include the installation of some pre-requisite software and also the AS3935 Python software with the associate config/support files. This is done by uploading an installation ZIP file, extracting it and then using an installation script to complete the process. Most of these instructions need to be carried out in a terminal window, either one opened from the Pi Desktop or a remote one using SSH and software such as Putty. The instructions are the same whether done from the Pi Desktop or remotely. You will need to press the after terminal window commands to make them run.

  1. Download the attached ‘as3935_install_v4_1.zip’ file, upload it into the Sensors directory on your Pi. Software such as WinSCP can be used to make the upload fairly straightforward.
    [li]Open a terminal window. You should be in the ‘/home/pi’ directory. Confirm this by typing ‘pwd’ to display the ‘present working directory’. If you’re in a different directory change into the correct directory by typing ‘cd /home/pi’.
    [/li][li]Change into the Sensors directory by typing ‘cd Sensors’.
  2. Unzip the uploaded ZIP file using ‘unzip -o as3935_install_v4_0.zip’. If you are asked about overwriting files answer press ‘y’ and RETURN to confirm.
  3. The next step depends on which socket you installed your Thunder Click Board into. Replace ‘X’ with ‘0’ or ‘1’ for socket 1 or 2 respectively. Firstly, make the installer script executable by typing ‘chmod +x as3935_installer.sh’. Next, run the script by typing ‘sudo ./as3935_installer.sh X’. You must use sudo to run the installer to ensure that all files are copied to the correct locations. The installer will take a few minutes to run, depending on what software needs to be installed, the speed of your Internet connection and which model of Raspberry Pi you’re using. It will display progress information as it runs.
    [li]When the installer has finished you should see some information displayed from a short test program. It looks something like this…
    The test run results below should hopfully indicate that the installed version of pigpio is v77 or higher.
    The register outputs should be something similar to 0x24, 0x07, 0xC2, 0xC0, 0x25, 0xE4, 0x07, 0x3F, 0x0A.
    There will be some variance in the register values due to the unknown state of the AS3935 sensor in your
    configuration. If they have completely different values, e.g. all 0x00 or 0xFF, then you will need to do
    additional checks to ensure that your AS3935 is accessible in the expected way on your Raspberry Pi.

    > Test starting
    > Good version of pigpio found - Version 77
    > Register 0x00 is 0x24
    > Register 0x01 is 0x02
    > Register 0x02 is 0xC2
    > Register 0x03 is 0x00
    > Register 0x04 is 0x00
    > Register 0x05 is 0x00
    > Register 0x06 is 0x00
    > Register 0x07 is 0x3F
    > Register 0x08 is 0x0A
    > Test complete

    Confirm that the pigpio is version is at least 77 and that the register values from your system are similar to those displayed. They may not be exactly the same but if they’re similar that’s good.[/li]
  4. Next use the configuration file editor to configure the system as you require. Type ‘sudo python3 as3935_config.py’. The editor allows you to select the options to be edited, describes what each one does and the default/allowed values. Most values can initially be left as default. The main value to check is the ‘address’ value in the [as3935] section. This should be set depending on which socket your Thunder Click Board is plugged into.
  5. After editing the configuration file you can start the monitor service (background process) by typing ‘sudo systemctl start as3935_monitor.service’. Once started you can check the status of the service by typing ‘sudo systemctl status as3935_monitor.service’. You should see a line including ‘Started AS3935 Monitor service’ which indicates that the service is running correctly. The service is re-started automatically each time you reboot your Pi. Apache2 and Mosquitto also re-start after a reboot. The monitor process will take over 2 minutes to start if you have AUTO-calibration enabled because the calibration process doesn’t run properly whilst the Pi OS is starting up, i.e. when the system startup processes are consuming a lot of CPU capacity.
  6. Now use your web browser to view the web server output. Navigate to ‘http:///as3935’ to see the main page. In my case I navigate to ‘http://192.168.1.241/as3935’. If the page is left on display the stats, settings and events data are automatically updated as new data is published to the MQTT broker. The page also contains buttons/links to access the provided JSON/text files. You can access these files directly using their URL, e.g. by adding the nowcast.txt URL to Weather Display to download lightning data in WD.
    [/ul]


    I have also provided a program calibrate_pig.py. To run this for the Thunder Click Board plugged into socket 1 or 2 use ‘sudo python3 calibrate_pig.py X’ where X is 0 or 1 respectively. The program will run for 16 seconds and provide the tuning capacitor setting that you can include in the as3935_monitor.config file if you wish. By default the monitor program will re-calibrate each time it’s run, just with a 16 second delay when it’s started (or longer just after a reboot). I would recommend leaving the AUTO option enabled in the config file, but you can hard code the tuning value using the calibration program if you want to. Note: The calibration needs to be within +/-3.25% for the sensor to run correctly. If the calibration program can’t find a capacitor value that is within tolerance then you will need to investigate why this is happening and the software won’t run (in AUTO mode).

    Notes:

    1) The Apache2 web server can easily serve your own web pages containing other data, e.g. you could install FreshWDL and display the output locally using Apache2. If you are doing this, please use other locations/directories than are used by the AS3935 web pages to avoid confusion.

    2) The Mosquitto MQTT broker can also be accessed by MQTT clients to read the AS3935 data. You can also publish data to it from other clients to use it as your local broker. Just avoid the ‘as3935’ topic if you publish your own data.

    3) For those not using the Pi Click Shield and/or the Thunder Click board, the software now supports you better. You can define the IRQ GPIO in the config file (and via the config editor too). If you want to use different bus/address values than allowed by the validation in the editor you can manually enter them into the config file. The config editor only supports the bus=0, address=0/1 and bus=1, address=0/1/2 combinations. I think that other bus/address values should work but have no current way to test this. If you do set alternative bus/address combinations make sure you set an IRQ value because I’ve no way to work out what that will be for your combination of settings. The as3935_pigpio_test.py and calibrate_pig.py programs also accept additional arguments for bus and irq. For each program arg1=address, arg2=bus and arg3=IRQ.

    4) Finally, I have provided all of the source code required to make this work or it’s available for download separately, e.g. Apache2 and Mosquitto. Feel free to see how I’ve done things and I’m happy to hear suggestions of how to do any of it better. I will be the first to admit that the code could be documented better and there should be more error traps. If you’d like to extend/modify the code please do. If you do then please share the mods/new code with the community as I have done. Then we can all learn and see how to do things differently. The only proviso on that is that the code that I’ve produced must not be used for commercial purposes, e.g. used in a device that detects lightning or sold as software that can run such a device. I’ve made the software freely available and I don’t want others profiting from my work.

as3935_install_v4_1.zip (64.4 KB)

Thanks for this, Chris. I’ll be installing shortly!

It looks like everything installed correctly and indicates that pigpio is the correct version.

Unfortunately, all the registers come back as 0x00.

I used the installer with a socket of ‘0’ as my board is in socket 2.

What to do?

If your board is in socket 2 then install using ‘1’ not ‘0’. Socket 1 = address 0 and Socket 2 = address 1

Sorry, Chris. That should have been a “1” on the installer, not a “0”.

I used the installer with a socket of '1' as my board is in socket 2.

Can you please look in the as3935_monitor.config file and let me know what values you have for [pi][bus] and [as3935][address] and whether there is a value set for [as3935][irq_override]?

[pi][bus] - 0
[as3935][address] - was ‘0’ changed to ‘1’
[as3935][irq_override] - This is in comments but not in the actual settings

The as3935_monitor.config doesn’t run during the initial install does it? The oxoo were all at the end of the initial install.

Try running…

sudo python3 as3935_pigpio_test.py 1

…to see if the registers show correctly with the address set to 1.

Chris,

Shouldn’t the following:

Next use the configuration file editor to configure the system as you require. Type 'sudo python3 as3935_config.py'. The editor allows you to select the options to be edited, describes what each one does and the default/allowed values. Most values can initially be left as default. The main value to check is the 'address' value in the [as3935] section. This should be set depending on which socket your Thunder Click Board is plugged into.

be actually ‘sudo nano as3935_monitor.config’?

Here’s what I get:

Test starting
Good version of pigpio found - Version 77
Register 0x00 is 0x00
Register 0x01 is 0x00
Register 0x02 is 0x00
Register 0x03 is 0x00
Register 0x04 is 0x00
Register 0x05 is 0x00
Register 0x06 is 0x00
Register 0x07 is 0x00
Register 0x08 is 0x00
Test complete

That’s the old way if don’t it. You can still edit the file manually but ‘sudo python3 as3935_config.py’ runs the configuration editor which edits the as3935_monitor.config file and validates the input as you enter it.

I’ll try swapping my board into socket 2 and see if I can find out what’s going wrong for you. It’s bedtime here now though so it will be a job for tomorrow. Just to confirm, you did an install onto a fresh built operating system. Did you copy any files from the old system to the new or was it a complete new install?

At the end of the installation i got the following error:


pigpio.error: 'bad SPI channel'

And after running ‘sudo python3 calibrate_pig.py 2’:

The same error:

sudo python3 calibrate_pig.py 2
Traceback (most recent call last):
  File "calibrate_pig.py", line 48, in <module>
    as3935 = RPi_AS3935(address=as3935_address, bus=pi_bus, mode=0b10, speed=800000, irq=irq)
  File "/home/pi/Sensors/RPi_AS3935_PIGPIO.py", line 16, in __init__
    self.as3935 = self.spi.spi_open(address, speed, spi_mode)
  File "/home/pi/Sensors/pigpio_spi_handles.py", line 68, in spi_open
    new_handle = super().spi_open(address, speed, spi_mode)
  File "/usr/local/lib/python3.7/dist-packages/pigpio.py", line 3939, in spi_open
    self.sl, _PI_CMD_SPIO, spi_channel, baud, 4, extents))
  File "/usr/local/lib/python3.7/dist-packages/pigpio.py", line 1011, in _u2i
    raise error(error_text(v))
pigpio.error: 'bad SPI channel'

Tried with 0, 1 for the other socket but that gives:

Starting calibration run - Please wait approximately 16 seconds
calibration_run - At 0 pF 0.0 IRQs/sec with variance from ideal of -3906.2 ( -100.00 %)
calibration_run - At 8 pF 0.0 IRQs/sec with variance from ideal of -3906.2 ( -100.00 %)
calibration_run - At 16 pF 0.0 IRQs/sec with variance from ideal of -3906.2 ( -100.00 %)
calibration_run - At 24 pF 0.0 IRQs/sec with variance from ideal of -3906.2 ( -100.00 %)
calibration_run - At 32 pF 0.0 IRQs/sec with variance from ideal of -3906.2 ( -100.00 %)
calibration_run - At 40 pF 0.0 IRQs/sec with variance from ideal of -3906.2 ( -100.00 %)
calibration_run - At 48 pF 0.0 IRQs/sec with variance from ideal of -3906.2 ( -100.00 %)
calibration_run - At 56 pF 0.0 IRQs/sec with variance from ideal of -3906.2 ( -100.00 %)
calibration_run - At 64 pF 0.0 IRQs/sec with variance from ideal of -3906.2 ( -100.00 %)
calibration_run - At 72 pF 0.0 IRQs/sec with variance from ideal of -3906.2 ( -100.00 %)
calibration_run - At 80 pF 0.0 IRQs/sec with variance from ideal of -3906.2 ( -100.00 %)
calibration_run - At 88 pF 0.0 IRQs/sec with variance from ideal of -3906.2 ( -100.00 %)
calibration_run - At 96 pF 0.0 IRQs/sec with variance from ideal of -3906.2 ( -100.00 %)
calibration_run - At 104 pF 0.0 IRQs/sec with variance from ideal of -3906.2 ( -100.00 %)
calibration_run - At 112 pF 0.0 IRQs/sec with variance from ideal of -3906.2 ( -100.00 %)
calibration_run - At 120 pF 0.0 IRQs/sec with variance from ideal of -3906.2 ( -100.00 %)
Failure - minimum variance ( 100.00 %) > 3.5% of LCO frequency

SPI is enabled.
PI was running good on version 3.2.

My install was a brand new Pi setup. No V3 files to be seen.

Asobig - you would get bad SPI channel using address=2 in calibrate_pig.py because address 2 is only valid for bus=1 and I suspect you’re probably not running on the secondary Pi SPI bus. Calibrate_pig.py is fairly primitive and doesn’t do any validation of the input values.

Which socket is your board plugged into? If it’s 2 then there’s a common thread with mldenison. If it’s not then I’m wondering if I’ve missed a file out of the instill package…although I did an install onto a fresh build which should have picked that up.

I’ll do another install onto a fresh build tomorrow to see if I can see anything going wrong.

Sorry, you are right, it was late…
My thunder click is in socket 2.
Deleted the map and started over, i got the same error as midenison.
The end of the install:

Test starting
Good version of pigpio found - Version 77
Register 0x00 is 0x00
Register 0x01 is 0x00
Register 0x02 is 0x00
Register 0x03 is 0x00
Register 0x04 is 0x00
Register 0x05 is 0x00
Register 0x06 is 0x00
Register 0x07 is 0x00
Register 0x08 is 0x00
Test complete

The config file:

[pi]
bus = 0

[as3935]
address = 0
tuning_cap = AUTO
mask_disturbers = False
location = indoors
min_strikes = 1
noise_floor = 0
spike_rej = 2
watchdog_threshold = 1

No line with [as3935][irq_override]

The calibration run:

pi@thunder:~/Sensors $ sudo python3 calibrate_pig.py 1
Starting calibration run - Please wait approximately 16 seconds
calibration_run - At 0 pF 0.0 IRQs/sec with variance from ideal of -3906.2 ( -100.00 %)
calibration_run - At 8 pF 0.0 IRQs/sec with variance from ideal of -3906.2 ( -100.00 %)
calibration_run - At 16 pF 0.0 IRQs/sec with variance from ideal of -3906.2 ( -100.00 %)
calibration_run - At 24 pF 0.0 IRQs/sec with variance from ideal of -3906.2 ( -100.00 %)
calibration_run - At 32 pF 0.0 IRQs/sec with variance from ideal of -3906.2 ( -100.00 %)
calibration_run - At 40 pF 0.0 IRQs/sec with variance from ideal of -3906.2 ( -100.00 %)
calibration_run - At 48 pF 0.0 IRQs/sec with variance from ideal of -3906.2 ( -100.00 %)
calibration_run - At 56 pF 0.0 IRQs/sec with variance from ideal of -3906.2 ( -100.00 %)
calibration_run - At 64 pF 0.0 IRQs/sec with variance from ideal of -3906.2 ( -100.00 %)
calibration_run - At 72 pF 0.0 IRQs/sec with variance from ideal of -3906.2 ( -100.00 %)
calibration_run - At 80 pF 0.0 IRQs/sec with variance from ideal of -3906.2 ( -100.00 %)
calibration_run - At 88 pF 0.0 IRQs/sec with variance from ideal of -3906.2 ( -100.00 %)
calibration_run - At 96 pF 0.0 IRQs/sec with variance from ideal of -3906.2 ( -100.00 %)
calibration_run - At 104 pF 0.0 IRQs/sec with variance from ideal of -3906.2 ( -100.00 %)
calibration_run - At 112 pF 0.0 IRQs/sec with variance from ideal of -3906.2 ( -100.00 %)
calibration_run - At 120 pF 0.0 IRQs/sec with variance from ideal of -3906.2 ( -100.00 %)
Failure - minimum variance ( 100.00 %) > 3.5% of LCO frequency

If your board is in socket 2 then the address in the config file should be 1 but the test and calibrate programs don’t use the config file so there’s definitely something broken when using socket 2. I did most of my recent testing using socket 1 so maybe I changed something after moving my board from socket 2 to socket 1.

It works!!!

But when i tried to make other changes to the config i got this massage:

pi@thunder:~/Sensors $ sudo python3 as3935_config.py
Traceback (most recent call last):
  File "as3935_config.py", line 1533, in <module>
    main()
  File "as3935_config.py", line 1473, in main
    create_mon_new()
  File "as3935_config.py", line 1372, in create_mon_new
    val_as3935_address(int(curr_config.get('as3935','address'), 0) if (curr_exists and curr_config.has_option('as3935','address')) else None)
  File "as3935_config.py", line 150, in val_as3935_address
    new_config['as3935']['address'] = aa
  File "/usr/lib/python3.7/configparser.py", line 1255, in __setitem__
    self._parser._validate_value_types(option=key, value=value)
  File "/usr/lib/python3.7/configparser.py", line 1182, in _validate_value_types
    raise TypeError("option values must be strings")
TypeError: option values must be strings

I’ve fixed the problems. The as3936_pigpio_test.py and calibrate_pig.py problems were due to me forgetting to update the files in the installation package. The released versions had a missing ‘not’ in them which made a big difference. They worked OK for socket 1 but not socket 2. The config editor problem is also fixed.

Download the updated (v4_0_1) zip file attached to the first message of this thread and unzip it into the Sensors directory using ‘unzip -o as3935_install_v4_0_1.zip’ and that should fix the problems. There’s no need to do a re-install because it’s just replacing 3 files in the Sensors directory.

Sorry for having caused confusion so soon after release!

Yes, that worked.

One question.
In the old 3.2 version there was an option to get the last strikes as a JSOn file (events.json).
I used that to show the last strikes on my site.
Is that still there? I can’t find it.