Auorora (KP-index) file missing ** Updated **

A coronal hole is now facing the earth, so in 1-2 days time the A and K will spike and we can watch the files numbers and compare to actual values to better understand the a-running number.

Regards,

jim

The “a_running” numbers in the file are definitely direct conversions from the corresponding “Kp_fraction” numbers, as per the table on the page you referenced. I think you use those individual 3-hourly a-indexes to get the running average A-index.

You have a good point about the file being 3 hours behind. . .

Maybe we should ask NOAA. . .

Good idea!

BTW, i can't get your aurora_popup.php to load. The little cylon-eye just keeps cycling back and forth.

Whose? Mine seems OK here. . .

BTW, i can't get your aurora_popup.php to load. The little cylon-eye just keeps cycling back and forth.

Whose? Mine seems OK here. . .

Yes, I’ll try it again and let it run for a while, but something is amiss here. Perhaps i will re-c&p the code just to make sure i did accidently miss something.

Regards,

Jim

When testing use: . . /pwsWD/_test.php?test=aurora_popup.php
Or whatever script-name you used.
That will list the errors also.

Wim

Didn’t realise you were trying to run it! Note that my aurora_popup.php is a beta from 22 January 2019 for the July 2019 version of the dashboard, and all the include_once scripts are w34_, not PWS_, and I have made a lot of changes.

I did tell you it was very old :wink:

Yes, the errors i see surround the w34_livedata, which i can change to PWS, and w34_common but i don’t seem to have either a w34_common nor a PWS_common. So, I am trying to figure out which that code is in now.

Thanks. I am getting closer.

Regards,

Jim

You will also need to change the URL in PWS_loadfiles.php to load the new json. . .

Might have been easier just to lift the changes from my script to yours!

HA! Working! Thanks.

Regards,
jim

O.K. Working fine. Back to the data. The last entry from the swpc json shows a=3 but the latest wwv.txt says A=4. So, i suspect we need the highest value from the last x number of entries. Or, it is possible that the current A is not ascertainable from this file. I will keep watching the changes and try to ascertain what they are doing.

Thanks, we inch closer.

Regards,

Jim

Last entry is for 09:00 UTC. . .

[60] => Array
        (
            [0] => 2021-11-27 09:00:00.000
            [1] => 1
            [2] => 0.67
            [3] => 3
            [4] => 8
        )

while your solar-terrestrial data is bang up to date. I think this is the main problem.

Average of last 8 a-index values is 25/8 = 3, which is what my pop-up is showing.

Yes, but… the actual A-index now is 4.

:Product: Geophysical Alert Message wwv.txt :Issued: 2021 Nov 27 1200 UTC # Prepared by the US Dept. of Commerce, NOAA, Space Weather Prediction Center # # Geophysical Alert Message # Solar-terrestrial indices for 26 November follow. Solar flux 92 and estimated planetary A-index 4. The estimated planetary K-index at 1200 UTC on 27 November was 1.

No space weather storms were observed for the past 24 hours.

No space weather storms are predicted for the next 24 hours.

Regards,

Jim

As I said, the last data in the json is for 09:00 UTC. Can you look back at your e-mails and see what the A-index was at 09:00 UTC?

Yes,

:Product: Geophysical Alert Message wwv.txt :Issued: 2021 Nov 27 0900 UTC # Prepared by the US Dept. of Commerce, NOAA, Space Weather Prediction Center # # Geophysical Alert Message # Solar-terrestrial indices for 26 November follow. Solar flux 92 and estimated planetary A-index 4. The estimated planetary K-index at 0900 UTC on 27 November was 1.

No space weather storms were observed for the past 24 hours.

No space weather storms are predicted for the next 24 hours.

I think it will be the highest entry from the last x values and not an average… But i am not sure what x is.

Regards,

Jim

Solar flux 92 and [b]estimated[/b] planetary A-index 4. The [b]estimated[/b] planetary K-index at 0900 UTC on 27 November was 1.

Maybe that’s the problem, and the json is not updated until the values are certain?

From A NOAA website:

The relationship between K and A

The A-index was invented because there was a need to derive some kind of daily average level for geomagnetic activity. Because of the non-linear relationship of the K-scale to magnetometer fluctuations, it is not meaningful to take averages of a set of K indices. What is done instead is to convert each K back into a linear scale called the “equivalent three hourly range” a-index (note the lower case). The daily A index is merely the average of eight, three-hourly “a” indices.

Perhaps what’s needed is to use the table to convert the k index to equivalent ak and then take the average of the conversions:

ak Index

The ak index is a 3-hourly ”equivalent amplitude” index of geomagnetic activity for a specific station expressing the range of disturbance in the horizontal components. ”ak” is scaled from the 3-hourly station K-index according to the following table:

K 0 1 2 3 4 5 6 7 8 9
ak 0 3 7 15 27 48 80 140 240 400

The ak values can be converted to nanoteslas (nT) using a local, station-dependent conversion factor. The conversion factor is found by dividing the station’s lower limit for K=9 by 250. For example, at Boulder and Fredericksburg the lower limit for K=9 is 500 nT so the factor is 2; therefore the ak values for these stations are in units of 2 nT. (To obtain an equivalent amplitude in nanotesla for Boulder or Fredericksburg, the index value must be doubled.)

A index

The A-index is calculated for individual magnetometer stations. The value is calculated as the average of eight, three-hourly station ak-indices observed during a UT day, and provides a single, average value to indicate the activity level for that day.

or, we simply cannot calculate A from the information provided and need to find and current json source for the number.

That’s exactly what the “a_running” values in the json are: you can check by looking at each “Kp_fraction” value and using the table. They correspond exactly.

That’s why I think the A-index is the average of the last 8 “a_running” values. . . or it would be if the json was up to date :?

I have now crudely modified my old aurora_popup to show the 3-hourly Kp-index and ap equivalent-range index values from https://services.swpc.noaa.gov/products/noaa-planetary-k-index.json, and my understanding of the daily Ap-index (average of last 8 ap values). It also shows the date/time of the last file update (and if anyone can tell me how to cut the last :00.000, for seconds and thousandths of a second, off the published time they will earn my undying gratitude).*

I have added the letter p to the indexes to make it clear that they are “planetary”, but I have not changed any of the original storm/radio aurora warnings. I am not an expert in this field by any means.

It is becoming obvious that this file is not updated frequently enough for ham radio enthusiasts, who may prefer forecasted/estimated K and A indexes, but it is certainly better than the current one. I am only a scientist, and I know how long it can take to assemble data reported from several different observatories (indeed, I have worked at one) so I have no problem. The same thing happens with the AQI, sometimes: but nothing I do is affected by late K or AQ indexes, they are there for interest and day-to-day comparison. And as explained before I have no chance of viewing the Northern Lights from here!

I think we need a few more people to chip in to this discussion. . . I wonder how many dashboard users actually look at the popup?

  • LATER EDIT: I had defined the ridiculous NOAA timestamp in the json (e.g. “2021-11-29 12:00:00.000”) as $timest, and was unable to shorten it until I went back to w3schools.com and discovered strtotime().

$d 	  = strtotime("$timest");
$datetime = date("d-m-Y H:i",$d);

reformats it to a much friendlier “29-11-2021 12:00” :slight_smile:


json.png

I’ll take this as a chance to advance the discussion in another direction. Bitsostring and i engaged in some PM discussions about the shortcomings of the data sources and the math needed to come up with the conventional numbers found on geo-solar sites. The one thing we have agreed on is that neither of us have the php programming skills to modify this into where it ought to be.

Take a look at https://www.hamqsl.com/solar.html and see what Paul Herman has done. i wrote him and asked how/where he gets the A and K index data and he replied that he is parsing it from the text file found at https://services.swpc.noaa.gov/text/wwv.txt, which is the latest up to date source for A-index and K-index. But, it is well beyond my skills to parse a text file or write the PHP script to do so. Paul, however, kindly consented to allow access to his data in an XML format found at https://www.hamqsl.com/solarxml.php which is not only up-to-date, but also has a plethora of additional information. I have linked one of Paul’s widgets into my site in the bottom row, click my sticker below.

Ham Radio operators all over the world would rejoice in an updated script but, if you crack php programmers out there would consider adding a block for the solar flux and sunspot number ham radio operators will be ecstatic. Since it would be geometrically desirable to keep the number of blocks at an even number, i would suggest the solar wind and magnetic field numbers for a 4th block.

The solar flux and sunspot numbers give a more complete picture of HF radio conditions when you know the A and K indexes. The solar wind and magnetic field numbers give a better picture of where the A and K indexes are going and make it easier to predict aurora and the geomagnetic storms resulting in aurora. I can explain that in greater detail if folks so desire.

The bottom line here is that the aurora popup could be much more useful but the skills to rewrite the popup are way WAY WAY beyond me. Identified herein are the data sources, in a useful format, that could make an accurate and current popup.

If the gratitude of of bitsostring and myself are sufficient to entice a programmer, it would be courteous and desirable to credit Paul Herman, N0NBH and link his website.

Note that these are estimates/forecasts.

I also had a look at https://services.swpc.noaa.gov/products/noaa-planetary-k-index-forecast.json, which shows observed and predicted values of Kp. Unfortunately it does not show Kp_fraction, so we cannot produce a decent ap-index from the look-up table and hence an Ap-index by averaging.

I’ve been keeping an eye on these and I notice that the estimated K index is current but the estimated A-index can be for yesterday (or, rather, the last full UT day), e.g.

Solar-terrestrial indices for [b]26 November[/b] follow. Solar flux 92 and estimated planetary A-index 4. The estimated planetary K-index at 0900 UTC on [b]27 November[/b] was 1.

I therefore have to assume that we cannot calculate a current A-index from the last 8 a-indexes, because it is not a rolling average. You need the average of the 8 three-hourly a-indexes for a complete UT day:

The A-index is calculated. . . as the average of eight, three-hourly station a-indices observed [b]during a UT day[/b], and provides a single, average value to indicate the activity level for that day.

My mistake :frowning: (Although it is correct around midnight UTC! And notice that you can use a rolling average for a special Ap* index.)

I also have to assume that the A-index shown in the original aurora popup was in fact the a-index, calculated (as Wim said) from the K-index. Actually this should give a much more immediate indication of possible radio-aurora than an 24-hour A-index ever could. . .

The original json source provided a fractional K-index, so that it would have been possible to calculate a decent equivalent a-index from the table (although the script doesn’t actually do that), but the current json does not.

The json I am using provides a K-index and an equivalent a-index, so is preferable, but it is not updated regularly enough for some.

So Wim can relax until (if?) we get a consensus. Me? I’m going back to the drawing board tomorrow. . . :wink: