After a server failure from 01/21/2023 - 01/24/2023 my historical data (IBAYERNN7) disappeared in nirvana.
I haven’t heard anything from WU support apart from a Response number with nothing to say; I’m fed up with WU now.
Hence my question: Couldn’t I use my own historical data for further graphic processing instead of WU
provide? What must be changed?
Can you post the URL of your site?
The answer to your question is yes, you can change that in easyweathersetup, but I really would wait to see if they recover the historical data - mine is OK.
Very strange, it seems you are using two different WU-ids
Currently your graphs use INEUMA83
=> the cache wudata/INEUMA83-metric-2023.arr only has dat from Jan 25.
=> the cache for last year is empty
Your message says IBAYERNN7
=> the cache wudata/IBAYERNN7-metric-2023.arr has data from 20221231 (= correct) until 20230122
=> the cache wudata/IBAYERNN7-metric-2022.arr data contains the correct periode for 2022.
=> allk orther caches are there with data from 20100913
Could you please check your settings, if you want to use IBAYERNN7 it should be in your settings also.
Wim
I can’t find IBAYERNN7 on the WU dashboard website. . .
Thank you for your quick response. IBAYERNN7 no longer exists at WU;
(https://www.wunderground.com/dashboard/pws/IBAYERNN7), I can’t upload or download weather data either.
All my history data is gone. I reported this to WU Support. The answer is attached.
I created INEUMA83 on January 23, 2023 and have been uploading weather data to WU since then,
I activated it today in the dashboard.
But I also have this historical weather data on my server in a MySQL database.
How can I use my data instead of WU for the graphs?
Willi
Sorry, I didn’t realise you had lost IBAYERNN7 altogether 8O
There is no easy answer.
For non-WU users, the dashboard-cron-scripts store the station weather-data in text files and that data can be used for a subset of the graphs.
Only today/yesterday detailed graphs and summurised 360 day graphs.
The graph-scripts are primarily designed to use WU’s data.
That will give best / most extensive results. Multiple periods, daily and period graphs. Integration with analyze scripts.
The WU-data is loaded from WU using a MITM script as a few years ago the data format changed from CSV to json.
There are two different “requests” towards the WU-website:
- The detailed data for 1 day
- The High_Low_Averaqe data for a period of days.
WU returns the data in json format. The MITM script compresses that data into the old .CSV format which is saved into the cache files.
A solution ??
What could be done, if you are a PHP programmer, is to adapt the link to WU in the MITM script to load the data not from WU but from a script you have to write.
That script has to load the needed data from your database and return the data into the standard WU json format.
That is not a trivial task, but it seems doable.
Wim
P.S. Question: Are you using a standard DB-design defined by another weather-program or f.i. designed by MeteoTemplate?
Or is it your own database design?
Hello Wim,
thank you for your information. Well, I have basic knowledge of php, I’m not a real php programmer.
What is the MITM script? Yes, the database design is from MeteoTemplate. Maybe you could help me for the script
that I would have to program? #-o
Willi
Hi Wili,
MITM stands for “Min-In-The-Middle” . Often with a negative connotation when used by script-kiddies.
The positive meaning: A script which is used between old valuable scripts which use an old API and a data source with a new API.
[ul][li]The old script uses the old API, but not towards the data source but to the MITM script[/li]
[li]the MITM-script translates the old API-request to the new API-request[/li]
[li]retrieves the data from the data source[/li]
[li]converts the data to the old format[/li]
[li]and returns the data to the requesting script in the old format.[/li][/ul]
Example: Translating the new WU-json API towards the old WU-csv format.
That way the old scripts can still get the information in the old format.
You know the format of the MT-database.
In your case solution 1: Your script will get URL-loaded with
. . .yourscript.php?graphspan=custom&year=2022&month=02&day=05&yearend=2023&monthend=02&dayend=05&format=1
and it would have to return the data in the old standard format, check your files for more examples
Date,TemperatureHighC,TemperatureAvgC,TemperatureLowC,DewpointHighC,DewpointAvgC,DewpointLowC,HumidityHigh,HumidityAvg,HumidityLow,PressureMaxhPa,PressureMinhPa,WindSpeedMaxKMH,WindSpeedAvgKMH,GustSpeedMaxKMH,PrecipitationSumCM
2022-02-05,8.4,5.5,2.7,5.0,2.5,1.1,92,81,66,1022,1016,21,3,35,0.00
2022-02-06,9.6,7.5,4.3,8.4,5.9,3.1,96,90,79,1016,1001,35,8,47,1.19
. . .
2023-02-03,10.5,9.5,8.8,8.2,6.9,5.9,91,81,78,1032,1026,20,5,26,0.00
2023-02-04,9.8,8.4,7.2,8.6,7.4,6.5,94,90,84,1037,1032,10,1,18,0.28
Or solution 2:
If you have more time to spent, make it more modern and return the data in the format WU uses nowadays: the .json format.
That would leave the current dashboard MITM script as is, it would only need the new link to your script.
But it would have the advantage that your script can be used for more purposes in the future.
Also as I understand it correctly, there are more MT-users already using the PWS_Dashboard, so they could benefit using their MT-database. Which probably has more data-items.
Example of MITM-scripts already used in PWS_dashboard 2012_lts:
PWS_DailyHistory.php => translating WU-json to old WU-csv
PWS_quakes_load.php => translating earthquake-data-xyz file to the most used format from the past.
Hope this explains it. I have never installed MeteoTemplate so I have no idea how the data is stored.
I also do not know if there are high-level access scripts towards the MT-database already available.
When proposing this solution, I assumed MT stores 5 minute data with a time stamp.
Maybe MT uses multiple tables: a table with 5 minute data per day and a table with daily data with high-low-average data.
Hope this explains it enough,
Wim
Another idea:
After making a backup of the wu folders with the arrays!
You could rename the 2022 and earlier .arr files.
F.i. wudata/IBAYERNN7-metric-2022.arr to wudata/INEUMA83-metric-2022.arr
The name of the station is not kept inside the .arr
So “in theory” the old yearly-files should be used as is for the graphs.
If that works in the graphs delete the INEUMA83-metric-2023.arr (with only a few days)
and rename also wudata/IBAYERNN7-metric-2023.arr to wudata/INEUMA83-metric-2023.arr
Wim
Thanks very much. A good idea, I can integrate my MySQL-database as a database without any worries.
Something surprises me: Although there is no file with IBAYERNN/… in ./chartswudata,
is it in status array? (see Schreenshuts).
Willi
Yes, all files loaded by “the cron-job → PWS_load_files.php” will be saved and remain in that file.
Delete the status.arr and run the cron-job. The cron-job will recreate the status.arr.
At first it will contain only a few loaded files but within 24 hours all current active files will be in the status.arr.