WeatherUnderground data directories

[pwsdashboard]

@Mehdi post:1

Hi Wim,

I was clearing some disc space on my local drive and noticed my site’s back up contains 4 separate directories which appear to contain WeatherUnderground data. I’ve been sending data to Wunderground since 2016 and of course would love to have access to the data (and I believe I do).

I’m thinking I might have used some scripts to convert the data for pwsWD consumption, and the directories are left over from that conversion. Not sure if they’re all needed at this time. The directories are: wsreports, wuCSV_Replace, wudata and wuhistory. The directory structure picture is attached.

• Can you identify what the function of these directories are and if they’re all necessary, please?
• I notice that the wudata folder is missing years 2016 and 2017, while the wuCSV_Replace folder is missing many of the month specific folders. How can I ensure that I’ve captured/transferred all of my data from Wunderground to pwsWD?

Thanks.

Your pwsWD/wuCSV_Replace/
=> I am not familiar with that folder and do not know were it comes from.
=> I can not find that name in any of my recent dashboard or Leuven scripts.
=> checking the contents https://armon.com/pwsWD/wuCSV_Replace/ all files are dated October 24
==> it seems an older copy of the https://armon.com/pwsWD/wudata/ folder which has today time-stamps.

pwsWD/wsreports/
=> Leuven script, stand-alone version.
=> the supported version tailored to dashboard => wureports/
=> This version is not correctly adapted to the new MITM script it uses pwsWD/WXDailyHistory.php
=> discard this folder and install the correct version from External scripts which use WeatherUnderground .CSV data

=================================
The folders most often used in correct installations:

pwsWD/wudata/
=> that folder is located inside the pwsWD/ folder.
=> should be copied to the new version-folder when updating the dashboard
=> data is needed for the graphs and can be used for the extra wu-scripts outside the pwsWD/ folder
=> data is generated by the MITM script pwsWD/PWS_DailyHistory.php by using API to load data and storing that data for general use
=> the folder should contain each years WU data in PHP-array .arr files which are updatable by a separate script _wu_upd.php
=> the .arr files are the main data storage files used to generate the .CSV data as used by all other scripts.
=> there should be one file named YOURWUID-metric-YYYY.arr for every year (YYYY) you were uploading to WU after the year 2008 (older data is lost from the WU data base)
=> the pwsWD/wudata/ folder can also contain cached versions of .CSV files in extra folders/subfolders with name YYYY/
==> year and month files inside the yearly folders can be discarded at any time
==> if daily .CSV files are removed the data need to be retrieved from WU again as that datas is not stored in the .arr data storage

wuhistory/
=> outside dashboard folder
=> dashboard version of my Leuven WU history script using the old .CSV format and supporting day, month, year and custom periods
=> it should not have any cached data

wureports/
=> outside dashboard folder
=> dashboard version of my Leuven WU table/report script using the old .CSV format, only uses yearly files
=> wureports/wudata/ cache folder to speed up the data retrieval of yearly files
==> the cache folder can be emptied if the main wudata/ folder with the .arr afiles is updated.

Hope this helps,

Wim