GFS maps creation with OpenGrads - does it work?

Ask a Question
, , ,
1

Hi, Does anyone have a working installation of the GFS maps creation with OpenGrads running? I used to have it running with the nordic.gs file + bat file original created from Henkka (weatherc) … I havent used it for some years because of a computer change - but wants it up an run again.

But now I cant get it to work. I have installed the newest version of OpenGrads and a version of the Nordic.gs + bat file that I have been able to find her. But I get this error from OpenGrads

Config: v2.2.1.oga.1 little-endian readline grib2 netcdf hdf4-sds hdf5 opendap-grids,stn athena geotiff shapefile Issue 'q config' and 'q gxconfig' commands for more detailed configuration information Loading User Defined Extensions table ... ok. GX Package Initialization: Size = 11 8.5 Running in Batch mode Command line history in \Users\soren/.grads.log All files closed; all defined objects released; All GrADS attributes have been reinitialized OC xxdr depends on the assumption that sizeof(off_t) == sizeof(void*) syntax error, unexpected WORD_STRING, expecting WORD_WORD context: Error { code = 0; message = "/gfs_0p25/gfs20200418/gfs_0p25_00z.das is not an available dataset"^;}; Error: nc_open failed to open file https://nomads.ncep.noaa.gov:9090/dods/gfs_0p25/gfs20200418/gfs_0p25_00z.das NetCDF: Malformed or inaccessible DAP DDS gadsdf: Couldn't ingest SDF metadata. No Files Open

My question is: Does the nordic.gs script work anymore? does the bat file work ? (both attached in .txt version)

Does this work at all? All help and input appreciated.

Thanks, Soren

gfs_bat.txt (431 Bytes)

nordic_gs.txt (37.5 KB)

2

I’ve not used the scripts you’ve got but if it’s some years since you’ve used them it’s possible that the location of the GFS files, and possibly the access mechanism, has changed in that time. I use GFS data for other reasons and I can remember two occasions in the last few years where I’ve had to check/change my scripts to fit in with NWS file location changes.

3

Hi Chris,
Thanks for replying… i’ve updated the paths and urls to gfs maps - so they should be ok. But script does not do its job… and reports the errors I put in ny first post. So must be some code in the script that has to be changed - and thats not my expertise :wink:

4

I think the script is about 9-10 years old so possibly written for OpenGrads v1 which might account for the differences.

I’m no expert in OpenGrads although I have dabbled with it a little over the years. One problem I can see is that the ‘sdfopen’ line is trying to download a .das file. I’ve only seen sdfopen used to open .dds files and if you remove the .das you get further, but still end up with hundreds of errors once the data processing starts.

5
  1. I changed the path to the GFS 0.25 1 hour in the script.
  2. In places corrected inaccuracies that I understood
  3. Much has worked, but not all.

:~/site/karelia.weather/opengrads/almukantarat$ ./test.sh 

              Welcome to the OpenGrADS Bundle Distribution
              --------------------------------------------

For additional information enter "opengrads --manual".

Starting "/opt/opengrads/Linux/Versions/2.2.1.oga.1/x86_64/grads -HC 1  -lxc 'nordic.gs 20200418 00' " ...


Grid Analysis and Display System (GrADS) Version 2.2.1.oga.1
Copyright (C) 1988-2018 by George Mason University
GrADS comes with ABSOLUTELY NO WARRANTY
See file COPYRIGHT for more information

Config: v2.2.1.oga.1 little-endian readline grib2 netcdf hdf4-sds hdf5 opendap-grids,stn athena geotiff shapefile
Issue 'q config' and 'q gxconfig' commands for more detailed configuration information
Loading User Defined Extensions table </opt/opengrads/Linux/Versions/2.2.1.oga.1/x86_64/gex/udxt> ... ok.
GX Package Initialization: Size = 11 8.5 
Command line history in /home/gasha/.grads.log
All files closed; all defined objects released;
All GrADS attributes have been reinitialized
PRECIP
Cannot contour grid - all undefined values 
Cannot contour grid - all undefined values 
Cannot plot color bar: No shading information
CLOUD
Cannot contour grid - all undefined values 
Cannot plot color bar: No shading information
4LI
Warning: Contour levels are not strictly increasing
         This may lead to errors or undesired results
Warning: Contour levels are not strictly increasing
         This may lead to errors or undesired results
Cannot plot color bar: No shading information
CAPE
SF CINH
STORMINDEX
DRAW error: Syntax is DRAW STRING x y string
INSTABILITY
LOW LEVEL SHEAR
DEEP LEVEL SHEAR
SRH

nordic_reloaded.gs.txt (38.6 KB)

6

I can fix this one for you. acpcpsfc and apcpsfc don’t exist in the 000h grid. So you could fix this by changing…


say PRECIP
  i = 1
  while ( i<=maps )

…to…


say PRECIP
  i = 2
  while ( i<=maps )

The script only runs for one map though (‘maps = 1’ higher in the script) so you won’t get a precipitation image created.

However, I believe there’s a much bigger issue at play.

The script is running against data from a 1 hour model. The way the GFS model is processed means that the 00z model run starts at 00z (add 6, 12 and 18 hours for the other runs). The time to complete the processing and upload the files so that you can access them is about 4 hours (it varies a little from day to day). For example, this morning the 00z model data became available at 04:02UTC. With the script running with ‘maps = 1’ it would be processing the first grid file, i.e. 000h, which represents the conditions at 00z (I think that’s actually the model input data), so even if you ran the script immediately the data became available you’d be creating a forecast image for 4 hours in the past (a backcast?)

A quick and dirty fix for this is for the script to run with ‘maps = 5’ and then change all the ‘i = 1’ values through the script to ‘i = 5’. Your first image created would then be for 05z which might be just in the future.

A more elegant solution would be to calculate the current time (UTC) in the script and create a new variable ‘start_hour’ set current hour+1. Then for each image creation section in the script change ‘i = 1’ to 'i = 'start_hour. You would then get ‘maps’ number of hourly images starting at ‘start_hour’. This would probably need a separate counter creating for the image name. They’re created as ‘gfs_’.type.i, e.g. gfs_prec1.png. Assuming these are referenced on a web page having a random start count depending on when the script ran could make the web page more difficult to create. This elegant solution is a bit beyond my brain at 09:00 on a Sunday morning, but if I find time later I might try to re-engineer the script.

Note…none of this fixes the other reported errors, although it’s possible that like PRECIP some of the variables don’t exist in the 000h grid so the solutions above may fix those too.

7

After I gave my brain some food and drink I made some more progress.

The attached file runs far more successfully (using OpenGrads version 2.2.1.oga.1 on Linux). There is one significant difference if you’re running it from a command/bat file or script. It now requires 3 arguments passing to it. The first two are as normal - date, e.g. 20200419 and run hour, e.g. 00. The third parameter is the starting forcast hour. So to start creating images from 5 hours after the run start time, i.e. 05z for the 00z run, you would enter 5. The number is always the hours after the run start, so for the 12z run, ‘5’ would create the first image using the 17:00z forecast data.

There are still a couple of warnings in the STORMINDEX section.


STORMINDEX
Warning from LOG:  Data has 101931 values <= zero
                   These were set to the undefined value
Warning from LOG:  Data has 101815 values <= zero
                   These were set to the undefined value

I don’t know if these are significant. The image still gets created but I’m not sure if some data hasn’t been plotted as a result of these warnings.

Hopefully this works for other people too.

nordic2.txt (37.9 KB)

8

Hi Chris and almukantarat,

Thank you for your effort and advices … I will test is later today… I’m outside enjoying the beautiful spring weather in Denmark - although a bit chilly

cheers, soren

9

It’s sunny with blue skies here. I sat in the greenhouse for over an hour but had to come in because it was too hot!

10

Just to clarify… I have to change the BAT file I use… correct?

BAT file

@ECHO OFF

set FileDate=%date:~-4,4%%date:~-7,2%%date:~0,2%

set FileTime=%time:~0,2%
if /i %FileTime% gtr 00 (
set T=12
set /a FD=%FileDate% - 1
)
if /i %FileTime% gtr 01 (
set T=18
set /a FD=%FileDate% - 1
)
if /i %FileTime% gtr 07 (
set T=00
set FD=%FileDate%
)
if /i %FileTime% gtr 13 (set T=06)
if /i %FileTime% gtr 19 (set T=12)

@echo Year = %FD%
@echo T = %T%

opengrads -lbxc ‘nordic.gs %FD% %T%’

which part do I need to change?

11

You need to add another value onto this line…

opengrads -lbxc 'nordic.gs %FD% %T%'

You could hard code it for now to check if it works. Change it to something like…

opengrads -lbxc 'nordic.gs %FD% %T% 5'

That should start to generate files from 05z/11z/17z/23z (depending on the GFS run you’re processing)

12

:smiley: Just did a quick test - and it works. I will check further - this help was more than I hope for… much appreciated. When testing is done - I will post my scripts it any one else want to try this.

cheers, soren

13

I’m glad it works for you. Thanks to almukantarat for getting me started off. I actually started looking at OpenGrads a couple of days ago for something else so this has proved to be a very useful exercise to teach me more about how it works.

Thanks also to almukantarat for posting his location and encouraging me to find out that there’s a place called Karelia. It looks very nice, but I expect it’s pretty cold in winter!

14

I’m part way through creating a newer version of this script. The versions currently posted contain code that’s never used and also is rather ineffcient in having to code the text block on every graph in each graph. I’ve tested the mods so far but I also want to try to make it more general, i.e. allow people to make a couple of minor edits to make it work for their location.

One thing I wonder…the current script creates images in northern stereographic format which works for the Nordic area it was originally created for but might not be as useful more generally. I was thinking of changing the script to latlon or scaled which gives a view where the lines of lat and long intersect at 90 degrees. This makes it far easier to predict how a map will fill the image pane. No map projection is perfect unless you have a spherical monitor (!) but what do people think about switching to latlon/scaled?

15

Thanks for the work you’ve done. And thanks for the kind words about Karelia. In fact, the winters have long since deteriorated. There were no frosts for a long time.

16

No frosts. Wow! You’re another 10 degrees north than me and whilst it’s been a warmer winter here we’ve still had some frost. We don’t get much snow though, the Gulf Stream prevents that because we’re on the coast. We get heavy snow perhaps every 15-20 years.

17

September, 2019 Temperature Norma = +9.5 Fact = +9.6 Delta = +0.1 High = +23.5 Low = -0.8
October, 2019 Temperature Norma = +3.8 Fact = +2.8 Delta = -1.0 High = +11.6 Low = -7.2
November, 2019 Temperature Norma = -2.8 Fact = -1.2 Delta = +1.6 High = +7.6 Low = -12.5
December, 2019 Temperature Norma = -7.0 Fact = -1.1 Delta = +5.9 High = +3.9 Low = -12.1
January, 2020 Temperature Norma = -9.3 Fact = -2.0 Delta = +7.3 High = +5.5 Low = -17.1
February, 2020 Temperature Norma = -9.1 Fact = -2.7 Delta = +4.9 High = +5.5 Low = -15.2
March, 2020 Temperature Norma = -3.7 Fact = -0.1 Delta = +3.6 High = +14.9 Low = -11.1
April, 2020 Temperature Norma = +2.3 Fact = -1.4 Delta = -0.9 High = +13.2 Low = -6.9

I remember winter 1987, in January was t -46!!!

18

A low of -17.1C would be seriously cold here! We think it’s bad if it gets down to -10C. Coldest I’ve ever experienced was -23C in Idaho so I can’t really imagine -46C!

19

Sorry! -41.6 C. Петрозаводск

The coldest places in Russia are Оймякон map and Верхоянск map.

Okay, that everything is offtopic, back to the topic.

20

OK I’ve created v3 which I think is pretty generalised and can be used by others by making a few changes to the file without needing to worry about the rest of the script. The file (nordic3.txt) is attached as is an example of one of the graphs created using it. You should rename the nordic3.txt file to nordic3.gs, although it’s possible that OpenGrads will run a script from a .txt file.

The script has three arguments passed into it:

  1. The model run date you want to process in the format YYYYMMDD, e.g. 20200421
  2. The model run hour you want to process. This can be 00, 06, 12 or 18
  3. The start hour that you want to create images for as a number, e.g. 5. It’s probably not worth going below 5 because you’d probably be creating forecast images for the past

You can either pass the arguments on teh command line, e.g. ‘grads -lcbx nordic3.gs 20200441 06 5’ or if not you’ll be promted to enter the values when you run the script.

The configurable parts in the script are:


* How many hourly maps to generate
maps = 2

As the data set is hourly data, this setting would create two images, one for the start hour (see above) and a second for start hour + 1. You can create as many images as you like (obviously within the number of hours in the grid data.)


* Map copyright text
_map_copyright = 'Map (c) My WX-server/mywxdomain.com'

Set the copyright text you want to display in the bottom right hand corner of the map.


* Map Projection - See http://cola.gmu.edu/grads/gadoc/gradcomdsetmproj.html
_map_proj = 'scaled'

* Reference lon/lat for polar stereographic plots - See http://cola.gmu.edu/grads/gadoc/gradcomdsetmpvals.html
* Not used for latlon/scaled
_map_vals = '15 52 54 71'

* Map lon/lat to plot - See http://cola.gmu.edu/grads/gadoc/gradcomdsetlatlonlevtimeens.html
_map_lon = '-25 25'
_map_lat = '35 65'

This is where you might need to experiment to get a map covering the area you want in the format you want. The URLs take you to pages describing the options. If you choose nps or sps projection you’ll need to set _map_vals. This is the area to display. You’ll need _map_lon and _map_lat covering a much bigger area to get a square area to cut out for your map. Experiment until you get what you want.


* Define which plots are required
plot_t_2m = 0
plot_t_sfc = 0
plot_lcl = 0
plot_wc = 0

These are the first 4 of 20 settings which allow you to choose which maps you want to generate. If you want a map for ‘2m Temperature’ set ‘plot_t_2m = 1’. ‘0’ means the map won’t plot. Some of the maps are very specialist so you probably want to pick and choose which ones you want. Leave the last of the 20 (plot_srh_1km) set to 0. You would need to change the script to make this work. I think the unerlying GFS data has changed which means this doesn’t work.

Note: If you see a warning like the one below it can be ignored (I think!).


STORMINDEX - ENABLED
Warning from LOG:  Data has 4067 values <= zero
                   These were set to the undefined value

Feel free to modify the script in any way you choose. It should work simply as it is with just a few minor changes, or if you want to you can bash it about and make it do all sort of other things too!

nordic3.JPG
nordic3.JPG1345×1006 307 KB

nordic3.txt (26.1 KB)