MET12.0.0 - plot_data_plane

[odiaconu]

HI again,
I recently installed Met12.0.0 on a Rocky Linux 8.6 cluster. I tried plot_data_plane on grib and netcdf files. Some of the files require transformation of the parameters into units corresponding to the observations. I did this with "cdo", a utility that works well. After processing, plot_data_plane no longer accepts the resulting file. Met10.1.2 and 8.1.2 accept the file and plot it but Met12 not. I, am attaching the archive with two files, _R is the processed post.
.....
fic.tar.gz

FATAL ERROR (SEGFAULT): Process 1797900 got signal 11 @ local time = 2025-01-13 16:39:14Z
FATAL ERROR (SEGFAULT): Look for a core file in /home/oper/MET_AIFS/DATE/ERA5_R/03/2t
FATAL ERROR (SEGFAULT): Process command line: plot_data_plane 2t_20240331_1800.grb 2t_20240331_1800.ps name="2T"; level="Z0";
Segmentation fault (core dumped)

[jprestop]

HI @odiaconu. I see you are having trouble with MET-12.0.0's plot_data_plane on a Rocky Linux 8.6 cluster. Thank you for attaching your files.

I installed MET-12.0.0 on a Rocky Linux 8.10 (Green Obsidian) system this morning and successfully ran the command from your error:

FATAL ERROR (SEGFAULT): Process command line: plot_data_plane 2t_20240331_1800.grb 2t_20240331_1800.ps name="2T"; level="Z0";

See the output below and version information below:

[Julie.Prestopnik@fe1 2860]$ plot_data_plane 2t_20240331_1800.grb 2t_20240331_1800.ps 'name="2T"; level="Z0";'
DEBUG 1: Start plot_data_plane by Julie.Prestopnik(10845) at 2025-01-14 13:42:29Z  cmd: plot_data_plane 2t_20240331_1800.grb 2t_20240331_1800.ps name="2T"; level="Z0";
DEBUG 1: Opening data file: 2t_20240331_1800.grb
DEBUG 1: Creating postscript file: 2t_20240331_1800.ps
DEBUG 1: Finish plot_data_plane by Julie.Prestopnik(10845) at 2025-01-14 13:42:29Z

[Julie.Prestopnik@fe1 2860]$ plot_data_plane --version
DEBUG 1: Start plot_data_plane by Julie.Prestopnik(10845) at 2025-01-14 13:42:54Z  cmd: plot_data_plane --version

MET Version: V12.0.0
Repository: https://github.com/dtcenter/MET
Revision: development
Change Date: development 

Perhaps there was a problem with your installation? If you'd like us to take a look at the compilation filees, the MET-12.0.0 bulild directory should contain the following files: config.log, met.make.log, and met.make_test.log. Could you please attach those files here and I'll see if I can figure out what's going on? Could you also please pass along the core dump file from /home/oper/MET_AIFS/DATE/ERA5_R/03/2t?

Thanks!

[odiaconu]

fep:/home/oper/MET_AIFS/DATE/ERA5_R.amasc/03/2t=>plot_data_plane nam.t00z.awip1236.tm00.20070330.grb nam_TMP_Z2.ps 'name="TMP"; level="Z2";'
DEBUG 1: Start plot_data_plane by oper(1001) at 2025-01-14 20:02:22Z cmd: plot_data_plane nam.t00z.awip1236.tm00.20070330.grb nam_TMP_Z2.ps name="TMP"; level="Z2";
DEBUG 1: Opening data file: nam.t00z.awip1236.tm00.20070330.grb
DEBUG 1: Creating postscript file: nam_TMP_Z2.ps
DEBUG 1: Finish plot_data_plane by oper(1001) at 2025-01-14 20:02:23Z
is OK
But ..
pcp_combine -add 2t_20240331_1800.grb 'name="2T"; level="Z0";' met_2t_20240331_1800.nc
DEBUG 1: Start pcp_combine by oper(1001) at 2025-01-14 20:03:30Z cmd: pcp_combine -add 2t_20240331_1800.grb name="2T"; level="Z0"; met_2t_20240331_1800.nc
DEBUG 2: Since the "-field" command line option was not used, parsing the command line arguments a list of files, each followed by a configuration string.
DEBUG 2: Performing derivation command (sum) for 1 files.
DEBUG 1: Reading data (name="2T"; level="Z0";) from input file: 2t_20240331_1800.grb
FATAL ERROR (SEGFAULT): Process 2229403 got signal 11 @ local time = 2025-01-14 20:03:30Z
FATAL ERROR (SEGFAULT): Look for a core file in /home/oper/MET_AIFS/DATE/ERA5_R.amasc/03/2t
FATAL ERROR (SEGFAULT): Process command line: pcp_combine -add 2t_20240331_1800.grb name="2T"; level="Z0"; met_2t_20240331_1800.nc
Segmentation fault (core dumped)

[JohnHalleyGotway]

Hmm, this sure is a tough one to debug without being able to replicate the problem ourselves. I feel like I'm grasping at straws. If I could run this through a debugger, I could probably spot where things are going sideways.

The support for GRIB1 files is provided directly in MET's internal vx_data2d_grib library. GRIB2 support is provided by linking to the external grib2c library, and we've certainly had problems when MET and the grib2c library are compiled with conflicting options. But that's NOT what's going on here.

It is true that environment variables can impact MET's runtime behavior. While I doubt that's the culprit, you could run printenv | grep MET so we can look for anything suspicious.

So it segfaults on your machine and runs fine on our dev machine. Figured I'd check another environment. So I ran the following commands to process this in a Docker container:

docker run --rm -v `pwd`/data:/data dtcenter/met:12.0.0 plot_data_plane /data/2t_20240331_1800.grb /data/2t_20240331_1800.ps 'name="2T"; level="Z0";'

And that ran fine to produce this graphic.
Any suggestions on how/where we might be able to replicate the failure you're getting? I'm out of ideas.

[odiaconu]

I,ve send 2 files 2t_20240331_1800.grb and 2t_20240331_1800_R.grb the second one is produced using cdo
cdo subc,273.15 2t_20240331_1800.grb 2t_20240331_1800_R.grb (cdo does not modify the metadata.. oly substract a constant from field )
I can plot 2t_20240331_1800.grb but NOT 2t_20240331_1800_R.grb
If I appply: grib_set -s edition=2 2t_20240331_1800_R.grb 2t_20240331_1800_R.grb2.. I can plot 2t_20240331_1800_R.grb2. So, my pb is solved .. ! I wonder why cdo doesn't produce a correct file? I tried cdo on 3 workstations, 3 versions of cdo.. same result.. So, have you tried to plot same file 2t_20240331_1800_R.grb ? I saw the png and it seems to be a plot from 2t_20240331_1800.grb - (2t in K not in C). Many thanks for yours efforts !

[JohnHalleyGotway]

Oh, I was confused. I thought the problem was with the 2t_20240331_1800.grb file, not the 2t_20240331_1800_R.grb file writen by cdo. Yes, I can replicate that segfault via Docker

docker run --rm -v `pwd`/data:/data dtcenter/met:12.0.0 \
plot_data_plane /data/2t_20240331_1800_R.grb /data/2t_20240331_1800.ps \
'name="2T"; level="Z0";'

DEBUG 1: Start plot_data_plane by root(0) at 2025-01-14 22:05:24Z  cmd: plot_data_plane /data/2t_20240331_1800_R.grb /data/2t_20240331_1800.ps name="2T"; level="Z0"; 
DEBUG 1: Opening data file: /data/2t_20240331_1800_R.grb
FATAL ERROR (SEGFAULT): Process 1 got signal 11 @ local time = 2025-01-14 22:05:24Z
FATAL ERROR (SEGFAULT): Look for a core file in /met
FATAL ERROR (SEGFAULT): Process command line: plot_data_plane /data/2t_20240331_1800_R.grb /data/2t_20240331_1800.ps name="2T"; level="Z0"; 

Frankly, I doubt this is worth debugging further, because you can handle that conversion directly in MET using the convert(x) (search for "convert" in the docs) configuration option. Doing that unit conversion when you read the data is probably better than having to run cdo first. Either convert(x) = x - 273.15; or convert(x) = K_to_C(x); would work. A handful of standard unit conversion functions are provided in this ConfigConstants file.

docker run --rm -v `pwd`/data:/data dtcenter/met:12.0.0 \
plot_data_plane /data/2t_20240331_1800.grb /data/2t_20240331_1800_celsius.ps \
'name="2T"; level="Z0"; convert(x) = K_to_C(x);' -v 4

And running at verbosity level 4 (-v 4) confirms the range of values now match what you'd expect for Celsius:

DEBUG 4: Data plane information:
DEBUG 4:    plane min: -30.1416
DEBUG 4:    plane max: 38.7315

And you can see the same in the range of values in the colorscale of the plot.

[odiaconu]

Thank you. Yes, it is possible that cdo produce a broken grib file. We use cdo extensively in our activity because it offers many many functions, it is installed on all computers. It's the first time I've come across something like this. Thank you very much for your time and advice. Yes, there is no need to debug; Best regards

[JohnHalleyGotway]

Great. Thanks for following up. I'll go ahead and select and "Answer" for this discussion and close it.