Pinging @slevis-lmwg I put this as something to do as part of ctsm5.3.0, but we could do it in a post ctsm5.3.0 tag. It's easy to do, but involves enough steps that it could be done later.

I'll let others evaluate the timing here, but I worry about scope creep and further delays for making a CTSM5.3.0 and CESM_beta03 tags

Towards that end, I don't know this is show stopper for the scientific applications we need to meet for the 5.3 milestone or the "Aug" code chill, but we can discuss here or on Thurs.

I asked @jtruesdal offline and needs to have access to the files right away, but doesn't need it in a tag right away. So here's the plan

• We'll make the files with a branch tag when Derecho comes back up
• We'll add this work to b4b-dev and have it come in on the b4b-dev cycle...

Steps in ctsm5.3.0 with the mksurfdata_esmf tool for generating the 1850 file (similarly for 2000). Includes correction discussed below:

./gen_mksurfdata_build 
./gen_mksurfdata_namelist --start-year 1850 --end-year 1850 --res ne3np4 --model-mesh /glade/campaign/cesm/cesmdata/inputdata/share/meshes/ne3np4_ESMFmesh_c230714_cdf5.nc --model-mesh-nx 488 --model-mesh-ny 1
./gen_mksurfdata_jobscript_single --namelist-file surfdata_ne3np4_hist_1850_78pfts_c240925.namelist --number-of-nodes 1
qsub mksurfdata_jobscript_single.sh 

For Hist and SSP2-4.5:

./gen_mksurfdata_namelist --start-year 1850 --end-year 2100 --res ne3np4 --model-mesh /glade/campaign/cesm/cesmdata/inputdata/share/meshes/ne3np4_ESMFmesh_c230714_cdf5.nc --model-mesh-nx 488 --model-mesh-ny 1 --ssp-rcp SSP2-4.5
./gen_mksurfdata_jobscript_single --namelist-file surfdata_ne3np4_SSP2-4.5_1850_78pfts_c240925.namelist --number-of-nodes 1
qsub mksurfdata_jobscript_single_Hist.sh

Jobs in the queue on derecho. Let's see if they work on the first attempt...

You're welcome. Just to be clear though, I submitted these jobs, so you don't need to repeat the work :-)

@jtruesdal you will find the new files in
/glade/derecho/scratch/slevis/temp_work/new_rawdata/tools/mksurfdata_esmf

surfdata_ne3np4_hist_1850_78pfts_c240925.nc
surfdata_ne3np4_hist_2000_78pfts_c240925.nc

Feel free to test.

The landuse file (for transient simulations) is not ready. It ran out of wallclock with 1 node, so I gave it a second node and resubmitted.

John, the landuse file should be generated by this evening.

Started testing on derecho

• Addition to the Makefile

make crop-global-SSP2-4.5-ne3
make crop-global-1850-low-res
make crop-global-present-low-res
make global-present-low-res  (unnecessary)

• ./build-namelist_test.pl
• python testing
• clm_pymods

Opening the PR next and will post updates there.

@ekluzek let's discuss the last checkbox, due to my lack of familiarity with adding SCAM tests to our test-suites.

From meeting with @ekluzek
In testlist, change ptsRLA and ptsRLO tests to ne3np4.

@jtruesdal I wonder whether you have used the above ne3np4 fsurdat file(s) successfully. If so, could you point me to a case that worked that I could look at?

I've only tested these with CAM3d previously and they work but when I run them in SCAM it doesn't work. The format looks strange, they are using an older? lsmlon, lsmlat dimension for the variables where as everything up to this point always used gridcell for the unstructured grid formats.

Interesting. I suspect that this is a shortcoming of our mksurfdata_esmf tool. What do you think @ekluzek?

@ekluzek and I looked into this and eventually discovered user error on my part in a line that I listed above:
./gen_mksurfdata_namelist --start-year 1850 --end-year 1850 --res ne3np4 --model-mesh /glade/campaign/cesm/cesmdata/inputdata/share/meshes/ne3np4_ESMFmesh_c230714_cdf5.nc --model-mesh-nx 1 --model-mesh-ny 488
should have been
./gen_mksurfdata_namelist --start-year 1850 --end-year 1850 --res ne3np4 --model-mesh /glade/campaign/cesm/cesmdata/inputdata/share/meshes/ne3np4_ESMFmesh_c230714_cdf5.nc --model-mesh-nx 488 --model-mesh-ny 1

I will open an issue to alert users about this pitfall and to brainstorm how to remove the pitfall.

@jtruesdal I generated new files. I gave them identical names as before:

-rw-r--r-- 1 slevis cseg 229712212 Oct 28 13:35 ../../lnd/clm2/surfdata_esmf/ctsm5.3.0/landuse.timeseries_ne3np4_SSP2-4.5_1850-2100_78pfts_c240926.nc
-rw-r--r-- 1 slevis cseg  17072072 Oct 28 13:37 ../../lnd/clm2/surfdata_esmf/ctsm5.3.0/surfdata_ne3np4_hist_1850_78pfts_c240925.nc
-rw-r--r-- 1 slevis cseg  17072072 Oct 28 13:40 ../../lnd/clm2/surfdata_esmf/ctsm5.3.0/surfdata_ne3np4_hist_2000_78pfts_c240925.nc

Our CTSM SCAM test
SMS_D_Ld1_Mmpi-serial.ne3_ne3_mg37.I2000Clm50SpRs.derecho_intel.clm-ptsRLA
fails with a new error, so I wanted to ask you this:
@ekluzek and I were comparing the two ne3 mesh files, and Erik found it strange that the np4 file has maxNodePElement = 5. Do you have thoughts about that?

1c1
< netcdf ne3np4_ESMFmesh_c230714_cdf5 {
---
> netcdf ne3pg3_ESMFmesh_c221214_cdf5 {
3,5c3,5
<       nodeCount = 510 ;
<       elementCount = 488 ;
<       maxNodePElement = 5 ;
---
>       nodeCount = 488 ;
>       elementCount = 486 ;
>       maxNodePElement = 4 ;

UPDATE

I looked at the equivalent ne30 mesh files, and they differ in maxNodePElement in the same way, so probably the maxNodePElement = 5 is fine.

I submitted the same ptsRLA test with ne30_g17, ne30pg3_t232, and ne3pg3_ne3pg3_mg37 and they all failed, while with f45_f45_mg37 the test works.

...the error seems to always point to urban code.

Thank you, @jtruesdal. Let me know where to get the mods from. Unless you prefer to push them to my PR (#2835) yourself. If so, let me know, and I will give you appropriate permissions.

I'm hoping to get to the BFB test today and will also run the CAM regression suite to make sure I haven't broken anything. I'll hand off the mod when that is complete (Thur or Fri). Can I make a PR against the branch listed in #2835 and then you could just merge them in?

Yes, that'll work.

I made a PR to your branch with the additional mods to get SCAM running. Could you try them out and see if they work for you.

@slevis-lmwg is there anything for me to do here -- other than review? If not I'll remove myself as assignee.

I made a PR to your branch with the additional mods to get SCAM running. Could you try them out and see if they work for you.

Thank you, @jtruesdal. I merged your branch to mine and pushed to the PR. I will test when derecho comes back.

I'm marking my "sprint" card DONE here and opening a new card in the PR #2835 for the remaining work.

I'm not marking the issue itself DONE until I merge the PR.