CTSM PPE prep work

[wwieder]

We want to merge PPE branch back to master

• Make a PPE tag with the code mods from @linniahawkins
• Make a PR with the code mods update
• Make a PR with the previous n11 and n12
• Check and make sure everything on PPE is now on master

[wwieder]

@ekluzek thought this could go on b4b-dev?

[slevis-lmwg]

@slevis-lmwg will address #1652.

[slevis-lmwg]

I'm working through the TODOs in this card
and specifically running the aux_clm test-suite:

• derecho OK though many more than usual TPUTCOMP Error: Computation time increase > 25 pct from baseline
• izumi FAIL: The baseline may have been generated using master rather than the branch because I see NLCOMP and BASELINE failures in many (all?) tests. I will let it finish generating the new baseline which I hope is the correct one. Then I will try the following on izumi only (@ekluzek your feedback is welcome here):

git checkout -b branch_tags/PPE.n14_ctsm5.1.dev030 ekluzek/branch_tags/PPE.n14_ctsm5.1.dev030
./run_sys_tests -s aux_clm -g ctsm5.1.dev030.PPE14_NEW -c ctsm5.1.dev030.PPE15

[slevis-lmwg]

@ekluzek the git checkout idea didn't work but maybe you can point me to a commit from this list that would represent ctsm5.1.dev030.PPE14?

[slevis-lmwg]

@ekluzek and I resolved the above:
The tag I made is a duplicate of n14.

• I should push it to escomp (because I pushed it to Erik's fork): DONE assuming I did it correctly.
• The izumi baseline for n14 was problematic, so we removed it.
• I have now copied the code from Linnia's directory, changed namelist_defaults_ctsm.xml to point to a local params file
  ~slevis/paramfiles/ctsm51_params.c210507_kwo.c220322.nc
  and started aux_clm again:

derecho OK with caveats

• I see more TPUTCOMP increases (weird?) and also expected NLCOMP diffs.
• Rather than updating the custom param file clm45_params.soilCN30.c210614.nc temporarily, I removed it from the test
  SMS_Lm1.f10_f10_mg37.I1850Clm45Bgc.derecho_intel.clm-clm45_monthly_matrixcn_soilCN30 to confirm that the test would pass with the new default PPE param file. As a result I got different answers from the ctsm5.1.dev030.PPE15 baseline.

izumi OK

[ekluzek]

This was finished and brought to master in ctsm5.2.027

[olyson]

Reviewing the original PPE spinup and historical simulation in preparation for a new spinup (NCAR/LMWG_dev#70), I'm seeing two code modifications that are not appearing in ctsm5.2.028. These were in the last PPE branch tag (branch_tags/PPE.n16_ctsm5.1.dev030) in CanopyFluxesMod.F90 and CNVegStructUpdateMod.F90, but are not in ctsm5.2.028.

[olyson]

Looking closer in the latest CNVegStructUpdateMod.F90, it looks like maybe the changes were implemented in a different manner than in the last PPE tag. The only difference might be that leaf_biomass and stem_biomass are set to zero under certain conditions in the last PPE tag, while in the latest code they are not. But maybe that was deemed not necessary.
In CanopyFluxesMod.F90, I don't see this line that appears in the last PPE tag but not in the latest tag:

sa_leaf(p) = sa_leaf(p) + esai(p)

I can do some testing when I'm back in the office next week.

[olyson]

I tested the differences in CNVegStructUpdateMod.F90. They affect simulations with biomass heat storage (bhs) both on and off, so I tested both in 5 year I2000 cases:

bhs off:
/glade/work/oleson/ctsm5.3.n04_ctsm5.2.028/cime/scripts/ctsm53n04ctsm52028_f09_bhsoff_2000
/glade/work/oleson/ctsm5.3.n04_ctsm5.2.028/cime/scripts/ctsm53n04ctsm52028_f09_VegStruct_bhsoff_2000
bhs on:
/glade/work/oleson/ctsm5.3.n04_ctsm5.2.028/cime/scripts/ctsm53n04ctsm52028_f09_2000
/glade/work/oleson/ctsm5.3.n04_ctsm5.2.028/cime/scripts/ctsm53n04ctsm52028_f09_VegStruct_2000

The bhs off and bhs on simulation pairs were bfb, so no new code modifications needed.

I did a similar pair of I2000 cases (bhs on) with the sa_leaf change. I'll post in #2777 about those results.