[1pt] PR: Rating curves adjustments with ml-bathymetry data and optimized mannN

docs/CHANGELOG.md

@@ -1,7 +1,35 @@
 All notable changes to this project will be documented in this file.
 We follow the [Semantic Versioning 2.0.0](http://semver.org/) format.
 
+## vx.x.x.x - 2025-01-08 - [PR#1340](https://github.com/NOAA-OWP/inundation-mapping/pull/1340)
 
+This PR focuses on adjusting rating curves by using bathymetric data and optimized channel roughness values. The bathymetry data includes eHydro surveys and AI-based datasets created for all NWM streams. New manning roughness values were developed for each feature-id using a differential evolution objective function (OF). The OF minimizes the number of the false_positives and false_negatives cells in our flood inundation maps where we have test cases across the CONUS. 
+
+Even though the Python scripts of roughness manning number optimization were not included in this branch, optimized roughness values can be found here: `/fim-data/inputs/rating_curve/variable_roughness/mannings_optz_fe_clusters_so3.csv`. Detailed python scripts also can be found here: `/fim-data/outputs/heidi-mannN-optimization/projects/bathy_mannN_projects/dev-bathymetric-adjustment-mannN-optz/`.
+
+### Changes
+- `src/bathymetric-adjustment.py`: `correct_rating_for_ai_based_bathymetry` function was added to the script. This function processes AI-based bathymetry data and adjusts rating curves using this data. Also `apply_src_adjustment_for_bathymetry` function was added to prioritize USACE eHydro over AI-based bathymetry dataset. The multi-processing function `multi_process_hucs` was updated based on the latest code. Also, an ai_toggle parameter was added to `apply_src_adjustment_for_bathymetry` and `process_bathy_adjustment` functions. When ai_toggle = 1, The SRCs will be adjusted with the ai_based bathymetry data. the default value for ai_toggle = 0, means no ai_based bathy data is included. 
+
+- `src/bash_variables.env`: New variables and their paths were added. Also, a new input file with the nwm feature_ids and optimized channel roughness and overbank roughness attributes was created and stored here:
+`/fim-data/inputs/rating_curve/variable_roughness/mannings_optz_fe_clusters_so3.csv`
+The locations of these files were also added to the `bash_variables.env`.
+Please note that when ai_toggle = 1, the manning roughness values should be switched to `vmann_input_file=${inputsDir}/rating_curve/variable_roughness/mannings_optz_fe_clusters_so3.csv` in the current version. 
+
+Here is a list of new/updated input files:
+
+1. `/fim-data/inputs/rating_curve/variable_roughness/mannings_optz_fe_clusters_so3.csv`
+This CSV file contains the new optimized roughness values. It will replace this file:
+`vmann_input_file=${inputsDir}/rating_curve/variable_roughness/mannings_global_nwm3.csv`
+
+2. `bathy_file_aibased=${inputsDir}/bathymetry/ml_outputs_v1.01.parquet`
+This file contains the ml-bathymetry and manning roughness values data.
+
+3. `bathy_file_ehydro=${inputsDir}/bathymetry/final_bathymetry_ehydro.gpkg`
+We already had this file, the name of the variable has changed from `bathymetry_file` to `bathy_file_ehydro`, and it was updated.
+
+- `fim_post_processing.sh`: New arguments were added. Please note that the default value for ai_toggle = 0 is included here. 
+
+<br/><br/>
 
 ## v4.5.13.4 - 2024-01-03 - [PR#1382](https://github.com/NOAA-OWP/inundation-mapping/pull/1382)
 

fim_post_processing.sh

@@ -161,15 +161,18 @@ Tcount
 
 ## RUN BATHYMETRY ADJUSTMENT ROUTINE ##
 if [ "$bathymetry_adjust" = "True" ]; then
-    l_echo $startDiv"Performing Bathymetry Adjustment routine"
-    Tstart
+    echo -e $startDiv"Performing Bathymetry Adjustment routine"
     # Run bathymetry adjustment routine
+    ai_toggle=${ai_toggle:-0}
+    Tstart
     python3 $srcDir/bathymetric_adjustment.py \
         -fim_dir $outputDestDir \
-        -bathy $bathymetry_file \
+        -bathy_ehydro $bathy_file_ehydro \
+        -bathy_aibased $bathy_file_aibased \
         -buffer $wbd_buffer \
         -wbd $inputsDir/wbd/WBD_National_EPSG_5070_WBDHU8_clip_dem_domain.gpkg \
-        -j $jobLimit
+        -j $jobLimit \
+        -ait $ai_toggle
     Tcount
 fi
 

src/bash_variables.env

@@ -31,14 +31,17 @@ export                  input_nwm_lakes_Alaska=${inputsDir}/nwm_hydrofabric/nwm_
 export                          input_WBD_gdb=${inputsDir}/wbd/WBD_National_EPSG_5070_WBDHU8_clip_dem_domain.gpkg
 export                   input_WBD_gdb_Alaska=${inputsDir}/wbd/WBD_National_South_Alaska.gpkg
 export                 input_calib_points_dir=${inputsDir}/rating_curve/water_edge_database/calibration_points/
-export                        bathymetry_file=${inputsDir}/bathymetry/bathymetric_adjustment_data.gpkg
+export                        bathy_file_ehydro=${inputsDir}/bathymetry/final_bathymetry_ehydro.gpkg
+export                        bathy_file_aibased=${inputsDir}/bathymetry/ml_outputs_v1.01.parquet
+export                        mannN_file_aibased=${inputsDir}/bathymetry/ml_outputs_v1.01.parquet
 export                            osm_bridges=${inputsDir}/osm/bridges/241002/osm_all_bridges.gpkg
 
 # input file location with nwm feature_id and recurrence flow values
 export               bankfull_flows_file=${inputsDir}/rating_curve/bankfull_flows/nwm3_high_water_threshold_cms.csv
 
 # input file location with nwm feature_id and channel roughness and overbank roughness attributes
 export                  vmann_input_file=${inputsDir}/rating_curve/variable_roughness/mannings_global_nwm3.csv
+# export                  vmann_input_file=${inputsDir}/rating_curve/variable_roughness/mannings_optz_fe_clusters_so3.csv
 
 # input file location with nwm feature_id and recurrence flow values
 export                    nwm_recur_file=${inputsDir}/rating_curve/nwm_recur_flows/nwm3_17C_recurrence_flows_cfs.csv