-
Notifications
You must be signed in to change notification settings - Fork 241
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update write.config.FATES.R _ New Version #3372
base: develop
Are you sure you want to change the base?
Changes from all commits
ba974e3
69af60e
b169a7a
dd2fece
fcfca38
e4d6720
f04d07e
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,148 @@ | ||
# this needs to be at the top, what version are we building | ||
ARG IMAGE_VERSION="latest" | ||
|
||
# ---------------------------------------------------------------------- | ||
# BUILD PECAN FOR FATES | ||
# ---------------------------------------------------------------------- | ||
FROM pecan/models:${IMAGE_VERSION} | ||
|
||
# load packages before adding CTSM | ||
RUN apt update && apt upgrade -y | ||
RUN apt install -y \ | ||
sudo \ | ||
build-essential \ | ||
python3-dev \ | ||
cmake \ | ||
gfortran-9 \ | ||
zlib1g-dev \ | ||
byacc \ | ||
libblas-dev \ | ||
liblapack-dev \ | ||
libxml2-utils \ | ||
libxml-libxml-perl \ | ||
libgdal-dev \ | ||
xmlstarlet \ | ||
csh \ | ||
diffutils \ | ||
rsync | ||
|
||
RUN update-alternatives --install /usr/bin/gfortran gfortran /usr/bin/gfortran-9 100 | ||
|
||
RUN echo '/usr/local/lib' > /etc/ld.so.conf.d/local.conf && ldconfig | ||
|
||
RUN mkdir /tmp/sources | ||
|
||
WORKDIR /tmp/sources | ||
|
||
RUN wget -q http://www.mpich.org/static/downloads/3.3.2/mpich-3.3.2.tar.gz && \ | ||
tar zxf mpich-3.3.2.tar.gz && \ | ||
cd mpich-3.3.2 && \ | ||
./configure --prefix=/usr/local && \ | ||
make -j 2 install | ||
|
||
RUN wget -q https://support.hdfgroup.org/ftp/HDF5/releases/hdf5-1.12/hdf5-1.12.0/src/hdf5-1.12.0.tar.gz && \ | ||
tar zxf hdf5-1.12.0.tar.gz && \ | ||
cd hdf5-1.12.0 && \ | ||
./configure --prefix=/usr/local && \ | ||
make -j 2 install | ||
|
||
RUN wget -q https://github.com/Unidata/netcdf-c/archive/refs/tags/v4.7.4.tar.gz && \ | ||
tar zxf v4.7.4.tar.gz && \ | ||
cd netcdf-c-4.7.4 && \ | ||
./configure --prefix=/usr/local && \ | ||
make -j 2 install && \ | ||
ldconfig | ||
|
||
RUN wget -q https://github.com/Unidata/netcdf-fortran/archive/refs/tags/v4.5.3.tar.gz && \ | ||
tar zxf v4.5.3.tar.gz && \ | ||
cd netcdf-fortran-4.5.3 && \ | ||
## original netcdf(version=4.7.3): /usr, ; fates netcdf(version>=4.7.4): /usr/local; --enable-fortran to ensure netcdf.mod | ||
./configure --prefix=/usr/local LDFLAGS="-L/usr/local/lib" CPPFLAGS="-I/usr/local/include" --enable-fortran && \ | ||
make -j 2 install && \ | ||
ldconfig | ||
|
||
RUN wget -q https://parallel-netcdf.github.io/Release/pnetcdf-1.12.1.tar.gz && \ | ||
tar zxf pnetcdf-1.12.1.tar.gz && \ | ||
cd pnetcdf-1.12.1 && \ | ||
./configure MPIF77=/usr/local/bin/mpif77 MPIF90=/usr/local/bin/mpif90 MPICXX=/usr/local/bin/mpicxx MPICC=/usr/local/bin/mpicc && \ | ||
make -j 2 install && \ | ||
ldconfig | ||
|
||
RUN cd /tmp/sources \ | ||
wget -q https://github.com/esmf-org/esmf/archive/refs/tags/ESMF_8_2_0.tar.gz && \ | ||
tar zxf ESMF_8_2_0.tar.gz && \ | ||
cd esmf-ESMF_8_2_0 && \ | ||
export ESMF_DIR='/tmp/sources/esmf-ESMF_8_2_0' && \ | ||
export ESMF_INSTALL_PREFIX='/usr/local' && \ | ||
export ESMF_LAPACK="netlib" && \ | ||
export ESMF_LAPACK_LIBS="-llapack -lblas" && \ | ||
export ESMF_LAPACK_LIBPATH="/usr/lib/x86_64-linux-gnu" && \ | ||
export ESMF_NETCDF="nc-config" && \ | ||
make && \ | ||
make install && \ | ||
ldconfig | ||
|
||
RUN rm -rf /tmp/sources | ||
RUN pip install --upgrade pip setuptools | ||
|
||
## Above has been tested in the pecan/model container | ||
|
||
######################################################################## | ||
|
||
## Below needs further discussion. | ||
|
||
## Load CTSM from https://github.com/NorESMhub/noresm-land-sites-platform/blob/main/docker/api/entrypoint_api.sh | ||
## 1. Standard address of /ctsm-api/resources/model in pecan ?? | ||
RUN if [ ! -d /ctsm-api/resources/model ]; then && \ | ||
mkdir -p /ctsm-api/resources && \ | ||
fi && \ | ||
## go to the folder for CTSM | ||
cd /ctsm-api/resources/ && \ | ||
## download CTSM | ||
git clone https://github.com/ESCOMP/CTSM.git && \ | ||
## go to model folder | ||
cd model && \ | ||
## Checkout specific CTSM version, which has been tested in the container, more recent version than this needs to be tested in the future. | ||
git checkout ctsm5.1.dev108 && \ | ||
## Download fates (tag = sci.1.58.1_api.24.1.0), the version can be modified in Externals_CLM.cfg | ||
./manage_externals/checkout_externals | ||
|
||
RUN set -e && \ | ||
## set home folder for running CTSM-fates. 2. what does 'HOME' should be in pecan? | ||
USER=root && \ | ||
HOME=/root && \ | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. what does |
||
## create .bashrc file for setting basic compiling parameters for CTSM-fates | ||
export USER="$USER" && \ | ||
export PYTHONPATH=/ctsm-api && \ | ||
export CIME_MACHINE=container && \ | ||
export MPICC=mpicc && \ | ||
export MPIFC=mpif90 && \ | ||
export MPIF90=mpif90 && \ | ||
export MPIF77=mpif77 && \ | ||
## 3. What is data folder for CTSM-fates in pecan? | ||
export CESMDATAROOT=${CESMDATAROOT:?} && \ | ||
ln -fs /ctsm-api/docker/dotcime "$HOME"/.cime | ||
|
||
## Copy dotcime and entrypoints from ctsm-api to fates | ||
## Dotcime is a key file for setting the compiling environment for CTSM-fates | ||
## Dotcime (https://github.com/NorESMhub/ctsm-api/tree/main/docker/dotcime) SHOULD be kept in the home directory as '.cime' | ||
RUN COPY docker/dotcime /<home folder>/.cime | ||
|
||
## 3. Download inputdata in dockerfile? | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Do we need to download input data in docker file? |
||
## 4. Build CTSM-fates here or in write.config file? | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Do we need to build CTSM-fates in Dockerfile or in write.config.R? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Since CLM-FATES needs to be re-complied every time when some setting changes, I would vote for putting the "building " phase of the model in template.job similar to what have been done there at line 134. |
||
|
||
######################################################################## | ||
|
||
## 5. Is the following part (copied from sipnet) necessary for fates? | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The following part is copied from sipnet. Are they (e.g., copy binaries) still necessary for fates in pecan? |
||
|
||
# Some variables that can be used to set control the docker build | ||
ARG MODEL_VERSION=git | ||
|
||
# Setup model_info file | ||
COPY model_info.json /work/model.json | ||
RUN sed -i -e "s/@VERSION@/${MODEL_VERSION}/g" \ | ||
-e "s#@BINARY@#/usr/local/bin/fates.${MODEL_VERSION}#g" /work/model.json | ||
|
||
# Is it needed to move binary to sepcific folder? | ||
COPY --from=model-binary /src/CESM/fates /usr/local/bin/fates.${MODEL_VERSION} | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
where to put
/ctsm-api/resources/model
in pecan?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just to clarify that this folder refers to where we should put the model source code in the container. I guess there should be a standard folder for keeping each model's source code in the model container?