Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature 3047 dev env seneca oneapi #3048

Open
wants to merge 13 commits into
base: develop
Choose a base branch
from

Conversation

jprestop
Copy link
Collaborator

Expected Differences

  • Do these changes introduce new tools, command line arguments, or configuration file options? [No]

    If yes, please describe:

  • Do these changes modify the structure of existing or add new output data types (e.g. statistic line types or NetCDF variables)? [No]

    If yes, please describe:

Pull Request Testing

  • Describe testing already performed for these changes:

I have not tested this file due to time constraints and my unfamiliarity with using this file.

  • Recommend testing for the reviewer(s) to perform, including the location of input datasets, and any additional instructions:

Please test the file in the same way you would use development.seneca

  • Do these changes include sufficient documentation updates, ensuring that no errors or warnings exist in the build of the documentation? [Yes]

  • Do these changes include sufficient testing updates? [Yes]

  • Will this PR result in changes to the MET test suite? [No]

    If yes, describe the new output and/or changes to the existing output:

  • Will this PR result in changes to existing METplus Use Cases? [No]

    If yes, create a new Update Truth METplus issue to describe them.

  • Do these changes introduce new SonarQube findings? [No]

    If yes, please describe:

  • Please complete this pull request review by [ASAP].

Pull Request Checklist

See the METplus Workflow for details.

  • Review the source issue metadata (required labels, projects, and milestone).
  • Complete the PR definition above.
  • Ensure the PR title matches the feature or bugfix branch name.
  • Define the PR metadata, as permissions allow.
    Select: Reviewer(s) and Development issue
    Select: Milestone as the version that will include these changes
    Select: Coordinated METplus-X.Y Support project for bugfix releases or MET-X.Y.Z Development project for official releases
  • After submitting the PR, select the ⚙️ icon in the Development section of the right hand sidebar. Search for the issue that this PR will close and select it, if it is not already selected.
  • After the PR is approved, merge your changes. If permissions do not allow this, request that the reviewer do the merge.
  • Close the linked issue and delete your feature or bugfix branch from GitHub.

@jprestop jprestop added this to the MET-12.1.0 milestone Jan 10, 2025
@JohnHalleyGotway
Copy link
Collaborator

I used the following commands to test this out on seneca:

cd /d1/projects/MET/MET_pull_requests/met-12.1.0/beta1/
runas met_test
git clone https://github.com/dtcenter/MET MET-feature_3047_dev_env_seneca_oneapi
cd MET-feature_3047_dev_env_seneca_oneapi/
git checkout feature_3047_dev_env_seneca_oneapi
source internal/scripts/environment/development.seneca_intel_oneapi
./configure --prefix=`pwd` --enable-all

However, config.log indicates that autoconf chose the GNU compilers:

CC='gcc'
CXX='g++'
F77='gfortran'

I'm thinking we should explicitly set these to the Intel compilers instead in development.seneca_intel_openapi since the default choice of GNU does not suffice.

@jprestop
Copy link
Collaborator Author

@JohnHalleyGotway Right! Great point! Thank you for catching that. I just modified the file to include the appropriate values. I also added a new file (internal/scripts/installation/config/install_met_env.seneca_intel_oneapi), which I used for compiling MET and its external libraries on seneca.

@JohnHalleyGotway
Copy link
Collaborator

JohnHalleyGotway commented Jan 14, 2025

Thanks for the updates @jprestop!

I retested this morning and got a lot further... but it did fail with a -lgrib2c linker error. Please see below:

cd /d1/projects/MET/MET_pull_requests/met-12.1.0/beta1/
runas met_test
git clone https://github.com/dtcenter/MET MET-feature_3047_dev_env_seneca_oneapi
cd MET-feature_3047_dev_env_seneca_oneapi/
git checkout feature_3047_dev_env_seneca_oneapi
source internal/scripts/environment/development.seneca_intel_oneapi
./configure --prefix=`pwd` --enable-all
make install test

Here's the error when compiling ensemble_stat which is the first application to be linked:

/bin/ld: cannot find -lgrib2c: No such file or directory

Looks like it's named /d1/projects/MET/intel/external_libs/lib/libg2c.a instead.

@jprestop
Copy link
Collaborator Author

Ah! Thanks, @JohnHalleyGotway. I have added the following:

export BUFRLIB_NAME=-lbufr_4
export GRIB2CLIB_NAME=-lg2c

which should resolve the problem.

@JohnHalleyGotway
Copy link
Collaborator

@jprestop, the next error is for hdf with lidar2nc:

lidar2nc.cc:40:10: fatal error: 'hdf.h' file not found
#include "hdf.h"
         ^~~~~~~
1 error generated.
make[4]: *** [Makefile:507: lidar2nc-lidar2nc.o] Error 1

Has this actually worked for you on seneca? Or am I testing this here for the first time?

@jprestop
Copy link
Collaborator Author

Please see this comment on the issue.

@JohnHalleyGotway
Copy link
Collaborator

@jprestop, thanks for clarifying. Instead of configuring with --enable-all, I switched to using:
--enable-grib2 --enable-python --enable-ugrid --enable-mode_graphics
Note that this excludes --enable-lidar2nc --enable-modis.

Here's the commands:

./configure --prefix=`pwd` --enable-grib2 --enable-python --enable-ugrid --enable-mode_graphics
make clean install test >& make.log

The next failure is due to the absence of the ifx compiler:

ifx  -g -O2 -c -o calcape.o calcape.f
/bin/bash: line 1: ifx: command not found

Checking on seneca, I don't see it:

ls /var/autofs/mnt/linux-amd64/debian/bookworm/local/intel-2023.2.0.49397/compiler/2023.2.1/linux/bin/ifx
ls: cannot access '/var/autofs/mnt/linux-amd64/debian/bookworm/local/intel-2023.2.0.49397/compiler/2023.2.1/linux/bin/ifx': No such file or directory

@jprestop
Copy link
Collaborator Author

@JohnHalleyGotway
Thanks for switching from --enable-all to --enable-grib2 --enable-python --enable-ugrid --enable-mode_graphics.

I just sent an email to helpdesk and copied you and @hsoh-u. It looks like the latest version is missing 'ifx' and I used a lesser version that had 'ifx'. Thank you @hsoh-u for suggesting I remove the paths to the compilers, which allowed this problem to come to our attention. So that John can proceed with testing, I have added the paths back in, but we can remove them once helpdesk gets 'ifx' added to the latest installation of Intel oneAPI.

@jprestop
Copy link
Collaborator Author

@JohnHalleyGotway Perhaps I should have waited until helpdesk responded before adding those paths back in. I didn't expect such a fast response.

To update from the information in the email, Tor is going to have /nrit/ral/intel/setvars.sh point to the newer version of the compiler here: /var/autofs/mnt/linux-amd64/debian/bookworm/local/intel-2024.2.1.100/compiler/2024.2/bin.

We'll see if there are any problems since I compiled with 2023.2.0... or would you prefer I recompile using 2024.2.1.100 before you test it out again? Apologies for not doing that initially, but I did not realize it was available. It looks like it was added in October 2024.

Copy link
Collaborator

@JohnHalleyGotway JohnHalleyGotway left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please install an updated version of the Intel compilers to include the ifx Fortran compiler needed to install MET.

@jprestop
Copy link
Collaborator Author

Please install an updated version of the Intel compilers to include the ifx Fortran compiler needed to install MET.

This action was completed by Tor. Is there anything you need me to do before requesting a re-review?

@JohnHalleyGotway
Copy link
Collaborator

@jprestop I figured you'd need to recompile the libraries upon which MET depends using the updated compiler version so that MET links to them smoothly. And that you'd ping me once you're confident that this setup is working as you'd like it to.

@JohnHalleyGotway JohnHalleyGotway self-requested a review January 16, 2025 20:39
@jprestop
Copy link
Collaborator Author

@JohnHalleyGotway I commented out the sourcing of setvars.sh and added the paths for the older versions of the compilers which I used to install MET and its dependent libraries. Please try testing again. It may be that we need to uncomment the sourcing of setvars.sh, but let's try this and see how it goes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: 🔎 In review
Development

Successfully merging this pull request may close these issues.

Create a new development environment file for Seneca for the Intel oneAPI compilers
2 participants