TrilinosHandsOnTutorial

Trilinos Hands-on Tutorial

Reference and tutorial material

• Trilinos website
• Each Trilinos package has Doxygen documentation.
• Trilinos "Getting Started" page.
• See links below to slides and video recordings of tutorials from previous Trilinos User Group meetings.

Argonne Training Program on Extreme-scale Scientific Computing (ATPESC) Tutorial

Presently, the most recent tutorial on Trilinos capabilities is from the Summer 2023 ATPESC tutorial.

The slides are here. The hands-on material is here and uses the MueLu_Stratimikos.exe example from here.

WebTrilinos Portal: Web interface to edit, build, and run C++ code

• Note: Useful for the Epetra-based Trilinos stack. Not up to date for Tpetra-based solvers (Tpetra is the newest generation of Trilinos data services based on Kokkos, providing support for multi-core, many-core and accelerated architectures).

• WebTrilinos provides source code copy-paste of simple driver programs, compilation and linking against a pre-built copy of Trilinos and execution of the generated executable within the browser window. This approach to use Trilinos can be particularly useful in tutorial settings since, once the browser window is launched, working with Trilinos becomes much simpler and interactive.

• WebTrilinos is available as a Docker Image. Presently, this image is usable but not current for Tpetra-based Trilinos capabilities.

• Instructions for getting started with Docker and the WebTrilinos Image

• Download WebTrilinos from Docker Hub.

• Once WebTrilinos is running in your browser, you will see a text box in which you can paste, type, or edit C++ code.

  • That code will compile and link against a recent release of Trilinos, and run.
  • The web page will show you the resulting output.
  • Use Ctrl+A to highlight all the example code, Ctrl+C to copy it, and Ctrl+V to paste it in the WebTrilinos window.
    • You can't read or write files, but you can embed input data in your program as a string.

HPCLinux virtual machine (VM)

• HPCLinux VM with same build environment as the student shell accounts
• We won"t cover this today, but you can download it and try it at home
• NOTE: The VM file is QUITE LARGE (11116475904 bytes)
• Download locations:
• University of Oregon (HTTP)
• ParaTools, Inc. (FTP)
• wopr.nic.uoregon.edu:~livetau/HPCLinux_June14.2.ova
• VM setup instructions from ParaTools

MueLu virtual machine (VM)

• VM and accompanying PDF delivering the MueLu tutorial

Build Trilinos yourself on your computer

• You can build your own local copy of Trilinos. This is not a recommended approach for a first-time user, unless you are very familiar with complex software installation processes. While we strive to make Trilinos easier to install, it is relatively difficult to install compared to other software you have probably used. Instructions for installation are on the Getting Started page.

Examples illustrating Kokkos thread-scalable expressions to generate Tpetra objects

If using cmake to build your own application on an installed trilinos, use the top level CMakeLists.txt and create two subdirectories for the examples below with the files CMakeLists.txt file for KokkosBasic directory and CMakeLists.txt file for KokkosTpetra directory.

1. Learn the basics of Kokkos

   • Lesson 1a: Learn how to Initialize Kokkos using functors.
   • Lesson 1b: Learn how to Initialize Kokkos using lambdas.
   • Lesson 2a: Learn how to perform a reduction using functors.
   • Lesson 2b: Learn how to perform a reduction using lambdas.
   • Lesson 3: Learn how to createa simple view.
   • Lesson 4: Learn the basic of memory spaces.
   • Lesson 5: Learn how to use atomics.

2. Learn how to create and use Kokkos with Tpetra.

   • Lesson 1: Learn how to initialize Kokkos.
   • Lesson 2: Learn how to write a simple parallel for loop.
   • Lesson 3: Learn how to construct a local tridiagonal matrix using a thread-parallel approach.
   • Lesson 4: Learn how to construct Tpetra matrices and vectors from existing Kokkos arrays.
   • Lesson 5: Learn how to write a simple conjugate gradients solver with Tpetra/Kokkos.

Examples illustrating the Tpetra-based solver stack

1. Learn how to create and use Tpetra dense and sparse linear algebra objects.

   • Lesson 1: "Hello world!" Learn different ways to initialize MPI (or not) and pass an MPI communicator to Tpetra.
   • Lesson 2: Learn how to make a Tpetra vector, given a communicator from Lesson 1.
   • Lesson 3: Learn how to implement a simple numerical algorithm (the power method) using Tpetra sparse matrices and vectors.
   • Lesson 4: Learn different ways to construct a Tpetra sparse matrix.
   • Lesson 5: Learn how to migrate data in a Tpetra object between two different parallel data distributions.
   • Advanced exercise: Learn how to compute the explicit transpose of a sparse matrix.

2. Learn how to solve linear systems using the Belos package of iterative linear solvers, and the Ifpack2 package of preconditioners

   • Create an Ifpack2 preconditioner
   • TODO: Update for Trilinos 12.0 and later: Solve a linear system using Belos and Ifpack2

Examples illustrating the Epetra-based solver stack

Create and use Epetra dense and sparse linear algebra objects

• Lesson 1: "Hello world!" Learn different ways to initialize MPI (or not) and pass an MPI communicator to Epetra.
• Lesson 2: Learn how to make an Epetra vector, given a communicator from Lesson 1.
• Lesson 3: Learn how to implement a simple numerical algorithm (the power method) using Epetra sparse matrices and vectors.
• Lesson 4: Learn different ways to construct an Epetra sparse matrix.
• Lesson 5: Learn how to migrate data in an Epetra object between two different parallel data distributions.

Generate test linear systems using the Galeri package

• Generate a matrix, discretized 2D Laplacian on a Cartesian grid.
• Try generating matrices for some different operators. The list of supported operators is here.

Create an algebraic preconditioner using the Ifpack package

• Create a preconditioner using the Ifpack preconditioner factory.
• Try generating different preconditioners:
  • The preconditioners supported in the factory interface are: "IC", "ICT", "ILU", "ILUT", and "Amesos".
  • The list of supported parameters for the factory is here.

Solve a linear system using Amesos

The Amesos package implements direct linear solvers. It wraps provides one native sparse direct solver called KLU and provides access to other community sparse direct solvers such as SuperLU. Amesos accepts Epetra data objects like any Trilinos solver.

Solve a linear system using Amesos with SuperLU

Solve a linear system using AztecOO

The AztecOO package implements iterative linear solvers. It wraps an earlier library which provided its own linear algebra implementation. !AztecOO can also work with Epetra matrices and vectors, and any preconditioner that implements Epetra_Operator. The latter includes Ifpack preconditioners.

• Solve a linear system using Ifpack and AztecOO

Solve a linear system using ML and AztecOO

The ML package implements multilevel solvers, including algebraic multigrid. You may use ML's solvers as preconditioners if you like; this is the way most of our users use ML. The following examples show how to do this with the iterative linear solvers provided by the AztecOO package.

• Use ML as a black-box preconditioner with AztecOO
• Use ML as a preconditioner with AztecOO, but set some multigrid options

Solve a linear system using Belos

The Belos package implements iterative linear solvers. Unlike AztecOO, Belos can work with just about any linear algebra implementation. Belos also provides block solvers and other algorithmic optimizations, like subspace recycling. Once you learn how to use Belos with Epetra objects, it is easy to learn how to use Belos with Tpetra or other linear algebra implementations.

• Solve a linear system using Ifpack and Belos

Solve a nonlinear system using NOX

The NOX package implements iterative nonlinear solvers.

• Example 1: Newton's method
• Example 2: Newton's method

Solve eigenvalue problems using Anasazi

The Anasazi package implements several different iterative solvers for both standard (A x = \lambda x) and generalized (K x = \lambda M x) eigenvalue problems. The first two examples show the simple use case of finding a few of the eigenvalues of largest magnitude:

• Compute the largest eigenpairs of an eigenvalue problem using block Davidson.
• Compute the largest eigenpairs of an eigenvalue problem using LOBPCG.

The next two examples show how to use inverse iteration with Block Krylov-Schur to find a few of the eigenvalues of smallest magnitude. You may use just about any linear solver for inverse iteration. The following examples illustrate this for two different Trilinos linear solvers.

• Inverse iteration using the KLU sparse direct solver through Amesos.
• Inverse iteration using an AztecOO iterative linear solver with an Ifpack preconditioner.

Zoltan tutorial

The Zoltan developers generously contributed a hands-on tutorial of their own.

Examples illustrating other packages

1. PyTrilinos tutorial materials.

   • PyTrilinos tutorial slides
   • WebTrilinos for Python
   • Solve a simple 2D Laplace problem using PyTrilinos

2. Learn how to use utilities in the Teuchos package.

   • Basic Support Tools
     • Build a parameter list (used to pass parameters to all Trilinos packages).
     • Build a reference-counted pointer (used to eliminate memory leak issues in most Trilinos packages).
     • Build a command-line parser (tool for changing runtime behavior of program, providing documentation for options). [unhighlighted]
     • Build a time monitor (tool for timing individual methods or operations in a program).
   • Template Support Tools
     • Create a templated BLAS wrapper.
     • Create a templated LAPACK wrapper.
     • Create a templated serial, dense matrix.

Other examples

Some examples don't work with the web tutorial, since they read from files. You can try them out by downloading Trilinos and looking in the examples in the source tree. For example, the Intrepid discretizations package has examples in the packages/trilinoscouplings/examples/scaling/ directory. The following might be of interest:

• example_CurlLSFEM.cpp: driver for solving div-curl first order system in 3D with tangential boundary condition using curl-conforming elements
• example_DivLSFEM.cpp: driver for the same system with normal boundary condition and div-conforming elements
• example_Poisson.cpp: solving the Poisson equation using a Galerkin finite element method

Learning more:

• Past Trilinos Events
• Mail lists