[Trilinos-Users] using OpenMP in Trilinos

[Hoemmen, Mark]

On 11/6/14, 12:00 PM, "trilinos-users-request at software.sandia.gov"
<trilinos-users-request at software.sandia.gov> wrote:
>Message: 8
>Date: Thu, 6 Nov 2014 12:45:21 -0500 (EST)
>From: Alicia M Klinvex <aklinvex at purdue.edu>
>To: trilinos-users <trilinos-users at software.sandia.gov>
>Subject: [Trilinos-Users] using openmp in Trilinos
>
>Hello,
>
>I'm interested in running some Tpetra-based code on a cluster of
>multicore nodes.  I want to run one MPI process per node, with several
>threads per process using OpenMP.
>
>I saw that to enable OpenMP in my code, I should add this to my
>configuration script:
>  -D Trilinos_ENABLE_OpenMP:BOOL=ON

Hi Alicia!  This is a necessary step, but please also note that not all
computational kernels have been parallelized or made thread-safe yet.
Also, it would be a good idea for you to try out the Kokkos refactor
version of Tpetra.  (See packages/tpetra/ReleaseNotes.txt for
instructions; poke me for questions.)

>Should I also build with the multithreaded version of BLAS/LAPACK, or
>does that interfere with the OpenMP in Trilinos?

Whether it interferes or not depends on the BLAS/LAPACK implementation,
and whether it uses the same OpenMP run-time environment as your
application.  Some implementations might be rude and pin worker threads to
cores, which will almost certainly interfere with your application's
OpenMP threads.  It generally doesn't hurt to disable threads in your
BLAS/LAPACK implementation, unless you plan on solving giant linear
systems or doing giant dense matrix-matrix multiplies.

Thanks!
mfh