[Trilinos-Users] Some experience in building 10.10.2 with CUDA support

[Dmitry N. Mikushin]

Dear Trilinos Community,

Recently we've tried to compile Trilinos in order to explore the GPU
performance of various packages and finally were able to build it with
several changes:

1) Required CUDA changed from 3.2 to 4.2. Is it possible to require minimum
version instead of particular version?
2) Some .cu files are compiled without -fPIC, resulting into shared
libraries linking errors
2) The Stokhos and Sacado are disabled because of complex Thrust-related
errors and some sources using old incompatible Tensor class interface
3) Some Teuchos headers should guard #include <mpi.h> with #if
defined(HAVE_MPI) && !defined(NVCC) instead of just #ifdef HAVE_MPI
4) Not sure if you really support this, but to enable CUDA, one could use
the following build config:

#!/bin/sh
INSTALL_PREFIX=/opt/trilinos
MPI_INSTALL_PATH=/opt/openmpi-1.6_gcc

EXTRA_ARGS=$@

_libdir=/usr/lib
_includedir=/usr/include

cmake \
    -G "Unix Makefiles"                     \
    -D CMAKE_INSTALL_PREFIX:PATH=$INSTALL_PREFIX        \
    -D CMAKE_BUILD_TYPE:STRING=RELEASE            \
    -D BUILD_SHARED_LIBS:BOOL=ON                \
    -D Trilinos_ENABLE_Didasko:BOOL=ON            \
    -D Trilinos_ENABLE_PyTrilinos:BOOL=ON            \
    -D Trilinos_ENABLE_ThreadPool:BOOL=ON            \
    -D Trilinos_ENABLE_Stokhos:BOOL=ON            \
    -D Trilinos_ENABLE_Phalanx:BOOL=ON            \
    -D Trilinos_ENABLE_TrilinosCouplings:BOOL=ON        \
    -D Trilinos_ENABLE_ALL_PACKAGES:BOOL=ON            \
    -D Trilinos_ENABLE_Stokhos:BOOL=OFF            \
    -D Trilinos_ENABLE_Sacado:BOOL=OFF            \
    -D TPL_ENABLE_MPI:BOOL=ON                \
    -D MPI_BASE_DIR:PATH=$MPI_INSTALL_PATH            \
    -D TEUCHOS_ENABLE_expat:BOOL=ON                \
    -D TEUCHOS_ENABLE_libxml2:BOOL=ON            \
    -D TEUCHOS_ENABLE_gmp:BOOL=ON                \
    -D AMESOS_ENABLE_SCALAPACK:BOOL=ON            \
    -D TPL_ENABLE_BLAS:BOOL=ON                \
    -D TPL_ENABLE_LAPACK:BOOL=ON                \
    -D NOX_ENABLE_lapack:BOOL=ON                \
    -D Trilinos_ENABLE_TESTS:BOOL=ON             \
    -D BLAS_LIBRARY_DIRS:PATH=${_libdir}/atlas-base/atlas/    \
    -D BLAS_LIBRARY_NAMES:STRING="atlas;cblas;blas"        \
    -D LAPACK_LIBRARY_DIRS:PATH=${_libdir}/atlas-base/atlas/    \
    -D LAPACK_LIBRARY_NAMES:STRING="lapack"         \
    -D TPL_ENABLE_Pthread:BOOL=ON                \
    -D TPL_ENABLE_Boost:BOOL=ON                \
    -D TPL_ENABLE_Zlib:BOOL=ON                \
    -D TPL_Thrust_INCLUDE_DIRS=/opt/cuda/include/        \
    -D TPL_Thrust_LIBRARY_DIRS=/opt/cuda/include/        \
    -D TPL_ENABLE_Thrust:BOOL=ON                \
    -D TPL_ENABLE_CUDA=ON                    \
    -D TPL_CppUnit_LIBRARIES:PATH=${_libdir}/        \
    -D CUDA_NVCC_FLAGS:STRING="-Xcompiler; -fPIC"        \
    $EXTRA_ARGS

Attached is a full diff to patch the current 10.10.2 tar ball.

Hope it's useful,
- Dima.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://software.sandia.gov/pipermail/trilinos-users/attachments/20120618/2d4e37c3/attachment.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: trilinos-10.10.2.patch
Type: application/octet-stream
Size: 4234 bytes
Desc: not available
Url : https://software.sandia.gov/pipermail/trilinos-users/attachments/20120618/2d4e37c3/attachment.obj