[Trilinos-Users] Some experience in building 10.10.2 with CUDA support

[Baker, Christopher G.]

Hello Dmitry,

The cuda version was recently increased because of a requirement cusparse. In the near future, I hope to allow the use of cusp instead, which will allow the cuda requirement to decrease.

Thank you for the feedback.

Chris

-----Original Message-----
From: Dmitry N. Mikushin [maemarcus at gmail.com<mailto:maemarcus at gmail.com>]
Sent: Sunday, June 17, 2012 06:47 PM Eastern Standard Time
To: Trilinos-Users at software.sandia.gov
Subject: [Trilinos-Users] Some experience in building 10.10.2 with CUDA support

Dear Trilinos Community,

Recently we've tried to compile Trilinos in order to explore the GPU performance of various packages and finally were able to build it with several changes:

1) Required CUDA changed from 3.2 to 4.2. Is it possible to require minimum version instead of particular version?
2) Some .cu files are compiled without -fPIC, resulting into shared libraries linking errors
2) The Stokhos and Sacado are disabled because of complex Thrust-related errors and some sources using old incompatible Tensor class interface
3) Some Teuchos headers should guard #include <mpi.h> with #if defined(HAVE_MPI) && !defined(NVCC) instead of just #ifdef HAVE_MPI
4) Not sure if you really support this, but to enable CUDA, one could use the following build config:

#!/bin/sh
INSTALL_PREFIX=/opt/trilinos
MPI_INSTALL_PATH=/opt/openmpi-1.6_gcc

EXTRA_ARGS=$@

_libdir=/usr/lib
_includedir=/usr/include

cmake \
��� -G "Unix Makefiles" ��� ��� ��� ��� ��� \
��� -D CMAKE_INSTALL_PREFIX:PATH=$INSTALL_PREFIX��� ��� \
��� -D CMAKE_BUILD_TYPE:STRING=RELEASE��� ��� ��� \
��� -D BUILD_SHARED_LIBS:BOOL=ON��� ��� ��� ��� \
��� -D Trilinos_ENABLE_Didasko:BOOL=ON��� ��� ��� \
��� -D Trilinos_ENABLE_PyTrilinos:BOOL=ON��� ��� ��� \
��� -D Trilinos_ENABLE_ThreadPool:BOOL=ON��� ��� ��� \
��� -D Trilinos_ENABLE_Stokhos:BOOL=ON��� ��� ��� \
��� -D Trilinos_ENABLE_Phalanx:BOOL=ON��� ��� ��� \
��� -D Trilinos_ENABLE_TrilinosCouplings:BOOL=ON��� ��� \
��� -D Trilinos_ENABLE_ALL_PACKAGES:BOOL=ON��� ��� ��� \
��� -D Trilinos_ENABLE_Stokhos:BOOL=OFF��� ��� ��� \
��� -D Trilinos_ENABLE_Sacado:BOOL=OFF��� ��� ��� \
��� -D TPL_ENABLE_MPI:BOOL=ON��� ��� ��� ��� \
��� -D MPI_BASE_DIR:PATH=$MPI_INSTALL_PATH��� ��� ��� \
��� -D TEUCHOS_ENABLE_expat:BOOL=ON��� ��� ��� ��� \
��� -D TEUCHOS_ENABLE_libxml2:BOOL=ON��� ��� ��� \
��� -D TEUCHOS_ENABLE_gmp:BOOL=ON��� ��� ��� ��� \
��� -D AMESOS_ENABLE_SCALAPACK:BOOL=ON��� ��� ��� \
��� -D TPL_ENABLE_BLAS:BOOL=ON��� ��� ��� ��� \
��� -D TPL_ENABLE_LAPACK:BOOL=ON��� ��� ��� ��� \
��� -D NOX_ENABLE_lapack:BOOL=ON��� ��� ��� ��� \
��� -D Trilinos_ENABLE_TESTS:BOOL=ON ��� ��� ��� \
��� -D BLAS_LIBRARY_DIRS:PATH=${_libdir}/atlas-base/atlas/��� \
��� -D BLAS_LIBRARY_NAMES:STRING="atlas;cblas;blas"��� ��� \
��� -D LAPACK_LIBRARY_DIRS:PATH=${_libdir}/atlas-base/atlas/��� \
��� -D LAPACK_LIBRARY_NAMES:STRING="lapack" ��� ��� \
��� -D TPL_ENABLE_Pthread:BOOL=ON��� ��� ��� ��� \
��� -D TPL_ENABLE_Boost:BOOL=ON��� ��� ��� ��� \
��� -D TPL_ENABLE_Zlib:BOOL=ON��� ��� ��� ��� \
��� -D TPL_Thrust_INCLUDE_DIRS=/opt/cuda/include/��� ��� \
��� -D TPL_Thrust_LIBRARY_DIRS=/opt/cuda/include/��� ��� \
��� -D TPL_ENABLE_Thrust:BOOL=ON��� ��� ��� ��� \
��� -D TPL_ENABLE_CUDA=ON��� ��� ��� ��� ��� \
��� -D TPL_CppUnit_LIBRARIES:PATH=${_libdir}/��� ��� \
��� -D CUDA_NVCC_FLAGS:STRING="-Xcompiler; -fPIC"��� ��� \
��� $EXTRA_ARGS

Attached is a full diff to patch the current 10.10.2 tar ball.

Hope it's useful,
- Dima.