[Trilinos-Users] [EXTERNAL] error building trilinos with CUDA support

[Elliott, James John]

Also,

Be careful with shell scripts. You probably need to re-export these in your shell before you run make.
Make sure you load the correct compiler modules as well. E.g,

./do_cmake_configure

. /usr/local/tools/dotkit/init.sh
 use  gcc-4.8.5p

export OMPI_CXX=/p/lscratche/hysom/trilinos-12.10.1-Source/Trilinos/packages/kokkos/config/nvcc_wrapper
export NVCC_WRAPPER_DEFAULT_COMPILER=/usr/apps/gnu/4.8.5/bin/g++
export CUDA_LAUNCH_BLOCKING=1

make -j

James
________________________________________
From: Trilinos-Users <trilinos-users-bounces at trilinos.org> on behalf of Elliott, James John <jjellio at sandia.gov>
Sent: Tuesday, February 21, 2017 10:06 AM
To: David Hysom; trilinos-users at trilinos.org
Subject: Re: [Trilinos-Users] [EXTERNAL] error building trilinos with CUDA support

Hi David,

Set
CMAKE_CXX_COMPILER
CMAKE_C_COMPILER

To
mpicxx and mpicc

This changes the compiler that mpicxx will use. Now, it will call nvcc_wrapper under the hood.
export OMPI_CXX=/p/lscratche/hysom/trilinos-12.10.1-Source/Trilinos/packages/kokkos/config/nvcc_wrapper

This sets the compiler that NVCC_WRAPPER uses to your g++ binary
export NVCC_WRAPPER_DEFAULT_COMPILER=/usr/apps/gnu/4.8.5/bin/g++

James

________________________________________
From: Trilinos-Users <trilinos-users-bounces at trilinos.org> on behalf of David Hysom <hysom1 at llnl.gov>
Sent: Tuesday, February 21, 2017 9:52 AM
To: trilinos-users at trilinos.org
Subject: [EXTERNAL] [Trilinos-Users] error building trilinos with CUDA support

acording to this page:
https://trilinos.org/about/documentation/building-with-cuda-support/

I should include this in my cmake script:

  -DCMAKE_CXX_FLAGS="-g -lineinfo -Xcudafe \
--diag_suppress=conversion_function_not_usable -Xcudafe \
--diag_suppress=cc_clobber_ignored -Xcudafe \
--diag_suppress=code_is_unreachable" \

However, those flags are not recognized by the compiler.
I was able to successfully configure with cuda support
(see script below), however, when I then run make I get
these errors:

/p/lscratche/hysom/trilinos-12.10.1-Source/packages/kokkos/core/src/Kokkos_Macros.hpp:163:40: error: '__device__' does not name a type
    #define KOKKOS_INLINE_FUNCTION       __device__  __host__  inline
                                         ^
/p/lscratche/hysom/trilinos-12.10.1-Source/packages/kokkos/core/src/impl/Kokkos_Utilities.hpp:58:1: note: in expansion of macro 'KOKKOS_INLINE_FUNCTION'
  KOKKOS_INLINE_FUNCTION
/p/lscratche/hysom/trilinos-12.10.1-Source/packages/kokkos/core/src/Kokkos_Core_fwd.hpp:182:3: note: in expansion of macro 'KOKKOS_INLINE_FUNCTION'
    KOKKOS_INLINE_FUNCTION static void verify(const void *) {}

Any help will be appreciated!

======
#!/bin/bash

. /usr/local/tools/dotkit/init.sh
use  gcc-4.8.5p

export OMPI_CXX=/p/lscratche/hysom/trilinos-12.10.1-Source/Trilinos/packages/kokkos/config/nvcc_wrapper
export NVCC_WRAPPER_DEFAULT_COMPILER=/usr/apps/gnu/4.8.5/bin/g++
export CUDA_LAUNCH_BLOCKING=1

export CUDA_TOOLKIT_ROOT_DIR=/opt/cudatoolkit-7.0
export CUDA_BIN_PATH=/opt/cudatoolkit-7.0

TRILINOS_SOURCE_PATH=/p/lscratche/hysom/trilinos-12.10.1-Source
TRILINOS_INSTALL_PATH=/p/lscratchf/hysom/trilinos_cuda

EXTRA_ARGS=$@

rm -f CMakeCache.txt

cmake  \
           -D CMAKE_CXX_COMPILER:STRING="/usr/apps/gnu/4.8.5/bin/g++" \
           -D CMAKE_C_COMPILER:STRING="/usr/apps/gnu/4.8.5/bin/gcc" \
           -DCMAKE_CXX_FLAGS="-g" \
            -DTPL_ENABLE_CUDA=ON \
            -DKokkos_ENABLE_Cuda=ON \
           -DKokkos_ENABLE_Cuda_UVM=ON \
           -D Trilinos_ENABLE_Fortran:BOOL=ON \
           -D Trilinos_ASSERT_MISSING_PACKAGES=OFF \
           -D Teuchos_ENABLE_COMPLEX:BOOL=OFF \
           -D Tpetra_INST_COMPLEX_DOUBLE:BOOL=OFF \
           -D Trilinos_ENABLE_ThreadPool:BOOL=OFF \
           -D Trilinos_ENABLE_TEUCHOS_TIME_MONITOR:BOOL=ON \
           -D TPL_ENABLE_Pthread:BOOL=OFF  \
           -D TPL_ENABLE_MPI:BOOL=OFF \
           -D TPL_ENABLE_TBB:BOOL=OFF \
           -D Trilinos_ENABLE_Kokkos:BOOL=ON \
           -D Kokkos_ENABLE_EXAMPLES:BOOL=ON \
           -D Kokkos_ENABLE_TESTS:BOOL=OFF \
           -D Tpetra_DefaultNode:STRING=Kokkos::Compat::KokkosCudaWrapperNode \
           -D Trilinos_ENABLE_OpenMP:BOOL=ON \
           -D Trilinos_ENABLE_CHECKED_STL:BOOL=OFF \
           -D Trilinos_ENABLE_EXPLICIT_INSTANTIATION:BOOL=OFF \
           -D Trilinos_ENABLE_ALL_OPTIONAL_PACKAGES:BOOL=OFF \
           -D Trilinos_ENABLE_Anasazi:BOOL=ON \
           -D Trilinos_ENABLE_Tpetra:BOOL=ON \
           -D Trilinos_ENABLE_Zoltan2:BOOL=OFF \
           -D Trilinos_ENABLE_MueLu:BOOL=OFF \
           -D Zoltan2_ENABLE_TESTS:BOOL=OFF \
           -D MueLu_ENABLE_TESTS:BOOL=OFF \
           -D Tpetra_ENABLE_TESTS:BOOL=OFF \
           -D Trilinos_ENABLE_EXAMPLES:BOOL=OFF \
           -D CMAKE_INSTALL_PREFIX:STRING=${TRILINOS_INSTALL_PATH} \
           -D CMAKE_BUILD_TYPE:STRING=RELEASE \
           -D Kokkos_ENABLE_Serial:BOOL=OFF \
           -D DART_TESTING_TIMEOUT:STRING=600 \
           -D Trilinos_ENABLE_Belos=ON \
           -D Trilinos_ENABLE_Ifpack2=ON \
           ${EXTRA_ARGS} \
           ${TRILINOS_SOURCE_PATH}

_______________________________________________
Trilinos-Users mailing list
Trilinos-Users at trilinos.org
https://trilinos.org/mailman/listinfo/trilinos-users
_______________________________________________
Trilinos-Users mailing list
Trilinos-Users at trilinos.org
https://trilinos.org/mailman/listinfo/trilinos-users