[Trilinos-Users] [EXTERNAL] error building trilinos with CUDA support

[David Hysom]

James,

I made these changes in my script:

           -D CMAKE_CXX_COMPILER:STRING="/usr/local/bin/mpiCC" \
           -D CMAKE_C_COMPILER:STRING="/usr/local/bin/mpicc" \
           -D TPL_ENABLE_MPI:BOOL=ON \

(our code doesn't use mpi, which is why I had it turned off)

$which mpicc
/usr/local/bin/mpicc

$mpicc --version
gcc (GCC) 4.8.5

/p/lscratche/hysom/trilinos-12.10.1-Source/packages/kokkos/config/nvcc_wrapper
doesn't exist; I modified this to:
export 
OMPI_CXX=/p/lscratche/hysom/trilinos-12.10.1-Source/packages/kokkos/bin/nvcc_wrapper

I also exported the variables before running make.

Unf, I'm getting the same errors as before

(our code doesn't use mpi, which is why I had it turned off)

Have you any other suggestions?

thanks, David


On 02/21/2017 09:06 AM, Elliott, James John wrote:
> Hi David,
>
> Set
> CMAKE_CXX_COMPILER
> CMAKE_C_COMPILER
>
> To
> mpicxx and mpicc
>
> This changes the compiler that mpicxx will use. Now, it will call nvcc_wrapper under the hood.
> export OMPI_CXX=/p/lscratche/hysom/trilinos-12.10.1-Source/Trilinos/packages/kokkos/config/nvcc_wrapper
>
> This sets the compiler that NVCC_WRAPPER uses to your g++ binary
> export NVCC_WRAPPER_DEFAULT_COMPILER=/usr/apps/gnu/4.8.5/bin/g++
>
> James
>
> ________________________________________
> From: Trilinos-Users <trilinos-users-bounces at trilinos.org> on behalf of David Hysom <hysom1 at llnl.gov>
> Sent: Tuesday, February 21, 2017 9:52 AM
> To: trilinos-users at trilinos.org
> Subject: [EXTERNAL] [Trilinos-Users] error building trilinos with CUDA support
>
> acording to this page:
> https://trilinos.org/about/documentation/building-with-cuda-support/
>
> I should include this in my cmake script:
>
>    -DCMAKE_CXX_FLAGS="-g -lineinfo -Xcudafe \
> --diag_suppress=conversion_function_not_usable -Xcudafe \
> --diag_suppress=cc_clobber_ignored -Xcudafe \
> --diag_suppress=code_is_unreachable" \
>
> However, those flags are not recognized by the compiler.
> I was able to successfully configure with cuda support
> (see script below), however, when I then run make I get
> these errors:
>
> /p/lscratche/hysom/trilinos-12.10.1-Source/packages/kokkos/core/src/Kokkos_Macros.hpp:163:40: error: '__device__' does not name a type
>      #define KOKKOS_INLINE_FUNCTION       __device__  __host__  inline
>                                           ^
> /p/lscratche/hysom/trilinos-12.10.1-Source/packages/kokkos/core/src/impl/Kokkos_Utilities.hpp:58:1: note: in expansion of macro 'KOKKOS_INLINE_FUNCTION'
>    KOKKOS_INLINE_FUNCTION
> /p/lscratche/hysom/trilinos-12.10.1-Source/packages/kokkos/core/src/Kokkos_Core_fwd.hpp:182:3: note: in expansion of macro 'KOKKOS_INLINE_FUNCTION'
>      KOKKOS_INLINE_FUNCTION static void verify(const void *) {}
>
> Any help will be appreciated!
>
> ======
> #!/bin/bash
>
> . /usr/local/tools/dotkit/init.sh
> use  gcc-4.8.5p
>
> export OMPI_CXX=/p/lscratche/hysom/trilinos-12.10.1-Source/Trilinos/packages/kokkos/config/nvcc_wrapper
> export NVCC_WRAPPER_DEFAULT_COMPILER=/usr/apps/gnu/4.8.5/bin/g++
> export CUDA_LAUNCH_BLOCKING=1
>
> export CUDA_TOOLKIT_ROOT_DIR=/opt/cudatoolkit-7.0
> export CUDA_BIN_PATH=/opt/cudatoolkit-7.0
>
> TRILINOS_SOURCE_PATH=/p/lscratche/hysom/trilinos-12.10.1-Source
> TRILINOS_INSTALL_PATH=/p/lscratchf/hysom/trilinos_cuda
>
> EXTRA_ARGS=$@
>
> rm -f CMakeCache.txt
>
> cmake  \
>             -D CMAKE_CXX_COMPILER:STRING="/usr/apps/gnu/4.8.5/bin/g++" \
>             -D CMAKE_C_COMPILER:STRING="/usr/apps/gnu/4.8.5/bin/gcc" \
>             -DCMAKE_CXX_FLAGS="-g" \
>              -DTPL_ENABLE_CUDA=ON \
>              -DKokkos_ENABLE_Cuda=ON \
>             -DKokkos_ENABLE_Cuda_UVM=ON \
>             -D Trilinos_ENABLE_Fortran:BOOL=ON \
>             -D Trilinos_ASSERT_MISSING_PACKAGES=OFF \
>             -D Teuchos_ENABLE_COMPLEX:BOOL=OFF \
>             -D Tpetra_INST_COMPLEX_DOUBLE:BOOL=OFF \
>             -D Trilinos_ENABLE_ThreadPool:BOOL=OFF \
>             -D Trilinos_ENABLE_TEUCHOS_TIME_MONITOR:BOOL=ON \
>             -D TPL_ENABLE_Pthread:BOOL=OFF  \
>             -D TPL_ENABLE_MPI:BOOL=OFF \
>             -D TPL_ENABLE_TBB:BOOL=OFF \
>             -D Trilinos_ENABLE_Kokkos:BOOL=ON \
>             -D Kokkos_ENABLE_EXAMPLES:BOOL=ON \
>             -D Kokkos_ENABLE_TESTS:BOOL=OFF \
>             -D Tpetra_DefaultNode:STRING=Kokkos::Compat::KokkosCudaWrapperNode \
>             -D Trilinos_ENABLE_OpenMP:BOOL=ON \
>             -D Trilinos_ENABLE_CHECKED_STL:BOOL=OFF \
>             -D Trilinos_ENABLE_EXPLICIT_INSTANTIATION:BOOL=OFF \
>             -D Trilinos_ENABLE_ALL_OPTIONAL_PACKAGES:BOOL=OFF \
>             -D Trilinos_ENABLE_Anasazi:BOOL=ON \
>             -D Trilinos_ENABLE_Tpetra:BOOL=ON \
>             -D Trilinos_ENABLE_Zoltan2:BOOL=OFF \
>             -D Trilinos_ENABLE_MueLu:BOOL=OFF \
>             -D Zoltan2_ENABLE_TESTS:BOOL=OFF \
>             -D MueLu_ENABLE_TESTS:BOOL=OFF \
>             -D Tpetra_ENABLE_TESTS:BOOL=OFF \
>             -D Trilinos_ENABLE_EXAMPLES:BOOL=OFF \
>             -D CMAKE_INSTALL_PREFIX:STRING=${TRILINOS_INSTALL_PATH} \
>             -D CMAKE_BUILD_TYPE:STRING=RELEASE \
>             -D Kokkos_ENABLE_Serial:BOOL=OFF \
>             -D DART_TESTING_TIMEOUT:STRING=600 \
>             -D Trilinos_ENABLE_Belos=ON \
>             -D Trilinos_ENABLE_Ifpack2=ON \
>             ${EXTRA_ARGS} \
>             ${TRILINOS_SOURCE_PATH}
>
> _______________________________________________
> Trilinos-Users mailing list
> Trilinos-Users at trilinos.org
> https://trilinos.org/mailman/listinfo/trilinos-users
> _______________________________________________
> Trilinos-Users mailing list
> Trilinos-Users at trilinos.org
> https://trilinos.org/mailman/listinfo/trilinos-users