[Trilinos-Users] about Tpetra + Kokkos app built against installed trilinos

[KESTENER Pierre]

Hello again,

I think I misunderstood that nvcc_wrapper specific flags are supposed to be set directly in
CMAKE_CXX_FLAGS variable and not in CUDA_NVCC_FLAGS

Everything is ok that way. 
Sorry for the noise.


________________________________________
De : KESTENER Pierre
Envoyé : mercredi 4 janvier 2017 12:13
À : trilinos-users at trilinos.org
Cc : KESTENER Pierre
Objet : RE:about Tpetra + Kokkos app built against installed trilinos

Hello,

Following up my previous message, I am realizing that setting option
-DCUDA_NVCC_FLAGS=...
passed to cmake in a trilinos configure script has no effect

Whatever the flags passed here, they are never transmitted to nvcc_wrapper later.

Even though, you correctly set CUDA_NVCC_FLAGS (correct cuda architecture, ...),
if you build Kokkos examples through Trilinos/cmake/tribits and then run on a GPU with an architecture different from sm_35 (default of nvcc_wrapper), you will get a warning like this
Kokkos::Cuda::initialize WARNING: running kernels compiled for compute capability 3.5 on device with compute capability 5.0 , this will likely reduce potential performance

Since the flags are not taken into account, nvcc_wrapper uses default values, here architecture sm_35.

Maybe someone could confirm, there is a problem in the way cmake/tribits passes flags down to nvcc_wrapper.

As a suggestion, i would propose to slightly modify nvcc_wrapper:
- if env variable CUDA_NVCC_FLAGS is set, just parse it and use it to build the nvcc_command variable.
That way, I think the change would minimal to make sure the correct flags are passed to nvcc_wrapper


best regards,
Pierre.



________________________________________
De : KESTENER Pierre
Envoyé : mardi 3 janvier 2017 23:41
À : trilinos-users at trilinos.org
Cc : KESTENER Pierre
Objet : about Tpetra + Kokkos app built against installed trilinos

Hello,

The following is a suggestion for a minor improvement in the installed TrilinosConfig.cmake.

On the trilinos tutorial wiki
https://github.com/trilinos/Trilinos_tutorial/wiki/TrilinosHandsOnTutorial
in the section "Learn how to create and use Kokkos with Tpetra."

the suggested CMakeLists.txt's are mostly OK, but if Trilinos has been built with Tpetra+Kokkos and CUDA enabled, i.e. the variable CUDA_NVCC_FLAGS was passed to the main trilinos configure script, activating some nvcc flags
passed to nvcc_wrapper (cuda architecture, uvm, lambda, ...), the precise flags used to build trilinos are not saved
in the installed TrilinosConfig.cmake

I think it would be useful to have those precise flags saved in a variable named something like
Trilinos_CUDA_NVCC_FLAGS
it would make easier afterwards to build a CUDA app against an installed trilinos.

Best regards,

Pierre Kestener.