[Trilinos-Users] [EXTERNAL] coupling Kokkos and Trilinos

[BLOCH Helene]

Hi Siva,

Thank you for your answer

1. We want to update only the values of the matrix, its structure does not change. We create the matrix with a const CrsGraph. Instead of using "replaceGlobalValues", we would like to use "replaceValues" from KokkosSparse.

2.  We can solve the linear system if we use a preconditioner from Ifpack2. If the matrix is not updated correctly, then Belos crashes because of NaN.

We are only using two maps because it is the only way we found to deal with our MPI domain decomposition, but we are open to any suggestion.

Hélène


________________________________________
De : Rajamanickam, Sivasankaran [srajama at sandia.gov]
Envoyé : lundi 24 juin 2019 17:55
À : BLOCH Helene; trilinos-users at trilinos.org
Objet : Re: [EXTERNAL] [Trilinos-Users] coupling Kokkos and Trilinos

Hi Helene
  Can you answer these questions ?

  1. What do you mean by update the matrix. It appears all you need to update is the values of the matrix. Is this correct ?
   2. Were you able to run one solve without update correctly ?

I ask these because it appears the error is pointing to something in your maps. This may not be a device update issue.

-Siva

________________________________________
From: Trilinos-Users <trilinos-users-bounces at trilinos.org> on behalf of BLOCH Helene <helene.bloch at cea.fr>
Sent: Monday, June 24, 2019 2:51 AM
To: trilinos-users at trilinos.org
Subject: [EXTERNAL] [Trilinos-Users] coupling Kokkos and Trilinos

Hi,

We are using Kokkos and Trilinos, especially the packages Tpetra, Belos and MueLu, and we have encountered some difficulties to update the matrix on the device.

 We are currently trying to couple Trilinos with an existing code parallelized with Kokkos and MPI. This code implements an explicit finite volume scheme on a fixed Cartesian grid, with an MPI domain decomposition with ghost cells. We are trying to add an implicit solver, for the finite volume scheme, using backward Euler time integration. We are solving a sparse matrix, using Tpetra, with a linear solver from Belos, with an AMG preconditioner from MueLu. At each iteration of a time loop, we have to
- call the explicit solver
- update the matrix with data coming from the explicit solver
- solve the linear system
- update the solution of the explicit solver with the solution of the implicit solver
As the data coming from the explicit solver lies on the device, it would be great to update the matrix on the device, using methods from KokkosSparse::CrsMatrix. Because we already have a domain decomposition with ghost cells, the matrix has a row map and a column map that are not the same. A first test is performed on the 2D Poisson equation, MueLu gives the following error:
Throw test that evaluated to true: !goodMap
MueLu: TentativePFactory_kokkos: for now works only with good maps (i.e. "matching" row and column maps)
Here are our questions:
- did we miss some examples on the Trilinos wiki to update the matrix on the device ?
- assuming the only way to deal with the domain decomposition is to use two different maps, is there a way to still use MueLu as a preconditioner ?

Thank you for your help

Hélène Bloch

_______________________________________________
Trilinos-Users mailing list
Trilinos-Users at trilinos.org
https://trilinos.org/mailman/listinfo/trilinos-users