[Trilinos-Users] [EXTERNAL] Threading for incomplete LU in Ifpack2

[Rajamanickam, Sivasankaran]

CC'ing the users as others can respond in the group as well.


The recommended pattern for solvers/preconditioners

initialize() - call when the graph changes

compute() - call when the matrix values change

apply() - when you apply the preconditioner


In your case, the cost of initialize becomes important. The SGS use case handles this. The coloring in the initialize is also multithreaded and GPU capable. However, this is not a very common use case.


I don't understand your last question.


-Siva


________________________________
From: Wen Yan <wenyan4work at gmail.com>
Sent: Tuesday, December 13, 2016 12:45 PM
To: Rajamanickam, Sivasankaran
Subject: Re: [EXTERNAL] [Trilinos-Users] Threading for incomplete LU in Ifpack2

Currently I'm using a naive implementation. Every timestep generates a new preconditioner, with one initialize() and one compute() call. Because the mesh is adaptively refined at every timestep, the matrix dimension and sparsity pattern are both changing so I am not sure how to reuse the initialized preconditioner from last timestep.

Is it necessary to use a geometric multigrid to keep the matrix structure constant to reuse the initialized matrix?

Thanks,
Wen




On Tue, Dec 13, 2016 at 2:27 PM, Rajamanickam, Sivasankaran <srajama at sandia.gov<mailto:srajama at sandia.gov>> wrote:

Yes, SGS is symmetric Gauss-Seidel.


Making sure multithreading works with third party libraries is really hard. Note that some solvers could use OpenMP and some could use PThreads. It needs to match with how you configure Trilinos first.


Second, we (and lot of other solvers) assume the cost of initialize can be amortized over multiple compute() calls. How many initialize() and compute() calls are you doing ?


-Siva



________________________________
From: Wen Yan <wenyan4work at gmail.com<mailto:wenyan4work at gmail.com>>
Sent: Tuesday, December 13, 2016 11:42 AM
To: Rajamanickam, Sivasankaran
Subject: Re: [EXTERNAL] [Trilinos-Users] Threading for incomplete LU in Ifpack2

Thanks for the explanation, Siva.

I am solving a many body boundary element problem in matrix-free formula with FMM and MPI. Does SGS mean symmetric Gauss-Seidel?

Also, I successfully got the multithreading SuperLU_DIST working in Ifpack2 through the Amesos2 interface. The compute() stage is threaded but the initialize() stage is not, and it takes very long time to initialize. Therefore the threading in compute() phase is no longer useful. Is it an expected behavior?

Thanks,
Wen








On Tue, Dec 13, 2016 at 1:21 PM, Rajamanickam, Sivasankaran <srajama at sandia.gov<mailto:srajama at sandia.gov>> wrote:

Wen Yan,

   The ILU(k) multithreading is being planned for later this year. ILU(t) multithreading is trickier.


   If you want to use a multithreaded local solve right away, you could try SGS now. There is an experimental capability available now. What is the problem you are trying to solve ?


-Siva


________________________________
From: Trilinos-Users <trilinos-users-bounces at trilinos.org<mailto:trilinos-users-bounces at trilinos.org>> on behalf of Wen Yan <wenyan4work at gmail.com<mailto:wenyan4work at gmail.com>>
Sent: Tuesday, December 13, 2016 9:32 AM
To: trilinos-users at trilinos.org<mailto:trilinos-users at trilinos.org>
Subject: [EXTERNAL] [Trilinos-Users] Threading for incomplete LU in Ifpack2

Hi Trilinos Users,

I was wondering if the RILUK and ILUT methods in Ifpack2 support multithreading. The online doxygen webpage for Ifpack2 says " Finally, Ifpack2<https://trilinos.org/docs/dev/packages/ifpack2/doc/html/namespaceIfpack2.html>'s algorithms use and produce Tpetra objects, so you can exploit Tpetra's hybrid (MPI + threads) parallelism features without effort."

However I see that RILUK and ILUT both use only one CPU core in the compute() phase for my test program with a sparse matrix dimension at around 10^4~10^5. The nnz of entries per line is around 10%. I have compiled trilinos with openmp support and enabled openmp by setting OMP_NUM_THREADS. Multithreading is confirmed in other packages in test programs in 'make test'.

Is it possible to activate multithreading RILUK and ILUT in Ifpack2? Or is it possible to call SuperLU_MT or SuperLU_DIST through the Amesos2 interface?

Thank you,
Wen Yan


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://trilinos.org/pipermail/trilinos-users/attachments/20161217/a10043d0/attachment-0001.html>