[Trilinos-Users] comment with respect to eventual future Epetra-Tpetra interfaces

[Hoemmen, Mark]

On Jan 22, 2014, at 12:54 AM, <trilinos-users-request at software.sandia.gov<mailto:trilinos-users-request at software.sandia.gov>>
 <trilinos-users-request at software.sandia.gov<mailto:trilinos-users-request at software.sandia.gov>> wrote:

Message: 2
Date: Wed, 22 Jan 2014 08:53:30 +0100
From: Riccardo Rossi <rrossi at cimne.upc.edu<mailto:rrossi at cimne.upc.edu>>
Subject: Re: [Trilinos-Users] Trilinos-Users Digest, Vol 101, Issue 9
To: "trilinos-users at software.sandia.gov<mailto:trilinos-users at software.sandia.gov>"
<trilinos-users at software.sandia.gov<mailto:trilinos-users at software.sandia.gov>>
Message-ID:
<CAOVdALxJkRMmAHTowWiJGj2gPnpuCU+mFXG_a89NH7bcQK1kaA at mail.gmail.com<mailto:CAOVdALxJkRMmAHTowWiJGj2gPnpuCU+mFXG_a89NH7bcQK1kaA at mail.gmail.com>>
Content-Type: text/plain; charset="utf-8"

Just as a comment with respect to eventual future Epetra-Tpetra interfaces,

in my opinion it would be rather nice-to-have a performant (ideally
sero-copy) translation of Epetra_CRS to Tpetra, particularly since as i
understand TPetra misses the FECrs matrix capabilities which are very very
handy for FE programs.

Hi Riccardo -- actually, Tpetra::CrsMatrix implements the same functionality as Epetra_FECrsMatrix.  This includes insertion into non-owned matrix rows (via insertGlobalValues and sumIntoGlobalValues) and global assembly (which is done automatically, if necessary, at fillComplete).

The algebraic multigrid package MueLu, the iterative linear solvers package Belos, and the iterative eigensolvers package Anasazi all work with either Epetra or Tpetra objects.  If you don't mind committing to use MueLu and Belos, that would let you try out a complete linear solver stack with minimal code changes.

It is currently possible to wrap an Epetra_(Multi)Vector in a Tpetra::(Multi)Vector.  If there is strong interest, we could implement a zero-copy Tpetra::RowMatrix wrapper for Epetra_CrsMatrix.  This would incur some performance penalties for some Ifpack2 preconditioners, but it would let users try out Ifpack2 with Epetra.

having said this i shall also say that i never used TPetra, but indeed it
would be nice to have a simple way to try out eventual performance
improvements without writing a lot of code...

Last May, we did a major performance comparison of the Epetra and Tpetra solver stacks in an internal finite-element application.  The application used MPI only (no threads), and did many time steps with a nonlinear solve at each time step.  "Solver stack" here means sparse matrices, dense vectors, iterative linear solvers, and preconditioners.  In general, Tpetra sparse matrix-vector multiply and sparse matrix fill are faster or within 10% of Epetra's performance for these kernels.  We exercised Belos GMRES with a single right-hand side, Ifpack2's symmetric Gauss-Seidel (or Jacobi), and MueLu multigrid solves with Chebyshev smoothing.  Everything but MueLu does as well or better than its Epetra solver stack equivalent.  MueLu apply is a bit slower than ML apply, and MueLu setup is a small constant factor slower.  We are working now to make MueLu apply and setup time comparable to ML's.

We have three main tasks at the moment:

  1. Tpetra: refactor to improve hybrid-parallel performance and GPU support
  2. MueLu setup time (e.g., sparse matrix-matrix multiply)
  3. Ifpack2: improvements to other preconditioners (e.g., domain decomposition and ILU)

If you are interested, it would be great if you could share more about your application's needs.

Thanks!
mfh
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://software.sandia.gov/pipermail/trilinos-users/attachments/20140122/e4a73c29/attachment.html