[Trilinos-Users] Belos Orthogonalization Performance

[John Moore]

I've run into a performance bottleneck in my Finite Element code, and it
seems to be in the Orthogonalization procedure. I was hoping that someone
could shed some light into what may be going on.

I'm solving large systems of linear equations that arise from a high order
finite element type of scheme. I have implemented my own Tpetra::operator
and Ifpack2::preconditioner that takes into account the variable block size
structure of the matrix.

Typically, 10-20 GMRES iterations are required to reach convergence for
each system solve. I recycle the solver using
solver->reset(Belos::RecycleSubspace) after each solve. However, I have
noticed two issues:

1. The first several Orthogonalizations are quite fast, as I would expect
them to be (they should be much faster than the matrix-vector product for
only 10 iterations). However, they progressively get slower, eventually
becoming 5-10 times more expensive than the first several calls even though
the number of GMRES iterations remains constant. (The timings were
determined from the output of TimeMonitor). Interestingly, this behavior is
only seen in larger simulations.

2. If I monitor the memory usage of my program, I can see that memory is
allocated during each solve (I would assume for the Krylov subspace). The
amount of memory is proportional to the Restart parameter. Is there a way
to have the solver recycle the subspace? I thought that calling
solver->(reset(Belos::RecycleSubspace) would do the trick, but it does not
appear to have any effect. Perhaps the performance issues in 1. are due to
this behavior?

Any thoughts would be greatly appreciated.

Thank you,
John
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://trilinos.org/pipermail/trilinos-users/attachments/20150821/aaf9313e/attachment.html>