[Trilinos-Users] AztecOO Convergence

[Mike Heroux]

Ammar,

Sorry for the delayed reply.  We were all on break the past week.

Here are some responses:

> > Note that (using Ifpack as the preconditioner package) it 
> is possible 
> > to have a different data distribution for the 
> preconditioner than the 
> > rest of the problem.  In other words, you can leave the data 
> > distribution as-is for all of the problem except for the 
> formation and use of the preconditioner.
> > Just make sure that the domain and range maps of the preconditioner 
> > are compatible with the data distribution of the original 
> problem.  If 
> > you decide this is a route to take, I can point you to some 
> examples.  
> > Since the preconditioner is the only phase whose robustness is 
> > affected by data distribution, this is an effective way of 
> addressing this problem.
> 
> Agreed. Can you please point to good starting points?

After looking at this a bit more, I think you are probably better off, at
least initially, redistributing your original user matrix so that it has the
distribution that you want for the preconditioner.  It is still possible to
leave the vectors with their original distribution by calling the version of
Epetra_CrsMatrix.FillComplete that takes the domain and range map arguments,
making sure that the maps you pass in for the domain and range are those of
your vectors.

The reason it is best to use this approach for now is that most of Ifpack's
user-friendly preconditioner setup functions use the distribution of the
input matrix as the distribution of the preconditioner.  It is possible to
use the preconditioners at a lower level, but I suggest you get the basics
working first.

> 
> 
> > This technique also works for Amesos (see next).
> 
> I overlooked Amesos. I thought that Aztec was the only solver 
> interface (maybe because it has more publicity?). I can see 
> now that there are several solver packages. I'll start 
> learning more about how to use Amesos.
> 
> 
> > Finally, if the target problem for this solve is fairly small, a 
> > direct solver is probably best.  There are a number of 
> parallel direct 
> > solvers available via the Trilinos package Amesos.  Amesos 
> provides a 
> > consistent interface to many of the popular direct sparse solvers, 
> > including UMFPACK, SuperLU (serial and distributed), DSCPACK and 
> > other.  Amesos is quite easy to use, however all solvers except KLU 
> > must be built independently from Trilinos since they are 
> 3rd-party software.
> 
> The web page says that KLU is a serial direct solver, so I 
> assume that I cannot use it in my parallel program if I'm 
> running on more than one node. 9 other solvers are listed. 
> Can you recommend one of them at this time so I can ask our 
> cluster admin to install? Perhaps the one that is most 
> commonly used in the HPC community. Would be nice if there is 
> a comparison grid to help choosing.

Even though KLU is serial, you may pass a distributed matrix to Amesos and
it will redistribute the matrix to a single node, solve the problem and then
return the solution as a distributed vector.  

There is a fairly complete breakdown of third-party solver capabilities on
the following page (the link may wrap, so you will have to cut-and-paste it
by pieces into your browser address bar):

http://software.sandia.gov/trilinos/packages/docs/r6.0/packages/amesos/doc/h
tml/index.html

Mike