[Trilinos-Users] AztecOO Convergence

[Mike Heroux]

Ammar,

What you are seeing is the interplay between the preconditioner and
restarted GMRES.  As has been discussed on this list before, overlapping
Schwarz preconditioners become more "Jacobi-like" as the number of domains
increase for a fixed-sized problem.  In your case, as the number of
processors increase, and therefore the preconditioner get weaker, GMRES must
work harder.  Why you are seeing failure at 10 processors is because you
finally reach an iteration count where GMRES by default does a "restart".
Restarting is a efficiency feature of GMRES that attempts to keep the cost
of iterations in check by occasionally computing the current update to the
solution, adding it to the initial guess and then starting over.
Unfortunately for some difficult problems, restarting can actually prohibit
convergence.  This is your case.

If you add a statement:

 solver.SetAztecOption(AZ_kspace, 100); // At 10 processors, you could set
this to 38

 prior to calling solver.Iterate()

 you will see convergence for 10 processors (I got 38 iterations).

As a general rule, I like to avoid restarting GMRES if at all possible,
since this gives the best convergence.  As you have seen, for difficult
problems, the residual will be at 10e-1 for many iterations and then fall to
near zero in a few iterations.  If you restart, you will miss this
opportunity.

Mike

> -----Original Message-----
> From: Ammar T. Al-Sayegh [mailto:alsayegh at purdue.edu] 
> Sent: Friday, December 16, 2005 3:08 PM
> To: trilinos-users at software.sandia.gov
> Subject: Re: [Trilinos-Users] AztecOO Convergence
> 
> Hello All,
> 
> I'm still trying to figure out how to get more consistent 
> AztecOO results with different number of processors. To 
> simplify the problem, I wrote this short code.
> 
> int main(int argc, char *argv[])
> {
>     // init mpi and vars
>     double norm2;
>     MPI_Init(&argc,&argv);
>     Epetra_MpiComm Comm(MPI_COMM_WORLD);
>     Epetra_Map Map(300, 3, Comm);
>     Epetra_Vector *b;
>     Epetra_Vector *x = new Epetra_Vector(Map);
>     Epetra_CrsMatrix *A;
> 
>     // read A & B and solve linear problem
>     MatrixMarketFileToVector("b", Map, b);
>     MatrixMarketFileToCrsMatrix("A", Map, A);
>     Epetra_LinearProblem problem(A, x, b);
>     AztecOO solver(problem);
>     solver.Iterate(1000, 10e-6);
>     x->Norm2(&norm2);
>     cout << norm2 << endl;
> 
>     // finalize mpi
>     MPI_Finalize();
> }
> 
> The code reads A and b and solves for x, then displays the 
> norm2 of x. What I need to achieve is to find the solver that 
> will give me identical number of iterations and norm2 
> regardless of the number of processors I'm using. So far, I 
> haven't been able to get that with all the tests I did with 
> AztecOO. Following are sample results for this code with the 
> attached A and b files:
> 
> 1P)  947.26756007660583 [1 iter; 0.003615 sec]
> 2P)  947.26754915839501 [6 iter; 0.008490 sec]
> 4P)  947.26757223410937 [14 iter; 0.115180 sec]
> 6P)  947.26756549199729 [22 iter; 0.009980 sec]
> 8P)  947.26755742820296 [30 iter; 0.117318 sec]
> 10P) Nonconvergent!
> 
> As you see, not only the solution varies with the number of 
> processors, but it becomes nonconvergent when we reach 10 processors.
> 
> Any suggestion on how I can modify this code, either by using 
> different AztecOO options or by using a different solver, so 
> that I can get solution path and results insensitive to the 
> number of processors?
> 
> Thanks.
> 
> 
> -ammar 
>