[Trilinos-Users] Different results from AztecOO/ML when running serial vs parallel

[Eric Marttila]

I'm using AztecOO with ML to perform linear solves within a non-
linear outer iteration loop.  Experimentation has shown that I 
get the best overall convergence (i.e. fastest solution) of my 
non-linear solve if use a small number of iterations in AztecOO 
when doing each linear solve.

The problem I have is that when running the code with different 
numbers of processes I see significantly different results from the 
linear solve.

I've attached a small code sample that exercises this issue by 
performing a single linear solve.  When running this code with 
np=1 and np=4, I see differences in results that are about 0.5%.

If I let AztecOO do more iterations, most of the differences go 
away... but this isn't what I want to do since it causes my total 
non-linear solution time to increase.

I was not expecting to see the 0.5% difference in results when 
running serial vs parallel.  I was expecting differences that are 
closer to machine precision (like I see when I have AztecOO 
iterate more).  Is what I'm seeing the expected behavior? Or am I 
perhaps doing something wrong in setting up AztecOO?

Thanks,
--Eric

p.s. Attached is my source code, as well as a separate .tar.gz 
that contains the input files and sample output files.

-- 
Eric A. Marttila
-------------- next part --------------
A non-text attachment was scrubbed...
Name: linear-solve.cpp
Type: text/x-c++src
Size: 3429 bytes
Desc: not available
URL: <http://software.sandia.gov/pipermail/trilinos-users/attachments/20150130/eea4d336/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: linear-solve-w-input-and-output-files.tar.gz
Type: application/x-compressed-tar
Size: 56015 bytes
Desc: not available
URL: <http://software.sandia.gov/pipermail/trilinos-users/attachments/20150130/eea4d336/attachment-0001.bin>