[Trilinos-Users] Implicit Barriers in Trilinos / Epetra

[Corey Allen]

Chris,

Thanks for your response. My original e-mail was not very clear. It was
obvious to me (logically and confirmed by my own testing) that there had to
be a collective operation (either a barrier or AllReduce as you say) inside
Norm2. I assume that where it make logical sense, many other methods in
Trilinos would have to have implicit barriers (of whatever form). After
doing some searching, it's not always easy to determine which ones do. It
seems to me that as one decides how to use these methods one should take
into consideration how many collectives are implicitly added.

I will look into the Doxygen more as I go along.

Thanks for your help,

Corey



On Wed, Oct 15, 2014 at 9:12 AM, Chris Siefert <csiefer at sandia.gov> wrote:

>  Corey,
>
> There is no race condition in the code you sent.  The Norm2 routine has an
> MPI_Allreduce, which is a blocking collective,
> each MPI rank waits until every other MPI rank has finished the filling of
> x before the norm can be reported.
>
> As for documentation as to which routines have communication (collective
> or otherwise), that should be documented in the
> Trilinos Doxygen, but in many cases (e.g. Norm2) it is not.  This is an
> area we should improve in the future.
>
> -Chris
>
>
>
> On 10/14/2014 01:39 PM, Corey A. Henderson wrote:
>
>    Hello,
>
> I am experimenting with Trilinos and doing the Epetra hands-on. I have
> tried Google to learn if there is a reference indicating which methods have
> implicit MPI Barriers.
>
>  It seems obvious to me that there must be some, especially since no
> barrier is suggested in lesson 2 here:
>
>     // Use local indices to access the entries of x_data.
>     const int localLength = x.MyLength ();
>     for (int localIndex = 0; localIndex < localLength; ++localIndex) {
>       // Add the value of the local index to every entry of x.
>       x[localIndex] += static_cast<double> (localIndex);
>     }
>
>     // Print the norm of x.
>     theNorm = 0.0;
>     (void) x.Norm2 (&theNorm);
>     out << "Norm of x (modified random numbers): " << theNorm << endl;
>
>  It seems to me without a barrier before the Norm2 call (or in Norm2()
> itself) we'd get unexpected results.
>
> Are these barriers documented somewhere that I can use as a reference?
>
> Thanks,
>
> Corey A. Henderson
>  PhD Candidate / NSF Graduate Fellow
>  Dept. of Engineering Physics
>  Univ. of Wisconsin - Madison
>
>
> _______________________________________________
> Trilinos-Users mailing listTrilinos-Users at software.sandia.govhttps://software.sandia.gov/mailman/listinfo/trilinos-users
>
>
>
> _______________________________________________
> Trilinos-Users mailing list
> Trilinos-Users at software.sandia.gov
> https://software.sandia.gov/mailman/listinfo/trilinos-users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://software.sandia.gov/pipermail/trilinos-users/attachments/20141015/cfe9a0a8/attachment.html>