[Trilinos-Users] Epetra_Vector with overlap

Pawlowski, Roger P rppawlo at sandia.gov
Wed Jul 7 17:58:07 MDT 2010


Option 2 should not be fragile for NOX algorithms except in one single case.  The generic NOX algorithms have no notion of an overlapped vector or that we are even running in parallel (except for outputting text to cout).  It only uses unique-GIDs map/graphs.  NOX passes back unique GID vectors to the user interface.   The gather/scatters with overlapped objects occur in the user implemented interface for computeF and computeJacobian).  The only place where overlapping might be an issue is for the finite difference coloring algorithm that is specific to the Epetra support.

We should not be initializing nox solvers with overlapped vectors since the size() method that is used in scaling some stopping criteria tolerances will yield different accuracies when running with different numbers of processors.

Roger
________________________________________
From: trilinos-users-bounces at software.sandia.gov [trilinos-users-bounces at software.sandia.gov] On Behalf Of Heroux, Michael A [maherou at sandia.gov]
Sent: Wednesday, July 07, 2010 3:10 PM
To: Salinger, Andrew; Berndt, Markus (LANL)
Cc: trilinos-users at software.sandia.gov
Subject: Re: [Trilinos-Users] Epetra_Vector with overlap

Andy,

I think careful use of Option 2 is OK.  It is used quite frequently for what
Markus is doing.

Mike


On 7/7/10 3:25 PM, "Andrew Salinger" <agsalin at sandia.gov> wrote:

>
> Marcus,
>
> I see two choices:
> In both cases you have 2 Maps: owned (unique GIDs) and overlap.
> (1) Have two copies of the vector (which you want to avoid)
> (2) Construct your overlap map so that the owned elements
> are contiguous. Then your owned vector can be constructed
> as a View inside the overlap vector.
> In your example, if the overlap map on PE1 is
>  4 5 6 7 8 9   or    5 6 7 8 9 4,
> then the owned vector can be constructed with
> the map  5 6 7 8 9 and a pointer to the element
> with "5" in the overlap map.
>
> I think (1) is the Best Practices approach. Option (2)
> is fragile, since an algorithm (say in NOX) that makes a
> copy of an owned vector will allocate it at the shorter
> length. If you try this trick on that vector, it will
> try and write past the allocated length.
>
> Andy
>
> -----Original Message-----
> From: trilinos-users-bounces at software.sandia.gov
> [mailto:trilinos-users-bounces at software.sandia.gov] On Behalf Of Markus Berndt
> Sent: Wednesday, July 07, 2010 9:46 AM
> To: trilinos-users at software.sandia.gov
> Subject: [Trilinos-Users] Epetra_Vector with overlap
>
> Is it possible to set up communication schedules for an Epetra_Vector with
> overlap that can be used to simply update the overlap region in place? The
> documentation for Epetra_Export and Epetra_Import indicates that one of the
> maps that these use to initialize communication patterns must have GIDs that
> are unique. I interpret that as one of the vectors in an Import or Export call
> must have no overlap at all.
>
>
> To illustrate, here is an example:
>
> x_ovl is an Epetra_Vector with overlap such that on PE0 it owns GIDs 0 1 2 3 4
> and has a copy of 5 and on PE1 it owns GIDs 5 6 7 8 9 and has a copy of 4.
>
> How do I set up a communication pattern that
>
> - sends x_ovl(5) from PE1 to PE0 and
> - sends x_ovl(4) from PE0 to PE1
>
> without an intermediate vector that has only the local data and no overlap at
> all.
>
> If this is not possible, what would you consider a best practices approach to
> working with vectors that have overlap that avoids unnecessary copy operations
> between equivalent vectors with and without overlap?
>
> Thanks
>
> - Markus
>
> --
> Markus Berndt, LANL CCS-2, Mail Stop D413, 505-665-4711
>
>
> _______________________________________________
> Trilinos-Users mailing list
> Trilinos-Users at software.sandia.gov
> http://software.sandia.gov/mailman/listinfo/trilinos-users
>
> _______________________________________________
> Trilinos-Users mailing list
> Trilinos-Users at software.sandia.gov
> http://software.sandia.gov/mailman/listinfo/trilinos-users


_______________________________________________
Trilinos-Users mailing list
Trilinos-Users at software.sandia.gov
http://software.sandia.gov/mailman/listinfo/trilinos-users



More information about the Trilinos-Users mailing list