[Trilinos-Users] Tpetra::Vector::{replace, sumInto}GlobalValue() crashes

Thu Aug 19 10:36:10 MDT 2010

> Contributing off-processor entries in CrsMatrix is only supported in
> insertGlobalValues() method. All other methods operate on local matrix
> rows only.

Then maybe the documentation needs an update here.
From
<http://trilinos.sandia.gov/packages/docs/dev/packages/tpetra/doc/html/classTpetra_1_1CrsMatrix.html>
it's not clear that there's a structural difference between
{insert,sumInto,replace}GlobalValues. Also, I'm not quite sure now what
insertGlobalValues() does; Doxygen says "submit entries". Does that mean
that if I submit two entries at (i,j) these two are summed up? Or does
the second one replace the first one? Does the same behavior apply if
the matrix is already fillComplete()d?
To have a true equivalent for Epetra_FECrsMatrix, I guess one would
like to have the entries summed into their global positions.

As a follow-up on Alan suggestion on distributing data across an
overlapping map: Reducing the vector first to a non-overlapping vector
and then map it back to the original overlapping vector works all right,
but seems a bit wasteful in the sense that another vector needs to be
created along the way.

Cheers,
Nico

On Thu, 19 Aug 2010 12:12:25 -0400, "Baker, Christopher G."
<bakercg at ornl.gov> wrote:
> Contributing off-processor entries in CrsMatrix is only supported in
> insertGlobalValues() method. All other methods operate on local matrix
> rows only.
> 
> Chris
> 
> 
> On 8/19/10 10:52 AM, "Nico Schlömer" <nico.schloemer at ua.ac.be> wrote:
> 
>>  1. export from overlapping to non-overlapping (to do the summation
>> of shared values)
>>  2. export (import?) from non-overlapping back to overlapping (to get
>> the summed values on all sharing processors)
> 
> I guess that would work.
> 
> Just to make sure, Mike mentioned some time ago that
> 
>> Tpetra supports off-processor contributions in the standard Tpetra::CrsMatrix class.
> 
> Is that via the {replace,sumInto}GlobalValues() methods, or are these
> methods just calling getLocalElement() just like their Tpetra::Vector
> equivalents?
> 
> Cheers,
> Nico
> 
> 
> 
> 
> 
> On Thu, 19 Aug 2010 08:21:46 -0600, "Williams, Alan B"
> <william at sandia.gov> wrote:
>> Nico,
>>
>> I think you need two vectors, one with the overlap map and one with a
>> non-overlapping map. Then you would import/export between those two
>> vectors.
>> I think you would actually have to do two operations:
>>  1. export from overlapping to non-overlapping (to do the summation
>> of shared values)
>>  2. export (import?) from non-overlapping back to overlapping (to get
>> the summed values on all sharing processors)
>>
>> Alan
>>
>>
>>> -----Original Message-----
>>> From: trilinos-users-bounces at software.sandia.gov [mailto:trilinos-
>>> users-bounces at software.sandia.gov] On Behalf Of Nico Schlömer
>>> Sent: Wednesday, August 18, 2010 11:37 PM
>>> To: Baker, Christopher G.
>>> Cc: trilinos-users at software.sandia.gov
>>> Subject: Re: [Trilinos-Users] Tpetra::Vector::{replace,
>>> sumInto}GlobalValue() crashes
>>>
>>> Thanks for the hint, Chris.
>>> In fact, I've been playing around with import/export, but somehow I
>>> can't seem to get things straight. I'm sure you already know what's
>>> wrong when I'm doing the following:
>>> I have a vector x with an overlapping map, and I'd like the sum of all
>>> the overlaps to be present on all nodes. What I did was
>>>
>>>   Tpetra::Export<ORD> exporter( x->getMap(), x->getMap() );
>>>   x->doExport( *x, exporter, Tpetra::ADD );
>>>
>>> I'm reading: Export all of what's in x into x, and add the overlaps;
>>> apparently it doesn't work this way, though.
>>> Got a quick hint?
>>>
>>> Cheers,
>>> Nico
>>>
>>>
>>>
>>>
>>>
>>> On Wed, 18 Aug 2010 20:45:41 -0400, "Baker, Christopher G."
>>> <bakercg at ornl.gov> wrote:
>>> > My apologies, Nico. You are correct; in both cases, local/global
>>> > replace/sum works only on entries on the node. The only difference is
>>> > a translation between local and global, as you correctly surmised. I
>>> > have added a note to this effect in the documentation of these class
>>> > methods.
>>> >
>>> > If you want to Tpetra to handle the communication for you, I would
>>> > point you towards the import/export capability supported by Vector
>>> and
>>> > MultiVector.
>>> >
>>> > Chris
>>> >
>>> >
>>> > On 8/18/10 7:11 PM, "Nico Schlömer" <nico.schloemer at ua.ac.be> wrote:
>>> >
>>> > Okay so,
>>> >
>>> > I noticed that guarding the {replace, sumInto}GlobalValue() like
>>> >
>>> > ===================== *snip* =====================
>>> > if ( map->getLocalElement(i0) !=
>>> Teuchos::OrdinalTraits<ORD>::invalid()
>>> > )
>>> >     x->replaceGlobalValue( i0, val );
>>> > ===================== *snap* =====================
>>> >
>>> > makes the the corruption go away, indicating that the only thing
>>> > {replace, sumInto}GlobalValue() does for me is calling
>>> getLocalElement()
>>> > on the map before insertion.
>>> > Fair enough, I'm gonna need to make sure then that the corresponding
>>> > map allows access to this particular element.
>>> >
>>> > Cheers,
>>> > Nico
>>> >
>>> >
>>> >
>>> >
>>> >
>>> > On Wed, 18 Aug 2010 18:36:24 -0400, Nico Schlömer
>>> > <nico.schloemer at ua.ac.be> wrote:
>>> >> Hi,
>>> >>
>>> >> I'm experiencing some ungraceful crashes here using either of
>>> >>
>>> >>    Tpetra::Vector::replaceGlobalValue()
>>> >>    Tpetra::Vector::sumIntoGlobalValue()
>>> >>
>>> >> with something like
>>> >>
>>> >> ============================== *snip* ==============================
>>> >> *** glibc detected ***
>>> >> ../build/mpi/executables/piro-driver/piro-driver.exe: double free or
>>> >> corruption (out): 0x00000000028271c0 ***
>>> >> ======= Backtrace: =========
>>> >> /lib/libc.so.6(+0x72966)[0x7f2cefbcb966]
>>> >> /lib/libc.so.6(cfree+0x6c)[0x7f2cefbd071c]
>>> >> [...]
>>> >> ============================== *snap* ==============================
>>> >>
>>> >> Their local equivalents work all right.
>>> >>
>>> >> This is with Trilinos 10.4 on a simple two core build, openmpi
>>> 1.4.1,
>>> >> GCC 4.4.3.
>>> >>
>>> >> Anyone with similar experiences?
>>> >>
>>> >> Cheers,
>>> >> Nico
>>> >>
>>> >>
>>> >> _______________________________________________
>>> >> Trilinos-Users mailing list
>>> >> Trilinos-Users at software.sandia.gov
>>> >> http://software.sandia.gov/mailman/listinfo/trilinos-users
>>> >
>>> >
>>> >
>>> > _______________________________________________
>>> > Trilinos-Users mailing list
>>> > Trilinos-Users at software.sandia.gov
>>> > http://software.sandia.gov/mailman/listinfo/trilinos-users
>>>
>>>
>>>
>>> _______________________________________________
>>> Trilinos-Users mailing list
>>> Trilinos-Users at software.sandia.gov
>>> http://software.sandia.gov/mailman/listinfo/trilinos-users