[Trilinos-Users] Tpetra::Vector::{replace, sumInto}GlobalValue() crashes
Nico Schlömer
nico.schloemer at ua.ac.be
Thu Aug 19 19:00:38 MDT 2010
On Thu, 19 Aug 2010 12:12:25 -0400, "Baker, Christopher G."
<bakercg at ornl.gov> wrote:
> Contributing off-processor entries in CrsMatrix is only supported in
> insertGlobalValues() method. All other methods operate on local matrix
> rows only.
How about columns, though?
Just wrapped up a painful debugging session that shows that if a
Tpetra::CrsMatrix is instantiated with explicit column map,
insertGlobalValues() will only insert values local to the process, and
*silently forget about the rest.
Minimal code example attached: runs as expected on one process, fails
on two or more.
Cheers,
Nico
>
> Chris
>
>
> On 8/19/10 10:52 AM, "Nico Schlömer" <nico.schloemer at ua.ac.be> wrote:
>
>> 1. export from overlapping to non-overlapping (to do the summation
>> of shared values)
>> 2. export (import?) from non-overlapping back to overlapping (to get
>> the summed values on all sharing processors)
>
> I guess that would work.
>
> Just to make sure, Mike mentioned some time ago that
>
>> Tpetra supports off-processor contributions in the standard Tpetra::CrsMatrix class.
>
> Is that via the {replace,sumInto}GlobalValues() methods, or are these
> methods just calling getLocalElement() just like their Tpetra::Vector
> equivalents?
>
> Cheers,
> Nico
>
>
>
>
>
> On Thu, 19 Aug 2010 08:21:46 -0600, "Williams, Alan B"
> <william at sandia.gov> wrote:
>> Nico,
>>
>> I think you need two vectors, one with the overlap map and one with a
>> non-overlapping map. Then you would import/export between those two
>> vectors.
>> I think you would actually have to do two operations:
>> 1. export from overlapping to non-overlapping (to do the summation
>> of shared values)
>> 2. export (import?) from non-overlapping back to overlapping (to get
>> the summed values on all sharing processors)
>>
>> Alan
>>
>>
>>> -----Original Message-----
>>> From: trilinos-users-bounces at software.sandia.gov [mailto:trilinos-
>>> users-bounces at software.sandia.gov] On Behalf Of Nico Schlömer
>>> Sent: Wednesday, August 18, 2010 11:37 PM
>>> To: Baker, Christopher G.
>>> Cc: trilinos-users at software.sandia.gov
>>> Subject: Re: [Trilinos-Users] Tpetra::Vector::{replace,
>>> sumInto}GlobalValue() crashes
>>>
>>> Thanks for the hint, Chris.
>>> In fact, I've been playing around with import/export, but somehow I
>>> can't seem to get things straight. I'm sure you already know what's
>>> wrong when I'm doing the following:
>>> I have a vector x with an overlapping map, and I'd like the sum of all
>>> the overlaps to be present on all nodes. What I did was
>>>
>>> Tpetra::Export<ORD> exporter( x->getMap(), x->getMap() );
>>> x->doExport( *x, exporter, Tpetra::ADD );
>>>
>>> I'm reading: Export all of what's in x into x, and add the overlaps;
>>> apparently it doesn't work this way, though.
>>> Got a quick hint?
>>>
>>> Cheers,
>>> Nico
>>>
>>>
>>>
>>>
>>>
>>> On Wed, 18 Aug 2010 20:45:41 -0400, "Baker, Christopher G."
>>> <bakercg at ornl.gov> wrote:
>>> > My apologies, Nico. You are correct; in both cases, local/global
>>> > replace/sum works only on entries on the node. The only difference is
>>> > a translation between local and global, as you correctly surmised. I
>>> > have added a note to this effect in the documentation of these class
>>> > methods.
>>> >
>>> > If you want to Tpetra to handle the communication for you, I would
>>> > point you towards the import/export capability supported by Vector
>>> and
>>> > MultiVector.
>>> >
>>> > Chris
>>> >
>>> >
>>> > On 8/18/10 7:11 PM, "Nico Schlömer" <nico.schloemer at ua.ac.be> wrote:
>>> >
>>> > Okay so,
>>> >
>>> > I noticed that guarding the {replace, sumInto}GlobalValue() like
>>> >
>>> > ===================== *snip* =====================
>>> > if ( map->getLocalElement(i0) !=
>>> Teuchos::OrdinalTraits<ORD>::invalid()
>>> > )
>>> > x->replaceGlobalValue( i0, val );
>>> > ===================== *snap* =====================
>>> >
>>> > makes the the corruption go away, indicating that the only thing
>>> > {replace, sumInto}GlobalValue() does for me is calling
>>> getLocalElement()
>>> > on the map before insertion.
>>> > Fair enough, I'm gonna need to make sure then that the corresponding
>>> > map allows access to this particular element.
>>> >
>>> > Cheers,
>>> > Nico
>>> >
>>> >
>>> >
>>> >
>>> >
>>> > On Wed, 18 Aug 2010 18:36:24 -0400, Nico Schlömer
>>> > <nico.schloemer at ua.ac.be> wrote:
>>> >> Hi,
>>> >>
>>> >> I'm experiencing some ungraceful crashes here using either of
>>> >>
>>> >> Tpetra::Vector::replaceGlobalValue()
>>> >> Tpetra::Vector::sumIntoGlobalValue()
>>> >>
>>> >> with something like
>>> >>
>>> >> ============================== *snip* ==============================
>>> >> *** glibc detected ***
>>> >> ../build/mpi/executables/piro-driver/piro-driver.exe: double free or
>>> >> corruption (out): 0x00000000028271c0 ***
>>> >> ======= Backtrace: =========
>>> >> /lib/libc.so.6(+0x72966)[0x7f2cefbcb966]
>>> >> /lib/libc.so.6(cfree+0x6c)[0x7f2cefbd071c]
>>> >> [...]
>>> >> ============================== *snap* ==============================
>>> >>
>>> >> Their local equivalents work all right.
>>> >>
>>> >> This is with Trilinos 10.4 on a simple two core build, openmpi
>>> 1.4.1,
>>> >> GCC 4.4.3.
>>> >>
>>> >> Anyone with similar experiences?
>>> >>
>>> >> Cheers,
>>> >> Nico
>>> >>
>>> >>
>>> >> _______________________________________________
>>> >> Trilinos-Users mailing list
>>> >> Trilinos-Users at software.sandia.gov
>>> >> http://software.sandia.gov/mailman/listinfo/trilinos-users
>>> >
>>> >
>>> >
>>> > _______________________________________________
>>> > Trilinos-Users mailing list
>>> > Trilinos-Users at software.sandia.gov
>>> > http://software.sandia.gov/mailman/listinfo/trilinos-users
>>>
>>>
>>>
>>> _______________________________________________
>>> Trilinos-Users mailing list
>>> Trilinos-Users at software.sandia.gov
>>> http://software.sandia.gov/mailman/listinfo/trilinos-users
-------------- next part --------------
CMAKE_MINIMUM_REQUIRED(VERSION 2.8)
FIND_PACKAGE( Trilinos REQUIRED )
SET( CMAKE_C_COMPILER ${Trilinos_C_COMPILER} )
SET( CMAKE_CXX_COMPILER ${Trilinos_CXX_COMPILER} )
SET( CMAKE_Fortran_COMPILER ${Trilinos_Fortran_COMPILER} )
PROJECT( extract CXX Fortran )
INCLUDE_DIRECTORIES ( ${Trilinos_INCLUDE_DIRS}
${Trilinos_TPL_INCLUDE_DIRS} )
LINK_DIRECTORIES( ${Trilinos_LIBRARY_DIRS}
${Trilinos_TPL_LIBRARY_DIRS} )
SET( MY_EXECUTABLE
"crsgraph.exe" )
SET ( extract_SRCS
main.cpp )
ADD_EXECUTABLE( ${MY_EXECUTABLE}
${extract_SRCS} )
SET_TARGET_PROPERTIES( ${MY_EXECUTABLE}
PROPERTIES LINKER_LANGUAGE Fortran )
TARGET_LINK_LIBRARIES( ${MY_EXECUTABLE}
${Trilinos_LIBRARIES}
${Trilinos_TPL_LIBRARIES} )
-------------- next part --------------
#include <Teuchos_DefaultComm.hpp>
#include <Tpetra_CrsMatrix.hpp>
typedef int ORD;
void showMatrix( const Teuchos::RCP<const Tpetra::CrsMatrix<double,ORD> > matrix );
int
main (int argc, char** argv)
{
#ifdef HAVE_MPI
MPI_Init( &argc, &argv );
#endif
// Create a communicator for Tpetra objects
const Teuchos::RCP<const Teuchos::Comm<ORD> > comm =
Teuchos::DefaultComm<int>::getComm();
// allocate map
int NumGlobalElements = 4;
Teuchos::RCP<Tpetra::Map<ORD> > map =
Teuchos::rcp( new Tpetra::Map<ORD>( NumGlobalElements, 0, comm ) );
// allocate matrix
// TODO try ( map, map, maxNumEntriesPerRow ) in the intializer
int maxNumEntriesPerRow = NumGlobalElements;
Teuchos::RCP<Tpetra::CrsMatrix<double,ORD> > matrix =
Teuchos::rcp( new Tpetra::CrsMatrix<double,ORD>( map, map, maxNumEntriesPerRow ) );
// insert the first set of indices
Teuchos::Tuple<ORD,4> indices = Teuchos::tuple( 0, 1, 2, 3 );
Teuchos::Tuple<double,4> values = Teuchos::tuple( 3.0, 3.1, 3.14, 3.141 );
for ( ORD i=0; i<matrix->getNodeNumRows(); i++ )
matrix->insertGlobalValues( matrix->getRowMap()->getGlobalElement(i),
indices,
values
);
matrix->globalAssemble();
matrix->fillComplete();
// show matrix
showMatrix( matrix );
#ifdef HAVE_MPI
MPI_Finalize();
#endif
return( EXIT_SUCCESS );
}
void
showMatrix( const Teuchos::RCP<const Tpetra::CrsMatrix<double,ORD> > matrix )
{
for ( int i=0; i<matrix->getRowMap()->getNodeNumElements(); i++ )
{
Teuchos::ArrayRCP<const ORD> indices;
Teuchos::ArrayRCP<const double> values;
matrix->getLocalRowView( i, indices, values );
for ( int l=0; l<indices.size(); l++ )
std::cout << "G[ " << matrix->getRowMap()->getGlobalElement(i)
<< " , " << indices[l] << " ] = "
<< values[l]
<< std::endl;
}
return;
}
More information about the Trilinos-Users
mailing list