[Trilinos-Users] Question about inserting a row not owned by a processor.
Williams, Alan B
william at sandia.gov
Wed Jun 16 13:43:10 MDT 2010
When adding off-processor values to an Epetra_FEVector, you have to use one of the Epetra_FEVector member functions, such as SumIntoGlobalValues or ReplaceGlobalValues. Otherwise you're accessing memory or methods of the base-class (Epetra_MultiVector) which doesn't support off-processor contributions.
Alan
> -----Original Message-----
> From: M. Scot Breitenfeld [mailto:brtnfld at uiuc.edu]
> Sent: Wednesday, June 16, 2010 1:38 PM
> To: Williams, Alan B
> Cc: trilinos-users at software.sandia.gov
> Subject: Re: [Trilinos-Users] Question about inserting a row not owned
> by a processor.
>
> Alan, thanks that does seem to work for the K matrix, but I'm having
> problems getting the rhs vector to work.
>
> I changed: globalRhs_ = Teuchos::rcp(new
> Epetra_Vector(*globalRowMap_));
> to : globalRhsFE_ = Teuchos::rcp(new
> Epetra_FEVector(*globalRowMap_));
>
> where
>
> globalRowMap_ = Teuchos::rcp(new Epetra_Map(-1, numUnks,
> globalGIDList.Values(), 1, comm_));
>
> but when I try:
>
> int BasicLinProbMgr::insertRhsValue(int GID, double value) {
> int rhsLID = globalRowMap_->LID(GID); // Get solver LID
> if (rhsLID==-1) {
> ierr += rhsLID;
> }
> else
>
> I changed: *globalRhs_)[rhsLID] += value
> to: *((*globalRhsFE_)[rhsLID]) += value;
> return(ierr);
> }
>
> It seg faults at: *((*globalRhsFE_)[rhsLID]) += value. It loops twice,
> rhsLID = 0,1 then on 2 it seg faults.
>
> For rhsLID=2 it passes the tests in:
>
> T& RCP<T>::operator*() const
> {
> debug_assert_not_null();
> debug_assert_valid_ptr();
> return *ptr_;
> }
>
> returns to *((*globalRhsFE_)[rhsLID]) += value and then crashes.
>
> Is there something additional I need to do to use Epetra_FEVector as
> opposed to Epetra_Vector
>
> Here is the state of *globalRhsFE before it segfaults:
>
> *globalRhsFE_ = class Epetra_FEVector
> myFirstID_ 1
> myNumIDs_ 126
> nonlocalIDs_ NULL
> nonlocalElementSize_ NULL
> numNonlocalIDs_ 0
> numNonlocalIDsAlloc_ 0
> nonlocalCoefs_ 0xa382058
> *((*globalRhsFE_).nonlocalCoefs_) NULL
> numNonlocalCoefs_ 0
> numNonlocalCoefsAlloc_ 0
> ignoreNonLocalEntries_ false
> Values_ 0xa382638
> *((*globalRhsFE_).Values_) 0
> Pointers_ 0xa382048
> *((*globalRhsFE_).Pointers_) 0xa382638
> *(*((*globalRhsFE_).Pointers_)) 0
> MyLength_ 126
> GlobalLength_ 126
> NumVectors_ 1
> UserAllocated_ false
> ConstantStride_ true
> Stride_ 126
> Allocated_ true
> DoubleTemp_ NULL
> Vectors_ NULL
> Util_ = class Epetra_Util
> Seed_ 1714636915
> static const double chopVal_
> Map_ = class Epetra_BlockMap
> BlockMapData_ 0xa380788
> *(((*globalRhsFE_).Map_).BlockMapData_) = class Epetra_BlockMapData
> Comm_ 0xa38089c
> *((*(((*globalRhsFE_).Map_).BlockMapData_)).Comm_) = class Epetra_Comm
> Directory_ NULL
> LID_ = class Epetra_IntSerialDenseVector
> CV_ Copy
> A_Copied_ false
> M_ 0
> N_ 1
> LDA_ 0
> A_ NULL
> Label_ 0xa380010="Epetra::IntSerialDenseVector"
> *(((*(((*globalRhsFE_).Map_).BlockMapData_)).LID_).Label_) 'E'
> static int TracebackMode
> MyGlobalElements_ = class Epetra_IntSerialDenseVector
> CV_ Copy
> A_Copied_ true
> M_ 126
> N_ 1
> LDA_ 126
> A_ 0xa380bd0
> *(((*(((*globalRhsFE_).Map_).BlockMapData_)).MyGlobalElements_).A_)
> 1
> Label_ 0xa3808c0="Epetra::IntSerialDenseVector"
> static int TracebackMode
> FirstPointInElementList_ = class Epetra_IntSerialDenseVector
> CV_ Copy
> A_Copied_ false
> M_ 0
> N_ 1
> LDA_ 0
> A_ NULL
> Label_ 0xa3808e8="Epetra::IntSerialDenseVector"
> *(((*(((*globalRhsFE_).Map_).BlockMapData_)).FirstPointInElementList_).
> Label_)
> 'E'
> static int TracebackMode
> ElementSizeList_ = class Epetra_IntSerialDenseVector
> CV_ Copy
> A_Copied_ false
> M_ 0
> N_ 1
> LDA_ 0
> A_ NULL
> Label_ 0xa380910="Epetra::IntSerialDenseVector"
> *(((*(((*globalRhsFE_).Map_).BlockMapData_)).ElementSizeList_).Label_)
> 'E'
> static int TracebackMode
> PointToElementList_ = class Epetra_IntSerialDenseVector
> CV_ Copy
> A_Copied_ false
> M_ 0
> N_ 1
> LDA_ 0
> A_ NULL
> Label_ 0xa380938="Epetra::IntSerialDenseVector"
> static int TracebackMode
> NumGlobalElements_ 126
> NumMyElements_ 126
> IndexBase_ 1
> ElementSize_ 1
> MinMyElementSize_ 1
> MaxMyElementSize_ 1
> MinElementSize_ 1
> MaxElementSize_ 1
> MinAllGID_ 1
> MaxAllGID_ 126
> MinMyGID_ 1
> MaxMyGID_ 126
> MinLID_ 0
> MaxLID_ 125
> NumGlobalPoints_ 126
> NumMyPoints_ 126
> ConstantElementSize_ true
> LinearMap_ false
> DistributedGlobal_ false
> OneToOne_ true
> LastContiguousGID_ 126
> LastContiguousGIDLoc_ 125
> LIDHash_ NULL
> ReferenceCount_ 10
> Label_ 0xa3821a0="Epetra::Map"
> *(((*globalRhsFE_).Map_).Label_) 'E'
> static int TracebackMode
> Comm_ 0xa38089c
> DistributedGlobal_ false
> Exports_ NULL
> Imports_ NULL
> LenExports_ 0
> LenImports_ 0
> Sizes_ NULL
> Label_ 0xa382188="Epetra::MultiVector"
> *((*globalRhsFE_).Label_) 'E'
> static int TracebackMode
> FlopCounter_ NULL
>
>
>
> On 06/15/2010 01:32 PM, Williams, Alan B wrote:
> > Scot,
> > It's possible that the class Epetra_FECrsMatrix can do what you need.
> It allows for inserting entries that are owned by another processor,
> and then communicates them to that processor during the
> globalAssemble() method.
> > Alan
> >
> >
> >
> >> -----Original Message-----
> >> From: trilinos-users-bounces at software.sandia.gov [mailto:trilinos-
> >> users-bounces at software.sandia.gov] On Behalf Of M. Scot Breitenfeld
> >> Sent: Tuesday, June 15, 2010 10:52 AM
> >> To: trilinos-users at software.sandia.gov
> >> Subject: [Trilinos-Users] Question about inserting a row not owned
> by a
> >> processor.
> >>
> >> Greetings,
> >>
> >> I'll try to explain a simplified version of my situation the best
> that
> >> I
> >> can. Lets say I have a line discretized as follows:
> >>
> >> Proc 0 | Proc 1
> >> o------o------o---|---o-----o-----o
> >> 1 2 3 4 5 6 (global number)
> >>
> >> where o are nodes (there are no elements for this method) and there
> is
> >> only 1 d.o.f at each node
> >>
> >> For Processor 0 I register: as my global node list: 1, 2, 3, 4
> >> as the row list: 1, 2, 3,
> 4
> >> as my owned node list: 1,
> 2,
> >> 3
> >>
> >> Proc 0
> >> o------o------o---|---o
> >> 1 2 3 4 (global number)
> >> 1 2 3 4 (rows in global list)
> >>
> >> For Processor 1 I register: as my global node list: 4, 5, 6, 3
> >> as the row list: 4, 5, 6,
> 3
> >> as my owned node list: 4,
> 5,
> >> 6
> >>
> >> Proc 1
> >> o---|---o------o------o
> >> 3 4 5 6 (global number)
> >> 3 4 5 6 (rows in global list)
> >>
> >> Now up to this point it works fine in parallel. But for my method I
> >> have
> >> something like this:
> >>
> >> row 1: (T12-T21)*V2+(T13-T31)*V3+(T14-T41)*V4
> >> row 2: (T21-T12)*V2+(T23-T32)*V3+(T24-T42)*V4
> >> row 3: (T31-T13)*V2+(T34-T43)*V3
> >> row 4: (T41-T14)*V2+(T45-T54)*V3+(T34-T43)*V4
> >> ....
> >>
> >> T12, for example, means node 1 is influenced by node 2; meaning
> >> processor 0 will calculate T12, T13, T14 which is fine. But T14 also
> >> needs to be subtracted from row 4. The problem is row 4 (node 4) is
> not
> >> owned by processor 0, it's on processor 1. When I use
> >> InsertGlobalValues
> >> to insert the values in row 4 on processor 0 it will return an
> error.
> >> Do
> >> I have to write mpi code to pass the T14 matrix to processor one to
> >> insert into row 4, or is there a routine that I can use instead, or
> >> maybe I'm missing something.
> >>
> >> Thanks,
> >> Scot
> >>
> >>
> >>
> >>
> >> _______________________________________________
> >> Trilinos-Users mailing list
> >> Trilinos-Users at software.sandia.gov
> >> http://software.sandia.gov/mailman/listinfo/trilinos-users
> >>
> >
> >
>
More information about the Trilinos-Users
mailing list