[Trilinos-Users] Question about inserting a row not owned by a processor.
M. Scot Breitenfeld
brtnfld at uiuc.edu
Wed Jun 16 14:00:24 MDT 2010
Is this only for off-processor values that you have to use
Epetra_FEVector member functions, or do you always have to use them? The
case that I'm running now seg faults when run on one processor and has
no off-processor contributions. It also worked correctly with
Epetra_vector and Epetra_crsMatrix since it did not have an issue with
off-processor values, so I'm replacing them with the *FE* classes and
testing to make sure it still works.
On 06/16/2010 02:43 PM, Williams, Alan B wrote:
> When adding off-processor values to an Epetra_FEVector, you have to use one of the Epetra_FEVector member functions, such as SumIntoGlobalValues or ReplaceGlobalValues. Otherwise you're accessing memory or methods of the base-class (Epetra_MultiVector) which doesn't support off-processor contributions.
>
> Alan
>
>
>
>> -----Original Message-----
>> From: M. Scot Breitenfeld [mailto:brtnfld at uiuc.edu]
>> Sent: Wednesday, June 16, 2010 1:38 PM
>> To: Williams, Alan B
>> Cc: trilinos-users at software.sandia.gov
>> Subject: Re: [Trilinos-Users] Question about inserting a row not owned
>> by a processor.
>>
>> Alan, thanks that does seem to work for the K matrix, but I'm having
>> problems getting the rhs vector to work.
>>
>> I changed: globalRhs_ = Teuchos::rcp(new
>> Epetra_Vector(*globalRowMap_));
>> to : globalRhsFE_ = Teuchos::rcp(new
>> Epetra_FEVector(*globalRowMap_));
>>
>> where
>>
>> globalRowMap_ = Teuchos::rcp(new Epetra_Map(-1, numUnks,
>> globalGIDList.Values(), 1, comm_));
>>
>> but when I try:
>>
>> int BasicLinProbMgr::insertRhsValue(int GID, double value) {
>> int rhsLID = globalRowMap_->LID(GID); // Get solver LID
>> if (rhsLID==-1) {
>> ierr += rhsLID;
>> }
>> else
>>
>> I changed: *globalRhs_)[rhsLID] += value
>> to: *((*globalRhsFE_)[rhsLID]) += value;
>> return(ierr);
>> }
>>
>> It seg faults at: *((*globalRhsFE_)[rhsLID]) += value. It loops twice,
>> rhsLID = 0,1 then on 2 it seg faults.
>>
>> For rhsLID=2 it passes the tests in:
>>
>> T& RCP<T>::operator*() const
>> {
>> debug_assert_not_null();
>> debug_assert_valid_ptr();
>> return *ptr_;
>> }
>>
>> returns to *((*globalRhsFE_)[rhsLID]) += value and then crashes.
>>
>> Is there something additional I need to do to use Epetra_FEVector as
>> opposed to Epetra_Vector
>>
>> Here is the state of *globalRhsFE before it segfaults:
>>
>> *globalRhsFE_ = class Epetra_FEVector
>> myFirstID_ 1
>> myNumIDs_ 126
>> nonlocalIDs_ NULL
>> nonlocalElementSize_ NULL
>> numNonlocalIDs_ 0
>> numNonlocalIDsAlloc_ 0
>> nonlocalCoefs_ 0xa382058
>> *((*globalRhsFE_).nonlocalCoefs_) NULL
>> numNonlocalCoefs_ 0
>> numNonlocalCoefsAlloc_ 0
>> ignoreNonLocalEntries_ false
>> Values_ 0xa382638
>> *((*globalRhsFE_).Values_) 0
>> Pointers_ 0xa382048
>> *((*globalRhsFE_).Pointers_) 0xa382638
>> *(*((*globalRhsFE_).Pointers_)) 0
>> MyLength_ 126
>> GlobalLength_ 126
>> NumVectors_ 1
>> UserAllocated_ false
>> ConstantStride_ true
>> Stride_ 126
>> Allocated_ true
>> DoubleTemp_ NULL
>> Vectors_ NULL
>> Util_ = class Epetra_Util
>> Seed_ 1714636915
>> static const double chopVal_
>> Map_ = class Epetra_BlockMap
>> BlockMapData_ 0xa380788
>> *(((*globalRhsFE_).Map_).BlockMapData_) = class Epetra_BlockMapData
>> Comm_ 0xa38089c
>> *((*(((*globalRhsFE_).Map_).BlockMapData_)).Comm_) = class Epetra_Comm
>> Directory_ NULL
>> LID_ = class Epetra_IntSerialDenseVector
>> CV_ Copy
>> A_Copied_ false
>> M_ 0
>> N_ 1
>> LDA_ 0
>> A_ NULL
>> Label_ 0xa380010="Epetra::IntSerialDenseVector"
>> *(((*(((*globalRhsFE_).Map_).BlockMapData_)).LID_).Label_) 'E'
>> static int TracebackMode
>> MyGlobalElements_ = class Epetra_IntSerialDenseVector
>> CV_ Copy
>> A_Copied_ true
>> M_ 126
>> N_ 1
>> LDA_ 126
>> A_ 0xa380bd0
>> *(((*(((*globalRhsFE_).Map_).BlockMapData_)).MyGlobalElements_).A_)
>> 1
>> Label_ 0xa3808c0="Epetra::IntSerialDenseVector"
>> static int TracebackMode
>> FirstPointInElementList_ = class Epetra_IntSerialDenseVector
>> CV_ Copy
>> A_Copied_ false
>> M_ 0
>> N_ 1
>> LDA_ 0
>> A_ NULL
>> Label_ 0xa3808e8="Epetra::IntSerialDenseVector"
>> *(((*(((*globalRhsFE_).Map_).BlockMapData_)).FirstPointInElementList_).
>> Label_)
>> 'E'
>> static int TracebackMode
>> ElementSizeList_ = class Epetra_IntSerialDenseVector
>> CV_ Copy
>> A_Copied_ false
>> M_ 0
>> N_ 1
>> LDA_ 0
>> A_ NULL
>> Label_ 0xa380910="Epetra::IntSerialDenseVector"
>> *(((*(((*globalRhsFE_).Map_).BlockMapData_)).ElementSizeList_).Label_)
>> 'E'
>> static int TracebackMode
>> PointToElementList_ = class Epetra_IntSerialDenseVector
>> CV_ Copy
>> A_Copied_ false
>> M_ 0
>> N_ 1
>> LDA_ 0
>> A_ NULL
>> Label_ 0xa380938="Epetra::IntSerialDenseVector"
>> static int TracebackMode
>> NumGlobalElements_ 126
>> NumMyElements_ 126
>> IndexBase_ 1
>> ElementSize_ 1
>> MinMyElementSize_ 1
>> MaxMyElementSize_ 1
>> MinElementSize_ 1
>> MaxElementSize_ 1
>> MinAllGID_ 1
>> MaxAllGID_ 126
>> MinMyGID_ 1
>> MaxMyGID_ 126
>> MinLID_ 0
>> MaxLID_ 125
>> NumGlobalPoints_ 126
>> NumMyPoints_ 126
>> ConstantElementSize_ true
>> LinearMap_ false
>> DistributedGlobal_ false
>> OneToOne_ true
>> LastContiguousGID_ 126
>> LastContiguousGIDLoc_ 125
>> LIDHash_ NULL
>> ReferenceCount_ 10
>> Label_ 0xa3821a0="Epetra::Map"
>> *(((*globalRhsFE_).Map_).Label_) 'E'
>> static int TracebackMode
>> Comm_ 0xa38089c
>> DistributedGlobal_ false
>> Exports_ NULL
>> Imports_ NULL
>> LenExports_ 0
>> LenImports_ 0
>> Sizes_ NULL
>> Label_ 0xa382188="Epetra::MultiVector"
>> *((*globalRhsFE_).Label_) 'E'
>> static int TracebackMode
>> FlopCounter_ NULL
>>
>>
>>
>> On 06/15/2010 01:32 PM, Williams, Alan B wrote:
>>
>>> Scot,
>>> It's possible that the class Epetra_FECrsMatrix can do what you need.
>>>
>> It allows for inserting entries that are owned by another processor,
>> and then communicates them to that processor during the
>> globalAssemble() method.
>>
>>> Alan
>>>
>>>
>>>
>>>
>>>> -----Original Message-----
>>>> From: trilinos-users-bounces at software.sandia.gov [mailto:trilinos-
>>>> users-bounces at software.sandia.gov] On Behalf Of M. Scot Breitenfeld
>>>> Sent: Tuesday, June 15, 2010 10:52 AM
>>>> To: trilinos-users at software.sandia.gov
>>>> Subject: [Trilinos-Users] Question about inserting a row not owned
>>>>
>> by a
>>
>>>> processor.
>>>>
>>>> Greetings,
>>>>
>>>> I'll try to explain a simplified version of my situation the best
>>>>
>> that
>>
>>>> I
>>>> can. Lets say I have a line discretized as follows:
>>>>
>>>> Proc 0 | Proc 1
>>>> o------o------o---|---o-----o-----o
>>>> 1 2 3 4 5 6 (global number)
>>>>
>>>> where o are nodes (there are no elements for this method) and there
>>>>
>> is
>>
>>>> only 1 d.o.f at each node
>>>>
>>>> For Processor 0 I register: as my global node list: 1, 2, 3, 4
>>>> as the row list: 1, 2, 3,
>>>>
>> 4
>>
>>>> as my owned node list: 1,
>>>>
>> 2,
>>
>>>> 3
>>>>
>>>> Proc 0
>>>> o------o------o---|---o
>>>> 1 2 3 4 (global number)
>>>> 1 2 3 4 (rows in global list)
>>>>
>>>> For Processor 1 I register: as my global node list: 4, 5, 6, 3
>>>> as the row list: 4, 5, 6,
>>>>
>> 3
>>
>>>> as my owned node list: 4,
>>>>
>> 5,
>>
>>>> 6
>>>>
>>>> Proc 1
>>>> o---|---o------o------o
>>>> 3 4 5 6 (global number)
>>>> 3 4 5 6 (rows in global list)
>>>>
>>>> Now up to this point it works fine in parallel. But for my method I
>>>> have
>>>> something like this:
>>>>
>>>> row 1: (T12-T21)*V2+(T13-T31)*V3+(T14-T41)*V4
>>>> row 2: (T21-T12)*V2+(T23-T32)*V3+(T24-T42)*V4
>>>> row 3: (T31-T13)*V2+(T34-T43)*V3
>>>> row 4: (T41-T14)*V2+(T45-T54)*V3+(T34-T43)*V4
>>>> ....
>>>>
>>>> T12, for example, means node 1 is influenced by node 2; meaning
>>>> processor 0 will calculate T12, T13, T14 which is fine. But T14 also
>>>> needs to be subtracted from row 4. The problem is row 4 (node 4) is
>>>>
>> not
>>
>>>> owned by processor 0, it's on processor 1. When I use
>>>> InsertGlobalValues
>>>> to insert the values in row 4 on processor 0 it will return an
>>>>
>> error.
>>
>>>> Do
>>>> I have to write mpi code to pass the T14 matrix to processor one to
>>>> insert into row 4, or is there a routine that I can use instead, or
>>>> maybe I'm missing something.
>>>>
>>>> Thanks,
>>>> Scot
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Trilinos-Users mailing list
>>>> Trilinos-Users at software.sandia.gov
>>>> http://software.sandia.gov/mailman/listinfo/trilinos-users
>>>>
>>>>
>>>
>>>
>>
>
>
>
More information about the Trilinos-Users
mailing list