[Trilinos-Users] Question about inserting a row not owned by a processor.
M. Scot Breitenfeld
brtnfld at uiuc.edu
Thu Jun 17 08:50:10 MDT 2010
oh I think I understand my mistake, for Epetra_FEVector I used the
Epetra_MultiVector operator, so I changed
(*globalRhsFE_)[rhsLID] += value;
which for Epertra_Vector was: double & operator[]
<http://trilinos.sandia.gov/packages/docs/r10.2/packages/epetra/doc/html/classEpetra__Vector.html#44ab8096e05da1662db0693cc885610f>
(int index))
to *((*globalRhsFE_)[rhsLID]) += value; for Epetra_MultiVector double
*& operator[]
<http://trilinos.sandia.gov/packages/docs/r10.2/packages/epetra/doc/html/classEpetra__MultiVector.html#134683eb593b2bfece80a7aa6cce2985>
(int i)
but that operator is for the ith vector in the multi-vector, not for the
ith element in the vector. I see I need to use, as you said,
SumIntoGlobalValue to assign a value to an element in the rhs vector
when using the MultiVector operator:
int ierr = globalRhsFE_->SumIntoGlobalValues(rhsLID, 1, value);
On that note, I don't see a function that accesses an element in a
multi-vector, when globalLhs was Epetra_vector I could just use,
value = (*globalLhs_)[i];
do I have to extract the whole vector using
Epetra_MultiVector::ExtractView to access an element of the vector?
Thanks.
On 06/16/2010 03:08 PM, Williams, Alan B wrote:
> When working with on-processor values, it should be fine if you use Epetra_MultiVector methods, since Epetra_MultiVector is the base-class for Epetra_FEVector.
>
> Are you using the '[]' operator? Note that Epetra_Vector and Epetra_MultiVector both have '[]' and they are not the same. Note that Epetra_Vector is not a base-class for Epetra_FEVector.
>
> Alan
>
>
>
>> -----Original Message-----
>> From: trilinos-users-bounces at software.sandia.gov [mailto:trilinos-
>> users-bounces at software.sandia.gov] On Behalf Of M. Scot Breitenfeld
>> Sent: Wednesday, June 16, 2010 2:00 PM
>> To: trilinos-users at software.sandia.gov
>> Subject: Re: [Trilinos-Users] Question about inserting a row not owned
>> by a processor.
>>
>> Is this only for off-processor values that you have to use
>> Epetra_FEVector member functions, or do you always have to use them?
>> The
>> case that I'm running now seg faults when run on one processor and has
>> no off-processor contributions. It also worked correctly with
>> Epetra_vector and Epetra_crsMatrix since it did not have an issue with
>> off-processor values, so I'm replacing them with the *FE* classes and
>> testing to make sure it still works.
>>
>>
>> On 06/16/2010 02:43 PM, Williams, Alan B wrote:
>>
>>> When adding off-processor values to an Epetra_FEVector, you have to
>>>
>> use one of the Epetra_FEVector member functions, such as
>> SumIntoGlobalValues or ReplaceGlobalValues. Otherwise you're accessing
>> memory or methods of the base-class (Epetra_MultiVector) which doesn't
>> support off-processor contributions.
>>
>>> Alan
>>>
>>>
>>>
>>>
>>>> -----Original Message-----
>>>> From: M. Scot Breitenfeld [mailto:brtnfld at uiuc.edu]
>>>> Sent: Wednesday, June 16, 2010 1:38 PM
>>>> To: Williams, Alan B
>>>> Cc: trilinos-users at software.sandia.gov
>>>> Subject: Re: [Trilinos-Users] Question about inserting a row not
>>>>
>> owned
>>
>>>> by a processor.
>>>>
>>>> Alan, thanks that does seem to work for the K matrix, but I'm having
>>>> problems getting the rhs vector to work.
>>>>
>>>> I changed: globalRhs_ = Teuchos::rcp(new
>>>> Epetra_Vector(*globalRowMap_));
>>>> to : globalRhsFE_ = Teuchos::rcp(new
>>>> Epetra_FEVector(*globalRowMap_));
>>>>
>>>> where
>>>>
>>>> globalRowMap_ = Teuchos::rcp(new Epetra_Map(-1, numUnks,
>>>> globalGIDList.Values(), 1, comm_));
>>>>
>>>> but when I try:
>>>>
>>>> int BasicLinProbMgr::insertRhsValue(int GID, double value) {
>>>> int rhsLID = globalRowMap_->LID(GID); // Get solver LID
>>>> if (rhsLID==-1) {
>>>> ierr += rhsLID;
>>>> }
>>>> else
>>>>
>>>> I changed: *globalRhs_)[rhsLID] += value
>>>> to: *((*globalRhsFE_)[rhsLID]) += value;
>>>> return(ierr);
>>>> }
>>>>
>>>> It seg faults at: *((*globalRhsFE_)[rhsLID]) += value. It loops
>>>>
>> twice,
>>
>>>> rhsLID = 0,1 then on 2 it seg faults.
>>>>
>>>> For rhsLID=2 it passes the tests in:
>>>>
>>>> T& RCP<T>::operator*() const
>>>> {
>>>> debug_assert_not_null();
>>>> debug_assert_valid_ptr();
>>>> return *ptr_;
>>>> }
>>>>
>>>> returns to *((*globalRhsFE_)[rhsLID]) += value and then crashes.
>>>>
>>>> Is there something additional I need to do to use Epetra_FEVector as
>>>> opposed to Epetra_Vector
>>>>
>>>> Here is the state of *globalRhsFE before it segfaults:
>>>>
>>>> *globalRhsFE_ = class Epetra_FEVector
>>>> myFirstID_ 1
>>>> myNumIDs_ 126
>>>> nonlocalIDs_ NULL
>>>> nonlocalElementSize_ NULL
>>>> numNonlocalIDs_ 0
>>>> numNonlocalIDsAlloc_ 0
>>>> nonlocalCoefs_ 0xa382058
>>>> *((*globalRhsFE_).nonlocalCoefs_) NULL
>>>> numNonlocalCoefs_ 0
>>>> numNonlocalCoefsAlloc_ 0
>>>> ignoreNonLocalEntries_ false
>>>> Values_ 0xa382638
>>>> *((*globalRhsFE_).Values_) 0
>>>> Pointers_ 0xa382048
>>>> *((*globalRhsFE_).Pointers_) 0xa382638
>>>> *(*((*globalRhsFE_).Pointers_)) 0
>>>> MyLength_ 126
>>>> GlobalLength_ 126
>>>> NumVectors_ 1
>>>> UserAllocated_ false
>>>> ConstantStride_ true
>>>> Stride_ 126
>>>> Allocated_ true
>>>> DoubleTemp_ NULL
>>>> Vectors_ NULL
>>>> Util_ = class Epetra_Util
>>>> Seed_ 1714636915
>>>> static const double chopVal_
>>>> Map_ = class Epetra_BlockMap
>>>> BlockMapData_ 0xa380788
>>>> *(((*globalRhsFE_).Map_).BlockMapData_) = class Epetra_BlockMapData
>>>> Comm_ 0xa38089c
>>>> *((*(((*globalRhsFE_).Map_).BlockMapData_)).Comm_) = class
>>>>
>> Epetra_Comm
>>
>>>> Directory_ NULL
>>>> LID_ = class Epetra_IntSerialDenseVector
>>>> CV_ Copy
>>>> A_Copied_ false
>>>> M_ 0
>>>> N_ 1
>>>> LDA_ 0
>>>> A_ NULL
>>>> Label_ 0xa380010="Epetra::IntSerialDenseVector"
>>>> *(((*(((*globalRhsFE_).Map_).BlockMapData_)).LID_).Label_) 'E'
>>>> static int TracebackMode
>>>> MyGlobalElements_ = class Epetra_IntSerialDenseVector
>>>> CV_ Copy
>>>> A_Copied_ true
>>>> M_ 126
>>>> N_ 1
>>>> LDA_ 126
>>>> A_ 0xa380bd0
>>>> *(((*(((*globalRhsFE_).Map_).BlockMapData_)).MyGlobalElements_).A_)
>>>> 1
>>>> Label_ 0xa3808c0="Epetra::IntSerialDenseVector"
>>>> static int TracebackMode
>>>> FirstPointInElementList_ = class Epetra_IntSerialDenseVector
>>>> CV_ Copy
>>>> A_Copied_ false
>>>> M_ 0
>>>> N_ 1
>>>> LDA_ 0
>>>> A_ NULL
>>>> Label_ 0xa3808e8="Epetra::IntSerialDenseVector"
>>>>
>>>>
>> *(((*(((*globalRhsFE_).Map_).BlockMapData_)).FirstPointInElementList_).
>>
>>>> Label_)
>>>> 'E'
>>>> static int TracebackMode
>>>> ElementSizeList_ = class Epetra_IntSerialDenseVector
>>>> CV_ Copy
>>>> A_Copied_ false
>>>> M_ 0
>>>> N_ 1
>>>> LDA_ 0
>>>> A_ NULL
>>>> Label_ 0xa380910="Epetra::IntSerialDenseVector"
>>>>
>>>>
>> *(((*(((*globalRhsFE_).Map_).BlockMapData_)).ElementSizeList_).Label_)
>>
>>>> 'E'
>>>> static int TracebackMode
>>>> PointToElementList_ = class Epetra_IntSerialDenseVector
>>>> CV_ Copy
>>>> A_Copied_ false
>>>> M_ 0
>>>> N_ 1
>>>> LDA_ 0
>>>> A_ NULL
>>>> Label_ 0xa380938="Epetra::IntSerialDenseVector"
>>>> static int TracebackMode
>>>> NumGlobalElements_ 126
>>>> NumMyElements_ 126
>>>> IndexBase_ 1
>>>> ElementSize_ 1
>>>> MinMyElementSize_ 1
>>>> MaxMyElementSize_ 1
>>>> MinElementSize_ 1
>>>> MaxElementSize_ 1
>>>> MinAllGID_ 1
>>>> MaxAllGID_ 126
>>>> MinMyGID_ 1
>>>> MaxMyGID_ 126
>>>> MinLID_ 0
>>>> MaxLID_ 125
>>>> NumGlobalPoints_ 126
>>>> NumMyPoints_ 126
>>>> ConstantElementSize_ true
>>>> LinearMap_ false
>>>> DistributedGlobal_ false
>>>> OneToOne_ true
>>>> LastContiguousGID_ 126
>>>> LastContiguousGIDLoc_ 125
>>>> LIDHash_ NULL
>>>> ReferenceCount_ 10
>>>> Label_ 0xa3821a0="Epetra::Map"
>>>> *(((*globalRhsFE_).Map_).Label_) 'E'
>>>> static int TracebackMode
>>>> Comm_ 0xa38089c
>>>> DistributedGlobal_ false
>>>> Exports_ NULL
>>>> Imports_ NULL
>>>> LenExports_ 0
>>>> LenImports_ 0
>>>> Sizes_ NULL
>>>> Label_ 0xa382188="Epetra::MultiVector"
>>>> *((*globalRhsFE_).Label_) 'E'
>>>> static int TracebackMode
>>>> FlopCounter_ NULL
>>>>
>>>>
>>>>
>>>> On 06/15/2010 01:32 PM, Williams, Alan B wrote:
>>>>
>>>>
>>>>> Scot,
>>>>> It's possible that the class Epetra_FECrsMatrix can do what you
>>>>>
>> need.
>>
>>>>>
>>>> It allows for inserting entries that are owned by another processor,
>>>> and then communicates them to that processor during the
>>>> globalAssemble() method.
>>>>
>>>>
>>>>> Alan
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> -----Original Message-----
>>>>>> From: trilinos-users-bounces at software.sandia.gov [mailto:trilinos-
>>>>>> users-bounces at software.sandia.gov] On Behalf Of M. Scot
>>>>>>
>> Breitenfeld
>>
>>>>>> Sent: Tuesday, June 15, 2010 10:52 AM
>>>>>> To: trilinos-users at software.sandia.gov
>>>>>> Subject: [Trilinos-Users] Question about inserting a row not owned
>>>>>>
>>>>>>
>>>> by a
>>>>
>>>>
>>>>>> processor.
>>>>>>
>>>>>> Greetings,
>>>>>>
>>>>>> I'll try to explain a simplified version of my situation the best
>>>>>>
>>>>>>
>>>> that
>>>>
>>>>
>>>>>> I
>>>>>> can. Lets say I have a line discretized as follows:
>>>>>>
>>>>>> Proc 0 | Proc 1
>>>>>> o------o------o---|---o-----o-----o
>>>>>> 1 2 3 4 5 6 (global number)
>>>>>>
>>>>>> where o are nodes (there are no elements for this method) and
>>>>>>
>> there
>>
>>>>>>
>>>> is
>>>>
>>>>
>>>>>> only 1 d.o.f at each node
>>>>>>
>>>>>> For Processor 0 I register: as my global node list: 1, 2, 3, 4
>>>>>> as the row list: 1, 2,
>>>>>>
>> 3,
>>
>>>>>>
>>>> 4
>>>>
>>>>
>>>>>> as my owned node list:
>>>>>>
>> 1,
>>
>>>>>>
>>>> 2,
>>>>
>>>>
>>>>>> 3
>>>>>>
>>>>>> Proc 0
>>>>>> o------o------o---|---o
>>>>>> 1 2 3 4 (global number)
>>>>>> 1 2 3 4 (rows in global list)
>>>>>>
>>>>>> For Processor 1 I register: as my global node list: 4, 5, 6, 3
>>>>>> as the row list: 4, 5,
>>>>>>
>> 6,
>>
>>>>>>
>>>> 3
>>>>
>>>>
>>>>>> as my owned node list:
>>>>>>
>> 4,
>>
>>>>>>
>>>> 5,
>>>>
>>>>
>>>>>> 6
>>>>>>
>>>>>> Proc 1
>>>>>> o---|---o------o------o
>>>>>> 3 4 5 6 (global number)
>>>>>> 3 4 5 6 (rows in global list)
>>>>>>
>>>>>> Now up to this point it works fine in parallel. But for my method
>>>>>>
>> I
>>
>>>>>> have
>>>>>> something like this:
>>>>>>
>>>>>> row 1: (T12-T21)*V2+(T13-T31)*V3+(T14-T41)*V4
>>>>>> row 2: (T21-T12)*V2+(T23-T32)*V3+(T24-T42)*V4
>>>>>> row 3: (T31-T13)*V2+(T34-T43)*V3
>>>>>> row 4: (T41-T14)*V2+(T45-T54)*V3+(T34-T43)*V4
>>>>>> ....
>>>>>>
>>>>>> T12, for example, means node 1 is influenced by node 2; meaning
>>>>>> processor 0 will calculate T12, T13, T14 which is fine. But T14
>>>>>>
>> also
>>
>>>>>> needs to be subtracted from row 4. The problem is row 4 (node 4)
>>>>>>
>> is
>>
>>>>>>
>>>> not
>>>>
>>>>
>>>>>> owned by processor 0, it's on processor 1. When I use
>>>>>> InsertGlobalValues
>>>>>> to insert the values in row 4 on processor 0 it will return an
>>>>>>
>>>>>>
>>>> error.
>>>>
>>>>
>>>>>> Do
>>>>>> I have to write mpi code to pass the T14 matrix to processor one
>>>>>>
>> to
>>
>>>>>> insert into row 4, or is there a routine that I can use instead,
>>>>>>
>> or
>>
>>>>>> maybe I'm missing something.
>>>>>>
>>>>>> Thanks,
>>>>>> Scot
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Trilinos-Users mailing list
>>>>>> Trilinos-Users at software.sandia.gov
>>>>>> http://software.sandia.gov/mailman/listinfo/trilinos-users
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>>
>> _______________________________________________
>> Trilinos-Users mailing list
>> Trilinos-Users at software.sandia.gov
>> http://software.sandia.gov/mailman/listinfo/trilinos-users
>>
>
More information about the Trilinos-Users
mailing list