[Trilinos-Users] Handling of ghosts in distributed vectors

Bart Janssens bart.janssens at lid.kviv.be
Fri Apr 19 06:04:02 MDT 2013


Hello,

We are using Epetra_Vectors to store the solution and RHS for our linear
systems. Up to now, we stored the ghost nodes in the vector (all ghosts at
the end). However, I am now trying to directly use the Apply method from
the Epetra_Operator interface, and it seems our vectors are not compatible
with the domain of the matrix.

We call FillComplete() without arguments when building the matrix, which
seems to set both range and domain for the matrix to the rowmap that is
used at construction, which excludes the ghosts.

For the linear solve, we use the Thyra interface, and it seems the
conversion to Thyra vectors discards any ghosts (they "fall off" since
they're at the end). This could explain why Thyra::solve and Thyra::apply
work while Epetra_Operator::Apply called directly doesn't.

If I leave the ghost nodes out of the solution and RHS vectors, both
Epetra_Operator::Apply and the thyra solve appear to work. So it seems it's
better not to include ghost nodes in the vectors, is that correct? This
also makes me wonder: how is the data at the ghost nodes -needed for matrix
application- obtained? Does this involve extra MPI communication?

Also one final question: If we do store ghost nodes in a vector, is there
any way to force an update of their values through MPI?

Cheers,

-- 
Bart
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://software.sandia.gov/pipermail/trilinos-users/attachments/20130419/7a6f562b/attachment.html 


More information about the Trilinos-Users mailing list