[Trilinos-Users] confusion about Epetra maps

[Jonathan Guyer]

In the course of trying to sort out a use case where sometimes I want to build part of a system of equations and calculate its residual and other times I want to build the entire set and solve it, I've run into an issue that I don't understand. At first I thought this was some issue with rectangular matrices, but I see the same issues with square matrices.

I found a presentation by Mike Heroux at http://www.hpcsw.org/presentations/workshops/scalable_tools/heroux_101.pdf that seems to cover the situations I'm interested in and I've attempted to replicate his Case 1, Case 2: Twist 1 and Case 2: Twist 2 with PyTrilinos at http://matforge.org/fipy/wiki/TrilinosMapPuzzle, and I get the same errors.

I am able to do the matvec operation to calculate the residual, but unless I instantiate the `CrsMatrix` with only a row map, `AztecOO` aborts with inexplicable things like "Received elements must be stored after all 1 local elements". Um... OK... I see where that error is generated in `az_comm.c`, but what does it mean?

It seems like the only reason the `CrsMatrix`-with-row-map-only case works at all is because `InsertValues()` magically ignores the fact that the columns overlap other nodes when `Graph_.HaveColMap()` is false. Otherwise, without an overlapping column map, I can't build the matrix properly, but if I use an overlapping column map, I can't solve the thing, no matter what I tell it about its domain and range maps. I've also tried `ReplaceColMap()` to no avail. 

What am I missing?

I'm using:

PyTrilinos 10.2.0
OpenMPI 1.4.1
Apple System Python 2.6.1
Snow Leopard 10.6.6



-- 
Jonathan E. Guyer, PhD
Metallurgy Division
National Institute of Standards and Technology
<http://www.nist.gov/msel/metallurgy>