[Trilinos-Users] Tpetra::VBRMatrix Issues

[John Pease Moore]

Hello trilinos users!

I have a code that currently uses a CrsMatrix for storing my linear system, but I would like to use a Tpetra::VBRMatrix since my matrix consists of large block entries (10x10  to 100x100 block size).
However, I have not been able to get a VBRMatrix to work on more than one processor. Can anyone confirm that Tpetra::VBRMatrix works on more than one processor, as of the current release?

I have tried two assembly approaches, detailed below.

First Approach:
1) Construct a VbrMatrix with a row  BlockMap constructor, with the BlockMap being a uniform, contiguous map.
2) Insert block entries using SumIntoGlobalBlockEntry()
3) Call FillComplete()

Second Approach:
1) Construct a VbrMatrix (which I will call "primary") with a row BlockMap constructor as above.
2) Construct a second VbrMatrix (which I will call "remote") with a row BlockMap containing only the non-local rows on each process.
3) Create an export object to export from the remote matrix to the primary matrix.
4) Sum into either the primary or remote matrix, depending on whether the block belongs to a global or local block row.
5) Call FillComplete() on the remote matrix.
6) Export the remote matrix into the primary matrix.
7) Call FillComplete() on the primary matrix.

Using these two approaches, I get the same(incorrect) assembled matrix as long as I sort the remote block indices first (which doesn't seem like it should be necessary). When the remote indices on each process are not sorted, I get an error like:

Teuchos::ArrayView<char const>::assert_in_range(): Error, [offset,offset+size) = [32,8) does not lie in the range [0,9472)!

So, it seems like something may be wrong with either my implementation or the way VbrMatrix is communicating non-local contributions. I wrote an function to write Vbr matrices in order to gain better insight into what may be going on, and visualized the results in Matlab.

If I write both the primary and remote matrix separately (without exporting the remote matrix into the primary) and sum the two together in Matlab, I get the correct result. So this would seem to indicate that both my primary and remote matrices are correct, but are not being exported correctly.

Does anyone have an idea about what could be going in here? If someone could confirm that VbrMatrix works with the current release, that would be fantastic!

Thank you for your time!

John