[Trilinos-Users] Converting 1D to 2D matrix in Tpetra

Heroux, Michael A maherou at sandia.gov
Mon Jan 5 09:09:10 MST 2015


David,

The import/export facilities are essential 1D in nature, but they can be used to construct the matrix you want, if the overhead of doing so is acceptable to you.

Thinking about this operation as an import is probably easiest.  Here is a sketch of what to do:


  1.  Create an import target map such that for each process the GID list contains the row IDs of any row for which a process has an edge (or column) ID.  Note that, since rows in a 2D decomposition are themselves distributed, each row in the global matrix will be imported to a number of processes, specifically each process for which there is a column in that row.
  2.  Construct a target matrix, specifying both the row and column maps for each process.  Since you know the edges that each process will own, you can construct both the column and row maps for your 2D matrix.  By specifying the column map, you will filter out column entries that come in from the import operation that are not assigned to a specific process in the 2D distribution (since all column entries will be imported for a given row).
  3.  Perform the import operation with your target matrix as the this object.

I don't think Tpetra has been used much in this scenario, although much of this functionality was, at least initially, ported directly from Epetra and this works in Epetra.  If you run into problems, please let us know.

Finally, as I mentioned, you are doing a fair bit of extra data communication and have a fair bit of extra temporary storage needed.  I believe it would be possible to use the existing DistObject framework, upon which the import/export functionality is built, to create efficient 2D versions of this functionality.  It would not be trivial, but it would be much easier that resorting to raw MPI, once you learn the DistObject architecture (which does require some time to learn).

I hope this is helpful.

Mike

From: <Hysom>, "David A." <hysom1 at llnl.gov<mailto:hysom1 at llnl.gov>>
Date: Sunday, January 4, 2015 at 4:18 PM
To: "trilinos-users at software.sandia.gov<mailto:trilinos-users at software.sandia.gov>" <trilinos-users at software.sandia.gov<mailto:trilinos-users at software.sandia.gov>>
Cc: "ayoo at llnl.gov<mailto:ayoo at llnl.gov>" <ayoo at llnl.gov<mailto:ayoo at llnl.gov>>
Subject: [EXTERNAL] [Trilinos-Users] Converting 1D to 2D matrix in Tpetra

Hi,

I'm a novice to Tpetra's import/export capabilities, and would like
to know if there's an easy way to do the following
(ref: http://www.sandia.gov/~srajama/publications/2dpart_SC13.pdf  )

We have a row-partitioned 1D CrsMatrix, and would like to construct a
2D matrix. Using some code that Karen D. gave us a couple of years ago,
a processor (wrt the 1D matrix) can determine which processor should
receive every edge that it (the 1D matrix) owns. However, I think that,
in order to construct the required Map for the 2D case, the receiving
processor (wrt the 2D matrix) needs this information. Is this the case?
If so, is there any (reasonably easy) way to use Tpetra to convey this mapping information, or do we need to write our own low-level MPI calls?

(I suspect I may not have explained this very well, so please ask
for clarification!)

thanks, David
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://software.sandia.gov/pipermail/trilinos-users/attachments/20150105/cb857b50/attachment.html>


More information about the Trilinos-Users mailing list