[Trilinos-Users] Typical memory usage of Epetra_FECrsMatrix

[Hoemmen, Mark]

On May 21, 2013, at 12:00 PM, <trilinos-users-request at software.sandia.gov>
 <trilinos-users-request at software.sandia.gov> wrote:
> Message: 2
> Date: Tue, 21 May 2013 12:43:06 +0000
> From: "Ramsey, James J CIV (US)" <james.j.ramsey14.civ at mail.mil>
> Subject: [Trilinos-Users] Typical memory usage of Epetra_FECrsMatrix?
> To: "trilinos-users at software.sandia.gov"
> 	<trilinos-users at software.sandia.gov>
> Message-ID:
> 	<B5E7491E7F72164997C336AA7C85A5FC14E0B200 at ucolhp9g.easf.csd.disa.mil>
> Content-Type: text/plain; charset=us-ascii
> 
> I'm trying to get a sense of what the expected memory usage of a Epetra_FECrsMatrix should be, given a certain number of rows and nonzeros per row. I've seen the description of the CRS format at http://netlib.org/utk/papers/templates/node91.html, but it seems to lead to an underestimation of my memory usage, which worries me. So far, using the "massif" tool from valgrind tells me that my program used about 193 MB of heap memory for a finite element problem with 29,791 brick elements and 3 DOF per node, and about 800 MB of the heap for a similar finite element problem, but with 125,000 elements. I know that there are typically 81 nonzeros per row here in these particular problems. Looking at GNOME's system monitor shows me the resident memory usage is more like 500 and 900 MB, respectively. Does this look to be in the ballpark of reasonableness (taking into account the other data structures in the program that also take up memory), or should I be more suspicious?

Hi -- are you measuring peak ("high water mark") memory usage, or memory usage after calling FillComplete()?  Peak memory usage for FECrsMatrix may vary depending on the following:

1. Whether you constructed it with dynamic or static profile (this matters a lot; static profile uses less memory)

2. How many entries per row you preallocated in the constructor

3. Accuracy of your upper bound on the number of entries in each row (a bad guess with dynamic profile may require lots of reallocation)

4. Whether you constructed it with a const graph, or with a column Map

5. How many entries you inserted into nonowned rows (i.e., rows that are not in the row Map on the calling process).  (Zero is best, in which case you could use Epetra_CrsMatrix instead.)

The following technical report explains best practices for maximizing performance and minimizing memory use of Epetra objects:

http://trilinos.sandia.gov/packages/epetra/EpetraPerformanceGuide.pdf

In particular, see Sections 4-6.

Best,
mfh