[Trilinos-Users] epetra_crsmatrix_storage_consumption

[Blahos, Jiri]

Hi Michael,

thanks for your reply. I am aware of the optimise storage method. In my example, I used the static profile with precisely defined numbers of nonzeros for each row and also called optimise storage at the end (not sure if necessary in static profile case, but just to be sure). However, the final matrix still seems to take more space than I would assume (based on my calculations in the previous email), so I wonder what I am missing.

Jiří
________________________________
From: Heroux, Michael A <maherou at sandia.gov>
Sent: 24 September 2019 15:49
To: Blahos, Jiri <j.blahos at imperial.ac.uk>; trilinos-users at trilinos.org <trilinos-users at trilinos.org>
Subject: Re: [Trilinos-Users] epetra_crsmatrix_storage_consumption


Jiri,



Memory complexity of Epetra sparse matrices depends on a few details of how the matrix was constructed.  Generally, the more flexibility you take advantage of in the construction process (relying on Epetra to accept partial contributions and discover the matrix pattern), the more memory used, at least temporarily.



There is a method called OptimizedStorage():



https://trilinos.org/docs/r12.6/packages/epetra/doc/html/classEpetra__CrsMatrix.html#ac54a33eafbaa0a966cf2c0ce454a7d59



that packs the matrix data and removes internal storage not needed for computing with a fully constructed matrix.



I hope this helps.



Mike



From: Trilinos-Users <trilinos-users-bounces at trilinos.org> on behalf of "Blahos, Jiri" <j.blahos at imperial.ac.uk>
Date: Wednesday, September 18, 2019 at 11:44 AM
To: Trilinos Users <trilinos-users at trilinos.org>
Subject: [EXTERNAL] [Trilinos-Users] epetra_crsmatrix_storage_consumption



Hi everyone,



I would like to ask about how much memory Epetra_CrsMatrix takes.



I made a test case where I create a matrix with 104857600 nonzero elements (the matrix has 10 * 1024 * 1024 rows and columns, each row has 10 nonzero elements).

My expectation would be that this matrix would take around 1.17 GB of memory (104857600 * 12 / 1024 / 1024 / 1024). The value 12 comes from 12 = 8 + 4, array of doubles for values and array of ints for column indices. Row offset array is significantly (10x in this case) smaller so it will only increase the total memory by a bit. I construct the matrix with static profile since I know it here. I call fillcomplete with the default value for optimise storage which is true.



However, when I run the test case and observe ram consumption (in htop), the matrix takes in total around 2.2GB. Even if I include the row array and use 8 bytes for int (long long) instead, I get the result of 1.64GB which is still significantly less.

(104857600 * 16 + 1024 * 1024 * 10 * 8) / 1024 / 1024 / 1024 = 1.640625

I am using Trilinos 12.12.1, built in release.



Is there anything that I am missing? Any other allocations that happen inside the matrix object, besides the arrays for values, col indices and row offsets?

Is checking ram in htop not correct?



Note, in practice I use much larger matrices so I wonder why the memory taken is larger than I expect.

I attached the test code if someone wants to run it on their machine.



Regards,

Jiří


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://trilinos.org/pipermail/trilinos-users/attachments/20190927/66c7a359/attachment-0001.html>