[Trilinos-Users] [EXTERNAL] Memory usage advice

[Heroux, Michael A]

Romain,

I don't know if you got any adequate response to the following question.  I was totally swamped last week through this weekend, so am only now catching up…

5% fill is denser than usual for a sparse representation.  But you should see reasonable performance on a single core and with OpenMP.  If you have some reordering routine, you would probably see some improvement in cache performance, RCM might be a good option.  But this is a secondary optimization for now.

The memory increase you are seeing before your call to instantiate W is probably coming from the request for MyGlobalElements.  This call creates an array of integer values of length NumMyElements on each MPI process.  Epetra_Map does not create this array unless it is require.  However, once it is create, there is no method to delete it.

You could delete your Map object, but since it was passed in to the Epetra_FECrsMatrix constructor, a reference-counted version of Map's data will still persist as a part of the Epetra_Map that Epetra_FECrsMatrix creates internally (Epetra_Map uses reference counting of map state when one map is copied to another, making subsequent copies very memory efficient).

A couple of questions:


  *   MyGlobalElements is not very large relative to other objects.  For example, X will require twice as many bytes (8 bytes per double vs. 4 bytes per int for MyGlobalElements).
  *   Do you need MyGlobalElements (My guess is yes, but I thought I would check).
  *   How much does memory increase after instantiating W?  This is where I would expect the memory usage to go way up, since you are pre-allocating memory using maxNbLR.

Mike

On 12/2/12 4:14 PM, "Veltz Romain" <romain.veltz at inria.fr<mailto:romain.veltz at inria.fr>> wrote:

Hello,

I seek some help for some large scale problem.

I am doing some computation with Epetra on a vector of X size 262144 component. Note that I am doing multicore programming.
I want to compute the Matrix-Vector product W*X where W is a Epetra_CrsMatrix. In my problem, roughly 5% of each row are nonzero.
Hence, my first question is:
- Q1: Is it too much for sparsity? Should the number of nonzero elements be much smaller?

Then, I create the matrix W:

int NumGlobalElements = nx_*ny_;
Epetra_Map * Map = new Epetra_Map(NumGlobalElements,0,*Comm);
int NumMyElements = Map->NumMyElements();
cout<< "NumMyElements = "<<NumMyElements<<endl;std::flush(cout);
int * MyGlobalElements = Map->MyGlobalElements();
// We ask for maxNbLR nonzero elements per row
cout<<"Creation de matrice...";std::flush(cout);
W = Teuchos::rcp(new Epetra_FECrsMatrix(Copy,*Map,maxNbLR));
cout<<"done.\n";
/*
do some stuff on the rows
*/
boolres = jLR->FillComplete();
if(boolres!=0)
{
cout<<"Pb with jLR->FillComplete();\n";
throw 1;
}
return 0;
}

My problem is that just before  the call
W = Teuchos::rcp(new Epetra_FECrsMatrix(Copy,*Map,maxNbLR));
the memory usage explode. Hence, to save up some memory,my second question is:

-Q2 Can I free the map Map with delete[] Map?

Thank you for your help,

Best regards,

Romain.

_______________________________________________
Trilinos-Users mailing list
Trilinos-Users at software.sandia.gov<mailto:Trilinos-Users at software.sandia.gov>
http://software.sandia.gov/mailman/listinfo/trilinos-users

-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://software.sandia.gov/pipermail/trilinos-users/attachments/20121211/ff4136b8/attachment.html