[Trilinos-Users] SuperLU_dist, KLU and reordering

[Jonas Thies]

Hi David, Erik,

thanks for your advice, maybe I'll contact some sparse direct people 
like the SuperLU developers. My matrix is a 'sparsified' 
Schur-complement used to precondition a Navier-Stokes solver.
I guess the parallel solvers don't perform too well because it is so 
small (20k rows).
MUMPS and all its dependencies are pre-installed on our machine, and 
Amesos has obviously recently been updated, so I'll play with that, too.

regards,
Jonas

David Day wrote:
> Hi Jonas,
> There is no simple answer. I recommend raising the issue with the
> SuperLU_Dist developers: 
>  http://crd.lbl.gov/~xiaoye/SuperLU/
> I would contact them with a description of the problem, answering the
> following questions. Is your matrix symmetric?  If so, is it positive
> definite?  What, approximately, are the number of unknowns and the
> number of nonzeros?  What is the application?  Is the matrix a
> discretization of a differential equation?  of a partial differential
> equation?  What version of SuperLU_Dist are you using?
> 
> Many people in your situation have re-considered iterative linear
> solvers.
> 
> Good luck,
> David Day
> 
> 
> 
> On Thu, 2008-11-27 at 05:54 -0700, Jonas Thies wrote:
>> Hi,
>>
>> can someone give me a hint on SuperLU_Dist in Amesos?
>>
>> One of my systems is currently solved sequentially using KLU,
>> which is very efficient for some reason. I'd like to do it in parallel 
>> now, but SuperLU seems a lot slower even sequentially.
>> So I was wondering if KLU does some reordering by default which SuperLU 
>> doesn't. I'm using the default settings for both solvers, which flags 
>> are worth trying? Or is there a fundamental algorithmic difference that 
>> could make KLU about ten times faster than SuperLU for a particular 
>> sparse nonsymmetric real matrix?
>> Finally I noticed that SuperLU_dist always seems to distribute the 
>> matrix even if I pass in a distributed matrix, is that intended or a 
>> wrong parameter setting?
>> Is it a good idea to use SuperLU_dist or are other parallel sparse 
>> solvers in Amesos a better choice?
>>
>> thanks,
>> Jonas
>>
>> Heroux, Michael A wrote:
>>> Daniele,
>>>
>>> If I understand what you are doing, then one way to eliminate the extra 
>>> copy of x values is to use View mode with the Epetra_Vector objects. 
>>>  You will still have overlappedXExtMap and xExtMap, and the 
>>> corresponding vectors overlappedXExt and xExt.  Furthermore, you would 
>>> create overlappedXExt in the standard way.  However, when you create 
>>> xExt, you should pass in the pointer to the first value in 
>>> overlappedXExt using View mode.
>>>
>>> If you do this, Epetra’s import function is smart enough to detect that 
>>> all but the last value are the same address and it will not do a 
>>> redundant copy.
>>>
>>> A further optimization would be to avoid the import, since you are 
>>> really doing a broadcast.  You could use the Broadcast() method on 
>>> Epetra_Comm to send to all processors and pass a pointer to the end of 
>>>  overlappedXExt vector as the target buffer for the broadcast.
>>>
>>> I hope this makes sense.
>>>
>>> Mike
>>>
>>>
>>> On 11/24/08 6:53 AM, "Daniele Avitabile" <d.avitabile at gmail.com> wrote:
>>>
>>>     Hi everybody,
>>>
>>>     the solution to the PDEs I am solving is stored in an Epetra_Vector
>>>     x, based on a certain linear Epetra_Map xMap. Now, I need to add an
>>>     extra ODE to my system, defined on the boundary. At the present
>>>     stage, whenever I want to compute the right-hand side of my PDE, I
>>>     need to access a new "bordered" Epetra_Vector xExt, based on a new
>>>     map Epetra_Map xExtMap.
>>>
>>>     The problem is that, from each processor, I need access to the last
>>>     component of xExt, the one on the boundary.
>>>
>>>     The way I am doing this is
>>>     1. define an Epetra_Map overlappedXExtMap. All the GID of xExtMap
>>>     are also contained in overlappedXExtMap, plus the last component of
>>>     xExt is replicated on all processors.
>>>     2. define an importer with xExtMap as source and overlappedXExtMap
>>>     as target
>>>     3. each time I evaluate the right-hand side of my equation, use the
>>>     importer.
>>>
>>>     I rely on the importer massively, because I also use it for printing
>>>     purposes: I want to separate the solution in the interior of my PDE
>>>     from the additional scalar component on the boundary.
>>>
>>>     My questions:
>>>     a. do you have a more efficient way to do this? It seems to me that
>>>     I am copying a lot of unnecessary data, namely all the components
>>>     relative to the GID of xEtxMap that are also in overlappedXExtMap.
>>>     b. how does the importer Work? Does he really copy all the elements?
>>>
>>>     Thanks in advance.
>>>
>>>     Daniele
>>>
>>>
>>>
>>> ------------------------------------------------------------------------
>>>
>>> _______________________________________________
>>> Trilinos-Users mailing list
>>> Trilinos-Users at software.sandia.gov
>>> http://software.sandia.gov/mailman/listinfo/trilinos-users
>>
>> _______________________________________________
>> Trilinos-Users mailing list
>> Trilinos-Users at software.sandia.gov
>> http://software.sandia.gov/mailman/listinfo/trilinos-users
> 
> _______________________________________________
> Trilinos-Users mailing list
> Trilinos-Users at software.sandia.gov
> http://software.sandia.gov/mailman/listinfo/trilinos-users