[Trilinos-Users] Parallel Performance and non-continuous block layout

[Erik Boman]

Scot,

In general, the data distribution (maps) will affect parallel
performance. You want to minimize communication between processors. I'm
not sure how to interpret the figures below. If the global numbers
correspond to the coordinates along the axis of a 1D PDE problem, then
clearly the contiguous map (top) is better than the non-contiguous map
(bottom), since there's less communication. However, if you're simply
relabeling your variables then I agree with Mike the difference should
be small (minor difference due to cache effects is possible).

Erik

On Mon, 2010-10-04 at 10:57 -0600, M. Scot Breitenfeld wrote:
>   On 10/04/2010 11:54 AM, Heroux, Michael A wrote:
> > Scot,
> >
> > The performance difference continuous and non-continuous should be minimal,
> > for sufficiently large problems.
> >
> > Regarding your last questions, I think Alan Williams can best answer it.  I
> > assume you are using the FECrsMatrix class, right?
> Correct, I'm using FECrsMartrix.
> 
> 
> > Mike
> >
> >
> > On 10/4/10 11:29 AM, "M. Scot Breitenfeld"<brtnfld at uiuc.edu>  wrote:
> >
> >>    Does it effect the parallel performance to have non-continuous nodal
> >> global ordering? For example, in 1D if I have (2 procs),
> >>
> >> 1    2    3   4    5   6   7   8
> >> o    o   o   o  | o   o   o  o
> >>
> >> then the processors will have continuous blocks of rows (proc 0, 0-3:
> >> proc 1: 4-7) or the other case:
> >>
> >> 1    4    5   8    3   6   7   2
> >> o    o   o   o  | o   o   o  o
> >>
> >> where proc 0 has rows 0,3,4,7 and proc 1 has rows 2, 5, 7, 1.
> >> Also, my method requires nodes in the region past partition boundary
> >> (for example nodes 3,6,7 contribute to proc 0), not just the nodes
> >> directly adjacent to the partition boundary.
> >>
> >> And another question, can I do my calculations for partition boundary
> >> nodes (nodes involved with communication) and then: call
> >> GlobalAssemble,  do the calculations for my interior nodes, and call
> >> FillComplete ?
> >>
> >> Thanks,
> >> Scot
> >>
> >>
> >>
> >>
> >>
> >> _______________________________________________
> >> Trilinos-Users mailing list
> >> Trilinos-Users at software.sandia.gov
> >> http://software.sandia.gov/mailman/listinfo/trilinos-users
> >
> >
> 
> 
> _______________________________________________
> Trilinos-Users mailing list
> Trilinos-Users at software.sandia.gov
> http://software.sandia.gov/mailman/listinfo/trilinos-users