[Trilinos-Users] Asynchronous Epetra_Import?

[Charles Boivin]

Re: [Trilinos-Users] Asynchronous Epetra_Import?Mike,

Thanks for the quick reply. I do not have hard data to provide. We are currently in an evaluation stage right now as far as the current project is concerned. As you mentioned, overlapping both computation and communication seems to make sense, in principle, so we were going off of that. 

Past projects (not based on Trilinos) that I have worked on have used latency hiding techniques such that I described, but they were always running on relatively small clusters (10-50 cpus) with slower interconnects. I suppose it doesn't really correspond to "most systems" that you "tend to use with Trilinos". 

The current project, however, is intended to run on anything ranging from the small off-the-shelf clusters to larger, more expensive setups. Trilinos seems to fit the bill for a whole set of other requirements we have, so I still think it might be worthwhile to work around that particular issue. It might be interesting to test a few things on that front. I will gladly communicate any results we get.

Thanks again for your reply,

Charles Boivin

----- Original Message ----- 
  From: Heroux, Michael A 
  To: Charles Boivin ; trilinos-users at software.sandia.gov 
  Sent: Tuesday, April 07, 2009 3:06 PM
  Subject: Re: [Trilinos-Users] Asynchronous Epetra_Import?


  Charles,

  The idea of overlapping computation and communication is attractive in principle.  In practice it has been less effective on most systems we tend to use with Trilinos.  The primary reason is that both the computation and communication steps put heavy demands on the local memory system and trying to overlap them can actually result in poorer performance.  As a result, although there is some underlying support in Epetra for what you suggest, we have not exposed in the import/export functions.

  There is evidence to suggest that future systems will benefit from overlapping, so we will re-evaluate our decision for Tpetra (the next generation of data classes).  I don't think we will try to expose this functionality in Epetra, unless there is compelling evidence to do so.

  Do you have a situation where you can see significant advantage from overlapping?  If so, I would like to learn more.

  Thanks.

  Mike


  On 4/7/09 1:38 PM, "Charles Boivin" <charles.boivin at mayahtt.com> wrote:


    Hello,

    I am rather new to Trilinos, so my apologies if this is a question that has
    already been asked. I went through examples and the mailing lists and could
    not find references to it, so here it is.

    Following Example 9 of the Epetra/Didasko examples, I know it is possible to
    set up an Epetra_Import object that will allow me to fetch off-processor
    (ghost) values required to perform computations on local elements. It looks
    like the subsequent call to Epetra_Vector::Import() is synchronous. Is there
    any way a Epetra_Import object can be used in an asynchronous fashion?

    What I would like to do is the following (for an explicit type of scheme):

    - Start import
    - Work on elements not dependent on ghost values
    - Wait for import to be completed
    - Work on elements dependent on ghost values

    I can't seem to find anything that would allow this to work properly. Would
    I have to go back to using some functionality in the communicator then? If
    that's the case, can some of the info computed by the Epetra_Import object
    be of any use there?

    Thank you in advance.

    Charles Boivin


    _______________________________________________
    Trilinos-Users mailing list
    Trilinos-Users at software.sandia.gov
    http://software.sandia.gov/mailman/listinfo/trilinos-users

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://software.sandia.gov/pipermail/trilinos-users/attachments/20090407/0d6423ae/attachment-0001.html