[Trilinos-Users] Error Compiling Trilinos on Ubuntu 11.04
Matt Bockman
mdbockma at ucsd.edu
Thu Jul 21 21:17:01 MDT 2011
Hi Nuno,
Thanks for the reply. The output of update-alternatives --config mpi:
mdbockman at mdbockman:~/Documents/Research/codes/trilinosTest3/trilinos-10.6.4-Source/build$
update-alternatives --config mpi
There are 5 choices for the alternative mpi (providing /usr/include/mpi).
Selection Path Priority Status
------------------------------------------------------------
0 /usr/include/mpich2 40 auto mode
1 /usr/include/mpich2 40 manual mode
2 /usr/lib/mpich-mpd/include 10 manual mode
3 /usr/lib/mpich-shmem/include 10 manual mode
4 /usr/lib/mpich/include 10 manual mode
5 /usr/lib/openmpi/include 40 manual mode
This is after I installed openmpi. However, I don't know how to tell
cmake/trilinos how to actually use openmpi. I am going to uninstall mpich2
and see where I can go from there.
Thanks,
Matt
On Thu, Jul 21, 2011 at 5:41 AM, <trilinos-users-request at software.sandia.gov
> wrote:
> Send Trilinos-Users mailing list submissions to
> trilinos-users at software.sandia.gov
>
> To subscribe or unsubscribe via the World Wide Web, visit
> http://software.sandia.gov/mailman/listinfo/trilinos-users
> or, via email, send a message with subject or body 'help' to
> trilinos-users-request at software.sandia.gov
>
> You can reach the person managing the list at
> trilinos-users-owner at software.sandia.gov
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Trilinos-Users digest..."
>
>
> Today's Topics:
>
> 1. Error Compiling Trilinos on Ubuntu 11.04 (Matt Bockman)
> 2. Re: Error Compiling Trilinos on Ubuntu 11.04 (Kurtis Nusbaum)
> 3. Re: Error Compiling Trilinos on Ubuntu 11.04 (Doug Reeder)
> 4. Re: Error Compiling Trilinos on Ubuntu 11.04 (Nuno Sucena Almeida)
> 5. Re: Error Compiling Trilinos on Ubuntu 11.04 (Dustin Langewisch)
> 6. Tpetra efficiency (Einar Otnes)
> 7. Re: Tpetra efficiency (Einar Otnes)
> 8. Re: Tpetra efficiency (Holger Brandsmeier)
> 9. Re: Tpetra efficiency (Einar Otnes)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Wed, 20 Jul 2011 17:22:32 -0700
> From: "Matt Bockman" <mdbockma at ucsd.edu>
> Subject: [Trilinos-Users] Error Compiling Trilinos on Ubuntu 11.04
> To: trilinos-users at software.sandia.gov
> Message-ID:
> <CAFozh4N_uEOO7dOKaRHmnA-RPtWWMOYooSLfxc46J7ekhZQFFg at mail.gmail.com
> >
> Content-Type: text/plain; charset="iso-8859-1"
>
> Hi,
>
> I took the steps Nuno and Matthew Goodman suggested, namely:
>
> 1) installing necessary libraries
> 2) installing atlas using Goodman's script
> 3) Using Nuno's config script to do the configuration
> 4) Run make
>
> However, during the build I get an error which Google can't seem to find
> any
> information on:
>
> /usr/bin/ld: /usr/local/lib/libmpich.a(allreduce.o): relocation R_X86_64_32
> against `MPIR_ThreadSingle' can not be used when making a shared object;
> recompile with -fPIC
> /usr/local/lib/libmpich.a: could not read symbols: Bad value
> collect2: ld returned 1 exit status
> make[2]: *** [packages/teuchos/src/libteuchos.so] Error 1
> make[1]: *** [packages/teuchos/src/CMakeFiles/teuchos.dir/all] Error 2
> make: *** [all] Error 2
>
>
> Anyone ever seen this before or knows what it means? I'm very confused.
>
> Thanks,
> Matt
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> https://software.sandia.gov/pipermail/trilinos-users/attachments/20110720/c5286d2d/attachment-0001.html
>
> ------------------------------
>
> Message: 2
> Date: Wed, 20 Jul 2011 18:28:13 -0600
> From: "Kurtis Nusbaum" <klnusbaum at gmail.com>
> Subject: Re: [Trilinos-Users] Error Compiling Trilinos on Ubuntu 11.04
> To: "Matt Bockman" <mdbockma at ucsd.edu>
> Cc: trilinos-users at software.sandia.gov
> Message-ID:
> <CAGo-8c1i0DS53MoFCxthQ18=haM+Q1_9bLEVUJKGZvNh7RrQ0A at mail.gmail.com
> >
> Content-Type: text/plain; charset="iso-8859-1"
>
> Try adding this to the cmake configure script
>
> -DCMAKE_CXX_FLAGS="-fPIC"
>
> On Wed, Jul 20, 2011 at 6:22 PM, Matt Bockman <mdbockma at ucsd.edu> wrote:
>
> > Hi,
> >
> > I took the steps Nuno and Matthew Goodman suggested, namely:
> >
> > 1) installing necessary libraries
> > 2) installing atlas using Goodman's script
> > 3) Using Nuno's config script to do the configuration
> > 4) Run make
> >
> > However, during the build I get an error which Google can't seem to find
> > any information on:
> >
> > /usr/bin/ld: /usr/local/lib/libmpich.a(allreduce.o): relocation
> R_X86_64_32
> > against `MPIR_ThreadSingle' can not be used when making a shared object;
> > recompile with -fPIC
> > /usr/local/lib/libmpich.a: could not read symbols: Bad value
> > collect2: ld returned 1 exit status
> > make[2]: *** [packages/teuchos/src/libteuchos.so] Error 1
> > make[1]: *** [packages/teuchos/src/CMakeFiles/teuchos.dir/all] Error 2
> > make: *** [all] Error 2
> >
> >
> > Anyone ever seen this before or knows what it means? I'm very confused.
> >
> > Thanks,
> > Matt
> >
> >
> > _______________________________________________
> > Trilinos-Users mailing list
> > Trilinos-Users at software.sandia.gov
> > http://software.sandia.gov/mailman/listinfo/trilinos-users
> >
> >
>
>
> --
> Peace, Love, and Source Code
> -Kurtis
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> https://software.sandia.gov/pipermail/trilinos-users/attachments/20110720/af8dad3d/attachment-0001.html
>
> ------------------------------
>
> Message: 3
> Date: Wed, 20 Jul 2011 17:29:09 -0700
> From: "Doug Reeder" <dlr1 at cox.net>
> Subject: Re: [Trilinos-Users] Error Compiling Trilinos on Ubuntu 11.04
> To: "Matt Bockman" <mdbockma at ucsd.edu>
> Cc: trilinos-users at software.sandia.gov
> Message-ID: <14E311EA-F27F-4161-BEF0-E2D6A0EF001A at cox.net>
> Content-Type: text/plain; charset="us-ascii"
>
> Matt,
>
> Try adding the "-fPIC" argument to the compiler command line.
>
> Doug Reeder
> On Jul 20, 2011, at 5:22 PM, Matt Bockman wrote:
>
> > Hi,
> >
> > I took the steps Nuno and Matthew Goodman suggested, namely:
> >
> > 1) installing necessary libraries
> > 2) installing atlas using Goodman's script
> > 3) Using Nuno's config script to do the configuration
> > 4) Run make
> >
> > However, during the build I get an error which Google can't seem to find
> any information on:
> >
> > /usr/bin/ld: /usr/local/lib/libmpich.a(allreduce.o): relocation
> R_X86_64_32 against `MPIR_ThreadSingle' can not be used when making a shared
> object; recompile with -fPIC
> > /usr/local/lib/libmpich.a: could not read symbols: Bad value
> > collect2: ld returned 1 exit status
> > make[2]: *** [packages/teuchos/src/libteuchos.so] Error 1
> > make[1]: *** [packages/teuchos/src/CMakeFiles/teuchos.dir/all] Error 2
> > make: *** [all] Error 2
> >
> >
> > Anyone ever seen this before or knows what it means? I'm very confused.
> >
> > Thanks,
> > Matt
> >
> > _______________________________________________
> > Trilinos-Users mailing list
> > Trilinos-Users at software.sandia.gov
> > http://software.sandia.gov/mailman/listinfo/trilinos-users
>
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> https://software.sandia.gov/pipermail/trilinos-users/attachments/20110720/527abcec/attachment-0001.html
>
> ------------------------------
>
> Message: 4
> Date: Wed, 20 Jul 2011 22:10:25 -0400
> From: "Nuno Sucena Almeida" <slug at aeminium.org>
> Subject: Re: [Trilinos-Users] Error Compiling Trilinos on Ubuntu 11.04
> To: trilinos-users at software.sandia.gov
> Message-ID: <4E278A91.9080700 at aeminium.org>
> Content-Type: text/plain; charset=iso-8859-1
>
> On 07/20/2011 08:22 PM, Matt Bockman wrote:
> > /usr/local/lib/libmpich.a: could not read symbols: Bad value
>
> Hi Matt,
> I see that you are using mpich. I usually compile against openmpi,
> so
> that might be the problem.
> What does
>
> update-alternatives --config mpi
>
> give you?
> Nuno
>
> --
> http://aeminium.org/slug/
>
>
>
> ------------------------------
>
> Message: 5
> Date: Wed, 20 Jul 2011 21:37:41 -0600
> From: "Dustin Langewisch" <dustinl at mit.edu>
> Subject: Re: [Trilinos-Users] Error Compiling Trilinos on Ubuntu 11.04
> To: "Nuno Sucena Almeida" <slug at aeminium.org>
> Cc: trilinos-users at software.sandia.gov
> Message-ID:
> <CADeC6cfhT=3oh_yHaciA6ZSxJsZgusm3yiho0g-U=S-aF_DxJg at mail.gmail.com
> >
> Content-Type: text/plain; charset="iso-8859-1"
>
> Hi Matt,
>
> I recently installed trilinos on Ubuntu 10.04 and saw a similar error. I'm
> a
> noob, so there's probably a better fix, but I ended up completely removing
> mpich and used openmpi instead, as Nuno recommended. I encountered these
> issues when installing on my workstation, on which I had previously
> installed mpich. On my laptop, I've never installed mpich and the trilinos
> install went flawlessly, so my guess is it's an mpich issue.
>
> As for the -fPIC problem, I had a similar issue but when linking to Dakota.
> Kurtis' suggestion is probably the correct fix. I had never used CMake
> before and wasn't sure how to add the flag so I just exported the following
> variables:
> export CFLAGS="-fPIC"
> export CXXFLAGS="-fPIC"
> export FCFLAGS="-fPIC"
> export F77FLAGS="-fPIC"
> That seemed to work.
>
> Good luck,
> Dustin
>
>
> On Wed, Jul 20, 2011 at 8:10 PM, Nuno Sucena Almeida <slug at aeminium.org
> >wrote:
>
> > On 07/20/2011 08:22 PM, Matt Bockman wrote:
> > > /usr/local/lib/libmpich.a: could not read symbols: Bad value
> >
> > Hi Matt,
> > I see that you are using mpich. I usually compile against openmpi,
> > so
> > that might be the problem.
> > What does
> >
> > update-alternatives --config mpi
> >
> > give you?
> > Nuno
> >
> > --
> > http://aeminium.org/slug/
> >
> > _______________________________________________
> > Trilinos-Users mailing list
> > Trilinos-Users at software.sandia.gov
> > http://software.sandia.gov/mailman/listinfo/trilinos-users
> >
> >
>
>
> --
> Dustin Langewisch
> Graduate Student
> MIT Nuclear Science and Engineering
> (573) 289-4411
> dustinl at mit.edu
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> https://software.sandia.gov/pipermail/trilinos-users/attachments/20110720/9a774311/attachment-0001.html
>
> ------------------------------
>
> Message: 6
> Date: Thu, 21 Jul 2011 12:08:11 +0100
> From: "Einar Otnes" <eotnes at gmail.com>
> Subject: [Trilinos-Users] Tpetra efficiency
> To: trilinos-users at software.sandia.gov
> Message-ID:
> <CAB4PwT0E9AmTjW8P1bB1kGBWiAaUYYBUtma46VJuqnZFFqboQA at mail.gmail.com
> >
> Content-Type: text/plain; charset="iso-8859-1"
>
> Dear experts,
>
> I have been testing the performance of a simple matrix multiplication using
> 2 different Matrix/Vector classes on a single node and using a single
> thread, i.e. Teuchos::SerialDenseMatrix/Vector and Tpetra::CrsMatrix and
> Tpetra::Vector, and I have seen that the run time for evaluating the same
> product differs by a factor ~2 between the SerialDenseMatrix and
> CrsMatrix.
> Is this behaviour expected when running on a single node/single thread? Or
> is the way I'm the Tpetra matrices making my code inefficient? Below
> follows
> the output from my code showing the time it takes to run 200 evaluations of
> Ax=b where the size of the matrix A is 5000x5000.
>
> I have attached the code I wrote to produce the results below.
>
> All the best,
> Einar Otnes
>
>
> ==========================================================================
>
> Teuchos in Trilinos 10.6.4
> Tpetra in Trilinos 10.6.4
>
> Evaluate Ax=b.
> Problem size: numRows m= 5000 numCols n= 5000
> Ax=b will be evaluated 200 time(s).
> Initialize the Matrices and Vectors with random numbers
> Start the Calculations!!
> Done Ax=b using Teuchos::SerialDenseMatrix
> Done Ax=b using Tpetra::CrsMatrix
> Done Ax=b using Thyra/Tpetra adapter
>
> Calculate norm(b) for each of the three methods applied.
> bNorm= 1671.55
> dNorm= 1671.55
> b2Norm= 1671.55
>
> Tpetra Vector
> Tpetra::Vector<double, int, int, Kokkos::TPINode>{length=5000}
> node 0: local length=5000
>
> Thyra wrapped Tpetra Vector
> Thyra::TpetraVector<double, int, int,
> Kokkos::TPINode>{spmdSpace=Thyra::TpetraVectorSpace<double, int, int,
>
> Kokkos::TPINode>{globalDim=5000,localSubDim=5000,localOffset=0,comm=Teuchos::SerialComm<long
> int>}}
>
> ================================================================================
>
> TimeMonitor Results
>
> Timer Name Local time (num calls)
>
> --------------------------------------------------------------------------------
> SerialDenseMatrix apply Time 3.69 (200)
> CrsMatrix apply Time 7.53 (200)
> Thyra CrsMatrix apply Time 7.43 (200)
>
> ================================================================================
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> https://software.sandia.gov/pipermail/trilinos-users/attachments/20110721/761fb5ca/attachment-0001.html
> -------------- next part --------------
> A non-text attachment was scrubbed...
> Name: main.cpp.gz
> Type: application/x-gzip
> Size: 1867 bytes
> Desc: not available
> Url :
> https://software.sandia.gov/pipermail/trilinos-users/attachments/20110721/761fb5ca/attachment-0001.gz
>
> ------------------------------
>
> Message: 7
> Date: Thu, 21 Jul 2011 12:47:17 +0100
> From: "Einar Otnes" <eotnes at gmail.com>
> Subject: Re: [Trilinos-Users] Tpetra efficiency
> To: "Holger Brandsmeier" <holger.brandsmeier at sam.math.ethz.ch>,
> trilinos-users at software.sandia.gov
> Message-ID:
> <CAB4PwT10CNgcoh9svbgY8DBwuDmRUG+ciQDw_MYn=YuR2q6mqg at mail.gmail.com
> >
> Content-Type: text/plain; charset="iso-8859-1"
>
> Holger,
> Thank you for the prompt response. I didn't think about the dense vs sparse
> matrix considerations. Thank you for pointing that out. Do you know whether
> there is a dense matrix class in Tpetra?
> I was thinking of trying out VbrMatrix using a single block, but I haven't
> been able to make that work, yet. Is that a possible path ?
>
> Thanks again,
> Einar
>
> On Thu, Jul 21, 2011 at 12:34 PM, Holger Brandsmeier <
> holger.brandsmeier at sam.math.ethz.ch> wrote:
>
> > Dear Einar,
> >
> > Tpetra::CrsMatrix is a sparse Matrix class, Teuchos::SerialDenseMatrix
> > is a dense matrix as the name implies. I assume that the matrix you
> > actually test this with is also dense, then the difference you observe
> > is certainly to be expected.
> >
> > Note that sparse matrices are matrices where many entries are zero.
> > When there are many zero entries Tpetra::CrsMatrix is fast. But when
> > all entries are different than zero, then Tpetra::CrsMatrix is slower,
> > as it has not been desinged for it. The factor of 2 is quite low,
> > considering that you are using Tpetra::CrsMatrix for something it has
> > not been designed for.
> >
> > -Holger
> >
> > On Thu, Jul 21, 2011 at 13:08, Einar Otnes <eotnes at gmail.com> wrote:
> > > Dear experts,
> > >
> > > I have been testing the performance of a simple matrix multiplication
> > using
> > > 2 different Matrix/Vector classes on a single node and using a single
> > > thread, i.e. Teuchos::SerialDenseMatrix/Vector and Tpetra::CrsMatrix
> and
> > > Tpetra::Vector, and I have seen that the run time for evaluating the
> same
> > > product differs by a factor ~2 between the SerialDenseMatrix and
> > CrsMatrix.
> > > Is this behaviour expected when running on a single node/single
> thread?
> > Or
> > > is the way I'm the Tpetra matrices making my code inefficient? Below
> > follows
> > > the output from my code showing the time it takes to run 200
> evaluations
> > of
> > > Ax=b where the size of the matrix A is 5000x5000.
> > >
> > > I have attached the code I wrote to produce the results below.
> > >
> > > All the best,
> > > Einar Otnes
> > >
> > >
> > >
> >
> ==========================================================================
> > >
> > > Teuchos in Trilinos 10.6.4
> > > Tpetra in Trilinos 10.6.4
> > >
> > > Evaluate Ax=b.
> > > Problem size: numRows m= 5000 numCols n= 5000
> > > Ax=b will be evaluated 200 time(s).
> > > Initialize the Matrices and Vectors with random numbers
> > > Start the Calculations!!
> > > Done Ax=b using Teuchos::SerialDenseMatrix
> > > Done Ax=b using Tpetra::CrsMatrix
> > > Done Ax=b using Thyra/Tpetra adapter
> > >
> > > Calculate norm(b) for each of the three methods applied.
> > > bNorm= 1671.55
> > > dNorm= 1671.55
> > > b2Norm= 1671.55
> > >
> > > Tpetra Vector
> > > Tpetra::Vector<double, int, int, Kokkos::TPINode>{length=5000}
> > > node 0: local length=5000
> > >
> > > Thyra wrapped Tpetra Vector
> > > Thyra::TpetraVector<double, int, int,
> > > Kokkos::TPINode>{spmdSpace=Thyra::TpetraVectorSpace<double, int, int,
> > >
> >
> Kokkos::TPINode>{globalDim=5000,localSubDim=5000,localOffset=0,comm=Teuchos::SerialComm<long
> > > int>}}
> > >
> >
> ================================================================================
> > >
> > > TimeMonitor Results
> > >
> > > Timer Name Local time (num calls)
> > >
> >
> --------------------------------------------------------------------------------
> > > SerialDenseMatrix apply Time 3.69 (200)
> > > CrsMatrix apply Time 7.53 (200)
> > > Thyra CrsMatrix apply Time 7.43 (200)
> > >
> >
> ================================================================================
> > >
> > >
> > > _______________________________________________
> > > Trilinos-Users mailing list
> > > Trilinos-Users at software.sandia.gov
> > > http://software.sandia.gov/mailman/listinfo/trilinos-users
> > >
> > >
> >
> >
> >
> > --
> > Holger Brandsmeier, SAM, ETH Z?rich
> > http://www.sam.math.ethz.ch/people/bholger
> >
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> https://software.sandia.gov/pipermail/trilinos-users/attachments/20110721/034c3304/attachment-0001.html
>
> ------------------------------
>
> Message: 8
> Date: Thu, 21 Jul 2011 14:26:13 +0200
> From: "Holger Brandsmeier" <holger.brandsmeier at sam.math.ethz.ch>
> Subject: Re: [Trilinos-Users] Tpetra efficiency
> To: "Einar Otnes" <eotnes at gmail.com>
> Cc: trilinos-users at software.sandia.gov
> Message-ID:
> <CAFP7zVtWxBFEkcVTJg3acOe2Gjvgn3WecS0LxE_9hqOfmjtuZw at mail.gmail.com
> >
> Content-Type: text/plain; charset=utf-8
>
> Einer,
>
> VbrMatrix is also a sparse matrix, just a different type of sparse Matrix.
>
> Can't you use Teuchos::SerialDenseMatrix for your needs, in particular
> as you talked about using a single thread. This is what
> Teuchos::SerialDenseMatrix has been developed for. For the Scalar
> types double and complex<double> it uses lapack and blas routines
> which are very efficient for dense matrices.
>
> Unfortunately I do not know about any dense _parallel_ matrix class in
> Trilinos. I don't even know about many dense parallel matrix
> implementations outside of trilinos, in particular not templated as in
> Tpetra / Teuchos. Maybe someone else knows more about this.
>
> -Holger
>
> On Thu, Jul 21, 2011 at 13:47, Einar Otnes <eotnes at gmail.com> wrote:
> > Holger,
> > Thank you for the prompt response. I didn't think about the dense vs
> sparse
> > matrix considerations. Thank you for pointing that out. Do you know
> whether
> > there is a dense matrix class in Tpetra?
> > I was thinking of trying out VbrMatrix using a single block, but I
> haven't
> > been able to make that work, yet. Is that a possible path ?
> >
> > Thanks again,
> > Einar
> >
> > On Thu, Jul 21, 2011 at 12:34 PM, Holger Brandsmeier
> > <holger.brandsmeier at sam.math.ethz.ch> wrote:
> >>
> >> Dear Einar,
> >>
> >> Tpetra::CrsMatrix is a sparse Matrix class, Teuchos::SerialDenseMatrix
> >> is a dense matrix as the name implies. I assume that the matrix you
> >> actually test this with is also dense, then the difference you observe
> >> is certainly to be expected.
> >>
> >> Note that sparse matrices are matrices where many entries are zero.
> >> When there are many zero entries Tpetra::CrsMatrix is fast. But when
> >> all entries are different than zero, then Tpetra::CrsMatrix is slower,
> >> as it has not been desinged for it. The factor of 2 is quite low,
> >> considering that you are using Tpetra::CrsMatrix for something it has
> >> not been designed for.
> >>
> >> -Holger
> >>
> >> On Thu, Jul 21, 2011 at 13:08, Einar Otnes <eotnes at gmail.com> wrote:
> >> > Dear experts,
> >> >
> >> > I have been testing the performance of a simple matrix multiplication
> >> > using
> >> > 2 different Matrix/Vector classes on a single node and using a single
> >> > thread, i.e. Teuchos::SerialDenseMatrix/Vector and Tpetra::CrsMatrix
> and
> >> > Tpetra::Vector, and I have seen that the run time for evaluating the
> >> > same
> >> > product differs by a factor ~2? between the SerialDenseMatrix and
> >> > CrsMatrix.
> >> > Is this? behaviour expected when running on a single node/single
> thread?
> >> > Or
> >> > is the way I'm the Tpetra matrices making my code inefficient? Below
> >> > follows
> >> > the output from my code showing the time it takes to run 200
> evaluations
> >> > of
> >> > Ax=b where the size of the matrix A is 5000x5000.
> >> >
> >> > I have attached the code I wrote to produce the results below.
> >> >
> >> > All the best,
> >> > Einar Otnes
> >> >
> >> >
> >> >
> >> >
> ==========================================================================
> >> >
> >> > Teuchos in Trilinos 10.6.4
> >> > Tpetra in Trilinos 10.6.4
> >> >
> >> > Evaluate Ax=b.
> >> > Problem size: numRows m= 5000 numCols n= 5000
> >> > Ax=b will be evaluated 200 time(s).
> >> > Initialize the Matrices and Vectors with random numbers
> >> > Start the Calculations!!
> >> > Done Ax=b using Teuchos::SerialDenseMatrix
> >> > Done Ax=b using Tpetra::CrsMatrix
> >> > Done Ax=b using Thyra/Tpetra adapter
> >> >
> >> > Calculate norm(b) for each of the three methods applied.
> >> > bNorm= 1671.55
> >> > dNorm= 1671.55
> >> > b2Norm= 1671.55
> >> >
> >> > Tpetra Vector
> >> > ?Tpetra::Vector<double, int, int, Kokkos::TPINode>{length=5000}
> >> > ?node??? 0: local length=5000
> >> >
> >> > Thyra wrapped Tpetra Vector
> >> > ?Thyra::TpetraVector<double, int, int,
> >> > Kokkos::TPINode>{spmdSpace=Thyra::TpetraVectorSpace<double, int, int,
> >> >
> >> >
> Kokkos::TPINode>{globalDim=5000,localSubDim=5000,localOffset=0,comm=Teuchos::SerialComm<long
> >> > int>}}
> >> >
> >> >
> ================================================================================
> >> >
> >> > ????????????????????????????? TimeMonitor Results
> >> >
> >> > Timer Name????????????????????? Local time (num calls)
> >> >
> >> >
> --------------------------------------------------------------------------------
> >> > SerialDenseMatrix apply Time??? 3.69 (200)
> >> > CrsMatrix apply Time??????????? 7.53 (200)
> >> > Thyra CrsMatrix apply Time????? 7.43 (200)
> >> >
> >> >
> ================================================================================
> >> >
> >> >
> >> > _______________________________________________
> >> > Trilinos-Users mailing list
> >> > Trilinos-Users at software.sandia.gov
> >> > http://software.sandia.gov/mailman/listinfo/trilinos-users
> >> >
> >> >
> >>
> >>
> >>
> >> --
> >> Holger Brandsmeier, SAM, ETH Z?rich
> >> http://www.sam.math.ethz.ch/people/bholger
> >
> >
>
>
>
> --
> Holger Brandsmeier, SAM, ETH Z?rich
> http://www.sam.math.ethz.ch/people/bholger
>
>
>
>
> ------------------------------
>
> Message: 9
> Date: Thu, 21 Jul 2011 13:40:33 +0100
> From: "Einar Otnes" <eotnes at gmail.com>
> Subject: Re: [Trilinos-Users] Tpetra efficiency
> To: "Holger Brandsmeier" <holger.brandsmeier at sam.math.ethz.ch>,
> trilinos-users at software.sandia.gov
> Message-ID:
> <CAB4PwT0X4_mwjEYK5u4EsSgMimgovx+Ngd8dfPojR3LmKk6tTA at mail.gmail.com
> >
> Content-Type: text/plain; charset="iso-8859-1"
>
> Thanks,
> Well, the test I ran was a first initial test to check the performance on a
> single node/thread example. My purpose is to write an application where
> dense matrices will be distributed, and I want to use the functionality of
> Thyra and Tpetra to design compound operators etc.
>
> Thanks,
>
> Einar
>
>
> On Thu, Jul 21, 2011 at 1:26 PM, Holger Brandsmeier <
> holger.brandsmeier at sam.math.ethz.ch> wrote:
>
> > Einer,
> >
> > VbrMatrix is also a sparse matrix, just a different type of sparse
> Matrix.
> >
> > Can't you use Teuchos::SerialDenseMatrix for your needs, in particular
> > as you talked about using a single thread. This is what
> > Teuchos::SerialDenseMatrix has been developed for. For the Scalar
> > types double and complex<double> it uses lapack and blas routines
> > which are very efficient for dense matrices.
> >
> > Unfortunately I do not know about any dense _parallel_ matrix class in
> > Trilinos. I don't even know about many dense parallel matrix
> > implementations outside of trilinos, in particular not templated as in
> > Tpetra / Teuchos. Maybe someone else knows more about this.
> >
> > -Holger
> >
> > On Thu, Jul 21, 2011 at 13:47, Einar Otnes <eotnes at gmail.com> wrote:
> > > Holger,
> > > Thank you for the prompt response. I didn't think about the dense vs
> > sparse
> > > matrix considerations. Thank you for pointing that out. Do you know
> > whether
> > > there is a dense matrix class in Tpetra?
> > > I was thinking of trying out VbrMatrix using a single block, but I
> > haven't
> > > been able to make that work, yet. Is that a possible path ?
> > >
> > > Thanks again,
> > > Einar
> > >
> > > On Thu, Jul 21, 2011 at 12:34 PM, Holger Brandsmeier
> > > <holger.brandsmeier at sam.math.ethz.ch> wrote:
> > >>
> > >> Dear Einar,
> > >>
> > >> Tpetra::CrsMatrix is a sparse Matrix class, Teuchos::SerialDenseMatrix
> > >> is a dense matrix as the name implies. I assume that the matrix you
> > >> actually test this with is also dense, then the difference you observe
> > >> is certainly to be expected.
> > >>
> > >> Note that sparse matrices are matrices where many entries are zero.
> > >> When there are many zero entries Tpetra::CrsMatrix is fast. But when
> > >> all entries are different than zero, then Tpetra::CrsMatrix is slower,
> > >> as it has not been desinged for it. The factor of 2 is quite low,
> > >> considering that you are using Tpetra::CrsMatrix for something it has
> > >> not been designed for.
> > >>
> > >> -Holger
> > >>
> > >> On Thu, Jul 21, 2011 at 13:08, Einar Otnes <eotnes at gmail.com> wrote:
> > >> > Dear experts,
> > >> >
> > >> > I have been testing the performance of a simple matrix
> multiplication
> > >> > using
> > >> > 2 different Matrix/Vector classes on a single node and using a
> single
> > >> > thread, i.e. Teuchos::SerialDenseMatrix/Vector and Tpetra::CrsMatrix
> > and
> > >> > Tpetra::Vector, and I have seen that the run time for evaluating the
> > >> > same
> > >> > product differs by a factor ~2 between the SerialDenseMatrix and
> > >> > CrsMatrix.
> > >> > Is this behaviour expected when running on a single node/single
> > thread?
> > >> > Or
> > >> > is the way I'm the Tpetra matrices making my code inefficient? Below
> > >> > follows
> > >> > the output from my code showing the time it takes to run 200
> > evaluations
> > >> > of
> > >> > Ax=b where the size of the matrix A is 5000x5000.
> > >> >
> > >> > I have attached the code I wrote to produce the results below.
> > >> >
> > >> > All the best,
> > >> > Einar Otnes
> > >> >
> > >> >
> > >> >
> > >> >
> >
> ==========================================================================
> > >> >
> > >> > Teuchos in Trilinos 10.6.4
> > >> > Tpetra in Trilinos 10.6.4
> > >> >
> > >> > Evaluate Ax=b.
> > >> > Problem size: numRows m= 5000 numCols n= 5000
> > >> > Ax=b will be evaluated 200 time(s).
> > >> > Initialize the Matrices and Vectors with random numbers
> > >> > Start the Calculations!!
> > >> > Done Ax=b using Teuchos::SerialDenseMatrix
> > >> > Done Ax=b using Tpetra::CrsMatrix
> > >> > Done Ax=b using Thyra/Tpetra adapter
> > >> >
> > >> > Calculate norm(b) for each of the three methods applied.
> > >> > bNorm= 1671.55
> > >> > dNorm= 1671.55
> > >> > b2Norm= 1671.55
> > >> >
> > >> > Tpetra Vector
> > >> > Tpetra::Vector<double, int, int, Kokkos::TPINode>{length=5000}
> > >> > node 0: local length=5000
> > >> >
> > >> > Thyra wrapped Tpetra Vector
> > >> > Thyra::TpetraVector<double, int, int,
> > >> > Kokkos::TPINode>{spmdSpace=Thyra::TpetraVectorSpace<double, int,
> int,
> > >> >
> > >> >
> >
> Kokkos::TPINode>{globalDim=5000,localSubDim=5000,localOffset=0,comm=Teuchos::SerialComm<long
> > >> > int>}}
> > >> >
> > >> >
> >
> ================================================================================
> > >> >
> > >> > TimeMonitor Results
> > >> >
> > >> > Timer Name Local time (num calls)
> > >> >
> > >> >
> >
> --------------------------------------------------------------------------------
> > >> > SerialDenseMatrix apply Time 3.69 (200)
> > >> > CrsMatrix apply Time 7.53 (200)
> > >> > Thyra CrsMatrix apply Time 7.43 (200)
> > >> >
> > >> >
> >
> ================================================================================
> > >> >
> > >> >
> > >> > _______________________________________________
> > >> > Trilinos-Users mailing list
> > >> > Trilinos-Users at software.sandia.gov
> > >> > http://software.sandia.gov/mailman/listinfo/trilinos-users
> > >> >
> > >> >
> > >>
> > >>
> > >>
> > >> --
> > >> Holger Brandsmeier, SAM, ETH Z?rich
> > >> http://www.sam.math.ethz.ch/people/bholger
> > >
> > >
> >
> >
> >
> > --
> > Holger Brandsmeier, SAM, ETH Z?rich
> > http://www.sam.math.ethz.ch/people/bholger
> >
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> https://software.sandia.gov/pipermail/trilinos-users/attachments/20110721/d28e3a44/attachment.html
>
> ------------------------------
>
> _______________________________________________
> Trilinos-Users mailing list
> Trilinos-Users at software.sandia.gov
> http://software.sandia.gov/mailman/listinfo/trilinos-users
>
>
> End of Trilinos-Users Digest, Vol 71, Issue 14
> **********************************************
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://software.sandia.gov/pipermail/trilinos-users/attachments/20110721/8f6b087c/attachment-0001.html
More information about the Trilinos-Users
mailing list