[Trilinos-Users] GMRES inconsistent when using multiple threads (Belos+Tpetra, since 12.12.1)
Hoemmen, Mark
mhoemme at sandia.gov
Wed Dec 13 00:25:14 EST 2017
Greetings!
Thanks for testing out Belos and Tpetra with threads!
The "rma10" matrix from the University of Florida collection:
https://www.cise.ufl.edu/research/sparse/matrices/Bova/rma10.html
is unsymmetric. Unsymmetric matrices may be nonnormal and may thus have more sensitive convergence properties. More importantly, I noticed that you are using a very large convergence tolerance: 1.0e-1. The initial iterations may be very sensitive to small rounding-error differences due to thread parallelization.
We did fix a bug with mixed-precision sparse matrix-vector multiply in the master Trilinos branch. It could still be in 12.12. I'm curious if that is related. Have you had a chance to try the latest version of Trilinos?
Thanks and best regards,
Mark Hoemmen
On 12/6/17, 9:26 AM, "Trilinos-Users on behalf of trilinos-users-request at trilinos.org" <trilinos-users-bounces at trilinos.org on behalf of trilinos-users-request at trilinos.org> wrote:
>> On Thu, Nov 9, 2017 at 6:26 PM, Christopher Thiele <ct37 at rice.edu> wrote:
>>
>> Hello,
>>
>> I noticed some strange behavior in Belos' GMRES solver when using Tpetra
>> and multiple threads. I wrote a little tool (solver_driver) that just reads
>> in a matrix market file and solves a linear system to reproduce it. Here
>> are some results with the rma10 matrix from the University of Florida
>> collection (the choice is rather arbitrary):
>>
>> ./solver_driver_12.10.1 --threads=2 --matrix=rma10.mtx --rhs=rma10_b.mtx
>> --tol=1.0e-1 --solver=GMRES --disable-preconditioner
>> ---> Converges in just 3 iterations, independent of the number of threads
>> etc. This seems reasonable given the tolerance of 1e-1.
>>
>> ./solver_driver_12.12.1 --threads=2 --matrix=rma10.mtx --rhs=rma10_b.mtx
>> --tol=1.0e-1 --solver=GMRES --disable-preconditioner
>> ---> Converges in either 61 or 62 iterations, but the final residual
>> changes in the order of 1e-4 with each run.
>>
>> OMP_PROC_BIND=true ./solver_driver_12.12.1 --threads=2 --matrix=rma10.mtx
>> --rhs=rma10_b.mtx --tol=1.0e-1 --solver=GMRES --disable-preconditioner
>> ---> Converges in 121 iterations, and the final residual changes in the
>> order of 1e-3.
>>
>> mpirun -n 1 ./solver_driver_12.12.1 --threads=2 --matrix=rma10.mtx
>> --rhs=rma10_b.mtx --tol=1.0e-1 --solver=GMRES --disable-preconditioner
>> ---> Seems to solve the problem, consistently reproduces the result
>> obtained with 12.10.1
>>
>> I also tested the current master branch from GitHub, and it shows the same
>> behavior as the 12.12.1 release. The problem does not occur with other
>> solvers like BiCGStab or CG. I added some information about my system and
>> the Trilinos configuration below. I can also share the code (~200 loc), but
>> it is really just a driver for the solvers.
>>
>> Best regards,
>> Christopher
>>
>>
>>
>> System:
>>
>> OS: RHEL 7 (x84_64)
>> Compiler: GCC 4.8.5
>> MPI: Open MPI 1.10.6
>> CPU: Xeon E3-1220 V2 (4 cores)
>> Memory: 8 GB
>>
>>
>>
>> Trilinos configuration script:
>>
>> #!/bin/bash
>>
>> set -e
>>
>> SOURCE_BASE=/path/to/source
>> INSTALL_DIR=/path/to/installation
>>
>> TRILINOS_PKG_CONF=" \
>> -DTrilinos_ENABLE_Teuchos=ON \
>> -DTrilinos_ENABLE_TeuchosParser=ON \
>> -DTrilinos_ENABLE_Belos=ON \
>> -DTrilinos_ENABLE_Ifpack2=ON \
>> -DTrilinos_ENABLE_MueLu=OFF \
>> -DTrilinos_ENABLE_Tpetra=ON \
>> -DTrilinos_ENABLE_Teko=OFF \
>> -DTrilinos_ENABLE_Zoltan2=OFF \
>> -DTrilinos_ASSERT_MISSING_PACKAGES=OFF \
>> "
>>
>> TP_PKG_CONF=" \
>> -DTPL_ENABLE_MPI=ON \
>> -DTPL_ENABLE_CUDA=OFF \
>> -DTPL_ENABLE_MKL=ON \
>> -DTPL_MKL_LIBRARIES=/path/to/mkl/lib/intel64/libmkl_rt.so \
>> -DTPL_MKL_INCLUDE_DIRS=/path/to/mkl/include \
>> "
>>
>> cmake \
>> -D MPI_C_COMPILER=mpicc \
>> -D MPI_CXX_COMPILER=mpicxx \
>> -D MPI_Fortran_COMPILER=mpifort \
>> -D CMAKE_C_COMPILER=mpicc \
>> -D CMAKE_CXX_COMPILER=mpicxx \
>> -D CMAKE_Fortran_COMPILER=mpifort \
>> -D CMAKE_CXX_FLAGS="-g -O3 -march=native" \
>> -D CMAKE_BUILD_TYPE=RELEASE \
>> -D Trilinos_ENABLE_TESTS=OFF \
>> -D Trilinos_ENABLE_EXPLICIT_INSTANTIATION=ON \
>> -D Trilinos_CXX11_FLAGS="-std=c++11" \
>> -D Kokkos_ENABLE_Pthread=OFF \
>> -D Kokkos_ENABLE_OpenMP=ON \
>> -D Kokkos_ENABLE_Cuda=OFF \
>> -D Kokkos_ENABLE_Cuda_UVM=OFF \
>> -D Trilinos_ENABLE_OpenMP=ON \
>> -D Tpetra_INST_SERIAL=ON \
>> -D Tpetra_INST_CUDA=OFF \
>> -D Tpetra_INST_OPENMP=ON \
>> -D Tpetra_INST_INT_INT=ON \
>> -D Tpetra_INST_INT_LONG=OFF \
>> -D Tpetra_INST_COMPLEX_DOUBLE=OFF \
>> -D Teuchos_ENABLE_LONG_LONG_INT=OFF \
>> -D CMAKE_INSTALL_PREFIX=${INSTALL_DIR} \
>> ${TRILINOS_PKG_CONF} \
>> ${TP_PKG_CONF} \
>> ${SOURCE_BASE}
>> _______________________________________________
>> Trilinos-Users mailing list
>> Trilinos-Users at trilinos.org
>> https://trilinos.org/mailman/listinfo/trilinos-users
>>
>>
>>
>-------------- next part --------------
>An HTML attachment was scrubbed...
>URL: <https://trilinos.org/pipermail/trilinos-users/attachments/20171110/cb49995d/attachment-0001.html>
>
>------------------------------
>
>Message: 2
>Date: Fri, 10 Nov 2017 01:35:43 +0100
>From: Eligiusz Postek <ewpostek at googlemail.com>
>To: trilinos-users at trilinos.org
>Subject: [Trilinos-Users] hdf5
>Message-ID:
> <CA+maUnpSAB34eXgZ=Q9wGX1vGPt82_qDz0CcwoUyPh-w479GCg at mail.gmail.com>
>Content-Type: text/plain; charset="utf-8"
>
>Hello,
>Could it mean that the cray libs (hdf5) are incomplete ?
>Is there any other error ?
>best regards, Elek
>
>
>[ 34%] Building CXX object
>packages/teuchos/parameterlist/src/CMakeFiles/teuchosparameterlist.dir/Teuchos_StringInputStream.cpp.o
>/opt/cray/hdf5/1.8.16/GNU/5.1/lib/libhdf5.a(H5PL.o): In function
>`H5PL_load':
>H5PL.c:(.text+0x634): warning: Using 'dlopen' in statically linked
>applications requires at runtime the shared libraries from the glibc
>version used for linking
>/opt/cray/hdf5/1.8.16/GNU/5.1/lib/libhdf5_hl.a(H5LT.o): In function
>`H5LT_dtype_to_text':
>H5LT.c:(.text+0x2871): undefined reference to `H5Tget_tag'
>/opt/cray/hdf5/1.8.16/GNU/5.1/lib/libhdf5_hl.a(H5LTparse.o): In function
>`H5LTyyparse':
>H5LTparse.c:(.text+0x108d): undefined reference to `H5Tset_tag'
>collect2: error: ld returned 1 exit status
>make[2]: *** [packages/seacas/applications/nem_slice/nem_slice] Error 1
>-------------- next part --------------
>An HTML attachment was scrubbed...
>URL: <https://trilinos.org/pipermail/trilinos-users/attachments/20171110/35555cee/attachment-0001.html>
>
>------------------------------
>
>Message: 3
>Date: Fri, 10 Nov 2017 16:13:50 +0000
>From: Eric Bavier <bavier at cray.com>
>To: Eligiusz Postek <ewpostek at googlemail.com>,
> "trilinos-users at trilinos.org" <trilinos-users at trilinos.org>
>Subject: Re: [Trilinos-Users] hdf5
>Message-ID:
> <CY4PR11MB17189277D1DAF790FF983E1AB7540 at CY4PR11MB1718.namprd11.prod.outlook.com>
>
>Content-Type: text/plain; charset="iso-8859-1"
>
>Hello Elek,
>
>The 'H5Tget_tag' symbol is defined in libhdf5.a, so you'll need to make sure that the HDF5 TPL includes that library when configuring. FWIW when I build Trilinos with cray-hdf5, I usually set:
>
> -D TPL_ENABLE_HDF5:BOOL=ON \
> -D TPL_HDF5_INCLUDE_DIRS:FILEPATH=${HDF5_DIR}/include \
> -D HDF5_LIBRARY_DIRS:FILEPATH=${HDF5_DIR}/lib \
> -D HDF5_LIBRARY_NAMES:STRING="hdf5_hl_parallel;hdf5_parallel;z;dl"
>
>Hope that helps,
>
>Eric Bavier, Scientific Libraries, Cray Inc.
>
>________________________________________
>From: Trilinos-Users <trilinos-users-bounces at trilinos.org> on behalf of Eligiusz Postek <ewpostek at googlemail.com>
>Sent: Thursday, November 9, 2017 18:35
>To: trilinos-users at trilinos.org
>Subject: [Trilinos-Users] hdf5
>
>Hello,
>Could it mean that the cray libs (hdf5) are incomplete ?
>Is there any other error ?
>best regards, Elek
>
>
>[ 34%] Building CXX object packages/teuchos/parameterlist/src/CMakeFiles/teuchosparameterlist.dir/Teuchos_StringInputStream.cpp.o
>/opt/cray/hdf5/1.8.16/GNU/5.1/lib/libhdf5.a(H5PL.o): In function `H5PL_load':
>H5PL.c:(.text+0x634): warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
>/opt/cray/hdf5/1.8.16/GNU/5.1/lib/libhdf5_hl.a(H5LT.o): In function `H5LT_dtype_to_text':
>H5LT.c:(.text+0x2871): undefined reference to `H5Tget_tag'
>/opt/cray/hdf5/1.8.16/GNU/5.1/lib/libhdf5_hl.a(H5LTparse.o): In function `H5LTyyparse':
>H5LTparse.c:(.text+0x108d): undefined reference to `H5Tset_tag'
>collect2: error: ld returned 1 exit status
>make[2]: *** [packages/seacas/applications/nem_slice/nem_slice] Error 1
>
>
>
>------------------------------
>
>Message: 4
>Date: Fri, 10 Nov 2017 18:33:16 +0000
>From: Eligiusz Postek <ewpostek at googlemail.com>
>To: Eric Bavier <bavier at cray.com>
>Cc: "trilinos-users at trilinos.org" <trilinos-users at trilinos.org>
>Subject: Re: [Trilinos-Users] hdf5
>Message-ID:
> <CA+maUnqBMXuFYYxWkr2Kg=3HOHq7dGKR1RZeTaTDNT7HgiW1DA at mail.gmail.com>
>Content-Type: text/plain; charset="utf-8"
>
>Thank you, HDF5 passed,
>but I still have a problem with netcdf
>
>[ 35%] Building Fortran object
>packages/seacas/applications/gjoin/CMakeFiles/gjoin.dir/gj_zmfixd.f.o
>[ 35%] Building Fortran object
>packages/seacas/libraries/plt/CMakeFiles/plt.dir/pltrsd.f.o
>/opt/cray/netcdf-hdf5parallel/4.4.0/GNU/5.1/lib/libnetcdf_parallel.a(libnetcdf4_la-nc4file.o):
>In function `sync_netcdf4_file':
>/b/ulib/netcdf-support/rpm/BUILD/cray-netcdf-4.4.0-gcc2/netcdf-4.4.0/libsrc4/nc4file.c:3006:
>undefined reference to `H5Fflush'
>/opt/cray/netcdf-hdf5parallel/4.4.0/GNU/5.1/lib/libnetcdf_parallel.a(libnetcdf4_la-nc4file.o):
>In function `close_netcdf4_file':
>/b/ulib/netcdf-support/rpm/BUILD/cray-netcdf-4.4.0-gcc2/netcdf-4.4.0/libsrc4/nc4file.c:3090:
>undefined reference to `H5Fclose'
>/b/ulib/netcdf-support/rpm/BUILD/cray-netcdf-4.4.0-gcc2/netcdf-4.4.0/libsrc4/nc4file.c:3094:
>undefined reference to `H5Fget_obj
>
>and ......
>
>
>I use
> -D TPL_ENABLE_Netcdf:BOOL=ON
>-D
>Netcdf_LIBRARY_DIRS:FILEPATH="/opt/cray/netcdf-hdf5parallel/4.4.0/GNU/5.1/lib"
>\
>-D
>Netcdf_INCLUDE_DIRS::FILEPATH="/opt/cray/netcdf-hdf5parallel/4.4.0/GNU/5.1/include"
>\
>-D Netcdf_LIBRARY_NAMES:STRING="netcdf_parallel" \
>
>Could you still help ?
>
>Best regards,
>Elek
>
>
>
>
>
>
>On Fri, Nov 10, 2017 at 4:13 PM, Eric Bavier <bavier at cray.com> wrote:
>
>> Hello Elek,
>>
>> The 'H5Tget_tag' symbol is defined in libhdf5.a, so you'll need to make
>> sure that the HDF5 TPL includes that library when configuring. FWIW when I
>> build Trilinos with cray-hdf5, I usually set:
>>
>> -D TPL_ENABLE_HDF5:BOOL=ON \
>> -D TPL_HDF5_INCLUDE_DIRS:FILEPATH=${HDF5_DIR}/include \
>> -D HDF5_LIBRARY_DIRS:FILEPATH=${HDF5_DIR}/lib \
>> -D HDF5_LIBRARY_NAMES:STRING="hdf5_hl_parallel;hdf5_parallel;z;dl"
>>
>> Hope that helps,
>>
>> Eric Bavier, Scientific Libraries, Cray Inc.
>>
>> ________________________________________
>> From: Trilinos-Users <trilinos-users-bounces at trilinos.org> on behalf of
>> Eligiusz Postek <ewpostek at googlemail.com>
>> Sent: Thursday, November 9, 2017 18:35
>> To: trilinos-users at trilinos.org
>> Subject: [Trilinos-Users] hdf5
>>
>> Hello,
>> Could it mean that the cray libs (hdf5) are incomplete ?
>> Is there any other error ?
>> best regards, Elek
>>
>>
>> [ 34%] Building CXX object packages/teuchos/parameterlist/src/CMakeFiles/
>> teuchosparameterlist.dir/Teuchos_StringInputStream.cpp.o
>> /opt/cray/hdf5/1.8.16/GNU/5.1/lib/libhdf5.a(H5PL.o): In function
>> `H5PL_load':
>> H5PL.c:(.text+0x634): warning: Using 'dlopen' in statically linked
>> applications requires at runtime the shared libraries from the glibc
>> version used for linking
>> /opt/cray/hdf5/1.8.16/GNU/5.1/lib/libhdf5_hl.a(H5LT.o): In function
>> `H5LT_dtype_to_text':
>> H5LT.c:(.text+0x2871): undefined reference to `H5Tget_tag'
>> /opt/cray/hdf5/1.8.16/GNU/5.1/lib/libhdf5_hl.a(H5LTparse.o): In function
>> `H5LTyyparse':
>> H5LTparse.c:(.text+0x108d): undefined reference to `H5Tset_tag'
>> collect2: error: ld returned 1 exit status
>> make[2]: *** [packages/seacas/applications/nem_slice/nem_slice] Error 1
>>
>>
>-------------- next part --------------
>An HTML attachment was scrubbed...
>URL: <https://trilinos.org/pipermail/trilinos-users/attachments/20171110/4cb6b7bb/attachment-0001.html>
>
>------------------------------
>
>Message: 5
>Date: Fri, 10 Nov 2017 18:53:17 +0000
>From: Eric Bavier <bavier at cray.com>
>To: 'Eligiusz Postek' <ewpostek at googlemail.com>
>Cc: "'trilinos-users at trilinos.org'" <trilinos-users at trilinos.org>
>Subject: Re: [Trilinos-Users] hdf5
>Message-ID:
> <CY4PR11MB17180F35A01E3E6C242D4EF1B7540 at CY4PR11MB1718.namprd11.prod.outlook.com>
>
>Content-Type: text/plain; charset="utf-8"
>
>Hi Elek,
>
>You?ll need to give cmake more of the libraries for netcdf. I use
>
> -D TPL_ENABLE_Netcdf:BOOL=ON \
> -D TPL_Netcdf_INCLUDE_DIRS:FILEPATH=${NETCDF_DIR}/include \
> -D Netcdf_LIBRARY_DIRS:FILEPATH=?${NETCDF_DIR}/lib;${HDF5_DIR}/lib? \
> -D Netcdf_LIBRARY_NAMES:STRING=?netcdf_parallel;hdf5_hl_parallel;hdf5_parallel;z;dl?
>
>The hdf5 libraries need to be included because cmake does not know about netcdf?s dependency on hdf5.
>
>Eric Bavier, Scientific Libraries, Cray Inc.
>
>From: Eligiusz Postek [mailto:ewpostek at googlemail.com]
>Sent: Friday, November 10, 2017 12:33 PM
>To: Eric Bavier
>Cc: trilinos-users at trilinos.org
>Subject: Re: [Trilinos-Users] hdf5
>
>Thank you, HDF5 passed,
>but I still have a problem with netcdf
>
>[ 35%] Building Fortran object packages/seacas/applications/gjoin/CMakeFiles/gjoin.dir/gj_zmfixd.f.o
>[ 35%] Building Fortran object packages/seacas/libraries/plt/CMakeFiles/plt.dir/pltrsd.f.o
>/opt/cray/netcdf-hdf5parallel/4.4.0/GNU/5.1/lib/libnetcdf_parallel.a(libnetcdf4_la-nc4file.o): In function `sync_netcdf4_file':
>/b/ulib/netcdf-support/rpm/BUILD/cray-netcdf-4.4.0-gcc2/netcdf-4.4.0/libsrc4/nc4file.c:3006: undefined reference to `H5Fflush'
>/opt/cray/netcdf-hdf5parallel/4.4.0/GNU/5.1/lib/libnetcdf_parallel.a(libnetcdf4_la-nc4file.o): In function `close_netcdf4_file':
>/b/ulib/netcdf-support/rpm/BUILD/cray-netcdf-4.4.0-gcc2/netcdf-4.4.0/libsrc4/nc4file.c:3090: undefined reference to `H5Fclose'
>/b/ulib/netcdf-support/rpm/BUILD/cray-netcdf-4.4.0-gcc2/netcdf-4.4.0/libsrc4/nc4file.c:3094: undefined reference to `H5Fget_obj
>and ......
>
>I use
> -D TPL_ENABLE_Netcdf:BOOL=ON
>-D Netcdf_LIBRARY_DIRS:FILEPATH="/opt/cray/netcdf-hdf5parallel/4.4.0/GNU/5.1/lib" \
>-D Netcdf_INCLUDE_DIRS::FILEPATH="/opt/cray/netcdf-hdf5parallel/4.4.0/GNU/5.1/include" \
>-D Netcdf_LIBRARY_NAMES:STRING="netcdf_parallel" \
>Could you still help ?
>Best regards,
>Elek
>
>
>
>
>On Fri, Nov 10, 2017 at 4:13 PM, Eric Bavier <bavier at cray.com<mailto:bavier at cray.com>> wrote:
>Hello Elek,
>
>The 'H5Tget_tag' symbol is defined in libhdf5.a, so you'll need to make sure that the HDF5 TPL includes that library when configuring. FWIW when I build Trilinos with cray-hdf5, I usually set:
>
> -D TPL_ENABLE_HDF5:BOOL=ON \
> -D TPL_HDF5_INCLUDE_DIRS:FILEPATH=${HDF5_DIR}/include \
> -D HDF5_LIBRARY_DIRS:FILEPATH=${HDF5_DIR}/lib \
> -D HDF5_LIBRARY_NAMES:STRING="hdf5_hl_parallel;hdf5_parallel;z;dl"
>
>Hope that helps,
>
>Eric Bavier, Scientific Libraries, Cray Inc.
>
>________________________________________
>From: Trilinos-Users <trilinos-users-bounces at trilinos.org<mailto:trilinos-users-bounces at trilinos.org>> on behalf of Eligiusz Postek <ewpostek at googlemail.com<mailto:ewpostek at googlemail.com>>
>Sent: Thursday, November 9, 2017 18:35
>To: trilinos-users at trilinos.org<mailto:trilinos-users at trilinos.org>
>Subject: [Trilinos-Users] hdf5
>
>Hello,
>Could it mean that the cray libs (hdf5) are incomplete ?
>Is there any other error ?
>best regards, Elek
>
>
>[ 34%] Building CXX object packages/teuchos/parameterlist/src/CMakeFiles/teuchosparameterlist.dir/Teuchos_StringInputStream.cpp.o
>/opt/cray/hdf5/1.8.16/GNU/5.1/lib/libhdf5.a(H5PL.o): In function `H5PL_load':
>H5PL.c:(.text+0x634): warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
>/opt/cray/hdf5/1.8.16/GNU/5.1/lib/libhdf5_hl.a(H5LT.o): In function `H5LT_dtype_to_text':
>H5LT.c:(.text+0x2871): undefined reference to `H5Tget_tag'
>/opt/cray/hdf5/1.8.16/GNU/5.1/lib/libhdf5_hl.a(H5LTparse.o): In function `H5LTyyparse':
>H5LTparse.c:(.text+0x108d): undefined reference to `H5Tset_tag'
>collect2: error: ld returned 1 exit status
>make[2]: *** [packages/seacas/applications/nem_slice/nem_slice] Error 1
>
>-------------- next part --------------
>An HTML attachment was scrubbed...
>URL: <https://trilinos.org/pipermail/trilinos-users/attachments/20171110/d25bbbe2/attachment-0001.html>
>
>------------------------------
>
>Message: 6
>Date: Sun, 12 Nov 2017 22:00:47 +0100
>From: Eligiusz Postek <ewpostek at googlemail.com>
>To: "Lofstead, Gerald F II" <gflofst at sandia.gov>, Eric Bavier
> <bavier at cray.com>, trilinos-users at trilinos.org
>Subject: Re: [Trilinos-Users] [EXTERNAL] hdf5
>Message-ID:
> <CA+maUnrypOtTUFjXFAmdePZMH0jNkZRqPiGz-651m_wPiq0KiQ at mail.gmail.com>
>Content-Type: text/plain; charset="utf-8"
>
>Hi,
>The hdf5/netcdf libs are linked.
>Thank you for your good avice.
>
>It appeared a new problem, but with X
>1) tri_x/Trilinos.git> cd build
>getline_int.c:(.text+0x49e8): warning: Using 'getpwuid' in statically
>linked applications requires at runtime the shared libraries from the glibc
>version used for linking
>/usr/lib64/libX11.a(GetDflt.o): In function `GetHomeDir':
>/usr/src/packages/BUILD/libX11-1.1.5/src/GetDflt.c:133: warning: Using
>'getpwnam_r' in statically linked applications requires at runtime the
>shared libraries from the glibc version used for linking
>/usr/src/packages/BUILD/libX11-1.1.5/src/GetDflt.c:135: warning: Using
>'getpwuid_r' in statically linked applications requires at runtime the
>shared libraries from the glibc version used for linking
>The option -static does not help
>
>2) There is an error:
>/usr/lib64/libX11.a(ClDisplay.o): In function `XCloseDisplay':
>/usr/src/packages/BUILD/libX11-1.1.5/src/ClDisplay.c:78: undefined reference
>to `xcb_disconnect'
>/usr/lib64/libX11.a(OpenDis.o): In function `OutOfMemory':
>/usr/src/packages/BUILD/libX11-1.1.5/src/OpenDis.c:929: undefined reference
>to `xcb_disconnect'
>/usr/lib64/libX11.a(OpenDis.o): In function `XOpenDisplay':
>/usr/src/packages/BUILD/libX11-1.1.5/src/OpenDis.c:366: undefined reference
>to `xcb_get_setup'
>/usr/src/packages/BUILD/libX11-1.1.5/src/OpenDis.c:676: undefined reference
>to `xcb_get_maximum_request_length'
>/usr/lib64/libX11.a(xcb_lock.o): In function `_XPutXCBBuffer':
>/usr/src/packages/BUILD/libX11-1.1.5/src/xcb_lock.c:181: undefined
>reference to `xcb_get_request_sent'
>/usr/lib64/libX11.a(xcb_lock.o): In function `issue_complete_request':
>
>and more of this kind,
>
>Could you still give a hand ?
>
>Thank you, Elek
>
>
>
>On Sun, Nov 12, 2017 at 3:04 PM, Lofstead, Gerald F II <gflofst at sandia.gov>
>wrote:
>
>> Hi Elek,
>>
>>
>>
>> Did you get it working?
>>
>>
>>
>> Best,
>>
>>
>>
>> Jay
>>
>>
>>
>> *From: *Eligiusz Postek <ewpostek at googlemail.com>
>> *Date: *Thursday, November 9, 2017 at 6:18 PM
>> *To: *"Lofstead, Gerald F II" <gflofst at sandia.gov>
>> *Subject: *Re: [EXTERNAL] [Trilinos-Users] hdf5
>>
>>
>>
>> Sorry about second mail on this topic.
>>
>> It finds the static libs, but there are shared as well.
>>
>> In consequence, there is the error.
>>
>> Could help with "how to" ?
>>
>> Thank you, Elek
>>
>>
>> - HDF5_LIBRARY_NAMES='hdf5;z;hdf5_hl'
>> -- Searching for libs in HDF5_LIBRARY_DIRS='/opt/cray/
>> hdf5/1.8.16/GNU/5.1/lib'
>> -- Searching for a lib in the set "hdf5":
>> -- Searching for lib 'hdf5' ...
>> -- Found lib '/opt/cray/hdf5/1.8.16/GNU/5.1/lib/libhdf5.a'
>> -- Searching for a lib in the set "z":
>> -- Searching for lib 'z' ...
>> -- Found lib '/usr/lib64/libz.a'
>>
>>
>>
>> On Fri, Nov 10, 2017 at 2:07 AM, Eligiusz Postek <ewpostek at googlemail.com>
>> wrote:
>>
>> Hi Jay,
>>
>> Yes, it looks.
>>
>>
>> I have such lines now. However, it does not change anything.
>>
>> -D CMAKE_CXX_FLAGS:STRING="-O2 -ansi -std=c++11 -shared-libgcc -ftrapv
>> -pedantic -Wall -Wno-long-long" \
>>
>> -D BUILD_SHARED_LIBS:BOOL=ON \
>>
>> Can you give a hint ?
>>
>> Best, Elek
>>
>>
>>
>> On Fri, Nov 10, 2017 at 1:38 AM, Lofstead, Gerald F II <gflofst at sandia.gov>
>> wrote:
>>
>> Hi Elek,
>>
>>
>>
>> It looks to me like the HDF5 library was built assuming shared libraries
>> only and you are trying to link just statically. Is that the case?
>>
>>
>>
>> Best,
>>
>>
>>
>> Jay
>>
>>
>>
>> *From: *Trilinos-Users <trilinos-users-bounces at trilinos.org> on behalf of
>> Eligiusz Postek <ewpostek at googlemail.com>
>> *Date: *Thursday, November 9, 2017 at 5:35 PM
>> *To: *"trilinos-users at trilinos.org" <trilinos-users at trilinos.org>
>> *Subject: *[EXTERNAL] [Trilinos-Users] hdf5
>>
>>
>>
>> Hello,
>>
>> Could it mean that the cray libs (hdf5) are incomplete ?
>>
>> Is there any other error ?
>>
>> best regards, Elek
>>
>>
>>
>> [ 34%] Building CXX object packages/teuchos/parameterlist/src/CMakeFiles/
>> teuchosparameterlist.dir/Teuchos_StringInputStream.cpp.o
>> /opt/cray/hdf5/1.8.16/GNU/5.1/lib/libhdf5.a(H5PL.o): In function
>> `H5PL_load':
>> H5PL.c:(.text+0x634): warning: Using 'dlopen' in statically linked
>> applications requires at runtime the shared libraries from the glibc
>> version used for linking
>> /opt/cray/hdf5/1.8.16/GNU/5.1/lib/libhdf5_hl.a(H5LT.o): In function
>> `H5LT_dtype_to_text':
>> H5LT.c:(.text+0x2871): undefined reference to `H5Tget_tag'
>> /opt/cray/hdf5/1.8.16/GNU/5.1/lib/libhdf5_hl.a(H5LTparse.o): In function
>> `H5LTyyparse':
>> H5LTparse.c:(.text+0x108d): undefined reference to `H5Tset_tag'
>> collect2: error: ld returned 1 exit status
>> make[2]: *** [packages/seacas/applications/nem_slice/nem_slice] Error 1
>>
>>
>>
>>
>>
>-------------- next part --------------
>An HTML attachment was scrubbed...
>URL: <https://trilinos.org/pipermail/trilinos-users/attachments/20171112/25e717ff/attachment-0001.html>
>
>------------------------------
>
>Message: 7
>Date: Sun, 12 Nov 2017 21:07:19 +0000
>From: "Bradley, Andrew Michael" <ambradl at sandia.gov>
>To: Eligiusz Postek <ewpostek at googlemail.com>, "Lofstead, Gerald F II"
> <gflofst at sandia.gov>, Eric Bavier <bavier at cray.com>,
> "trilinos-users at trilinos.org" <trilinos-users at trilinos.org>
>Subject: Re: [Trilinos-Users] [EXTERNAL] hdf5
>Message-ID: <1510520839575.40375 at sandia.gov>
>Content-Type: text/plain; charset="iso-8859-1"
>
>Hi Elek,
>
>It may help if you set
> -D TPL_ENABLE_X11:BOOL=OFF
>
>Cheers,
>Andrew
>
>
>From: Trilinos-Users <trilinos-users-bounces at trilinos.org> on behalf of Eligiusz Postek <ewpostek at googlemail.com>
>Sent: Sunday, November 12, 2017 2:00 PM
>To: Lofstead, Gerald F II; Eric Bavier; trilinos-users at trilinos.org
>Subject: Re: [Trilinos-Users] [EXTERNAL] hdf5
>?
>
>
>Hi,
>The hdf5/netcdf libs are linked.
>Thank you for your good avice.
>
>
>It appeared a new problem, but with X
>1)?tri_x/Trilinos.git> cd? build
>getline_int.c:(.text+0x49e8): warning: Using 'getpwuid' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
>/usr/lib64/libX11.a(GetDflt.o): In function `GetHomeDir':
>/usr/src/packages/BUILD/libX11-1.1.5/src/GetDflt.c:133: warning: Using 'getpwnam_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
>/usr/src/packages/BUILD/libX11-1.1.5/src/GetDflt.c:135: warning: Using 'getpwuid_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
>
>The option -static does not help
>
>
>2) There is an error:
>/usr/lib64/libX11.a(ClDisplay.o): In function `XCloseDisplay':
>/usr/src/packages/BUILD/libX11-1.1.5/src/ClDisplay.c:78: undefined reference to `xcb_disconnect'
>/usr/lib64/libX11.a(OpenDis.o): In function `OutOfMemory':
>/usr/src/packages/BUILD/libX11-1.1.5/src/OpenDis.c:929: undefined reference to `xcb_disconnect'
>/usr/lib64/libX11.a(OpenDis.o): In function `XOpenDisplay':
>/usr/src/packages/BUILD/libX11-1.1.5/src/OpenDis.c:366: undefined reference to `xcb_get_setup'
>/usr/src/packages/BUILD/libX11-1.1.5/src/OpenDis.c:676: undefined reference to `xcb_get_maximum_request_length'
>/usr/lib64/libX11.a(xcb_lock.o): In function `_XPutXCBBuffer':
>/usr/src/packages/BUILD/libX11-1.1.5/src/xcb_lock.c:181: undefined reference to `xcb_get_request_sent'
>/usr/lib64/libX11.a(xcb_lock.o): In function `issue_complete_request':
>
>
>
>and more of this kind,
>
>
>Could you still give a hand ?
>
>
>Thank you, Elek
>
>
>
>
>
>
>On Sun, Nov 12, 2017 at 3:04 PM, Lofstead, Gerald F II <gflofst at sandia.gov> wrote:
>
>
>Hi Elek,
>?
>Did you get it working?
>?
>Best,
>?
>Jay
>?
>
>From: Eligiusz Postek <ewpostek at googlemail.com>
>Date: Thursday, November 9, 2017 at 6:18 PM
>To: "Lofstead, Gerald F II" <gflofst at sandia.gov>
>Subject: Re: [EXTERNAL] [Trilinos-Users] hdf5
>
>?
>
>
>Sorry about second mail on this topic.
>
>
>
>It finds the static libs, but? there are shared as well.
>
>
>
>In consequence, there is the error.
>
>Could help with "how to" ?
>
>Thank you, Elek
>
>
>- HDF5_LIBRARY_NAMES='hdf5;z;hdf5_hl'
>-- Searching for libs in HDF5_LIBRARY_DIRS='/opt/cray/hdf5/1.8.16/GNU/5.1/lib'
>-- Searching for a lib in the set "hdf5":
>--?? Searching for lib 'hdf5' ...
>--???? Found lib '/opt/cray/hdf5/1.8.16/GNU/5.1/lib/libhdf5.a'
>-- Searching for a lib in the set "z":
>--?? Searching for lib 'z' ...
>--???? Found lib '/usr/lib64/libz.a'
>
>
>
>?
>
>On Fri, Nov 10, 2017 at 2:07 AM, Eligiusz Postek <ewpostek at googlemail.com> wrote:
>
>
>Hi Jay,
>
>Yes, it looks.
>
>
>I have such lines now. However, it does not change anything.
>
>-D CMAKE_CXX_FLAGS:STRING="-O2 -ansi -std=c++11 -shared-libgcc -ftrapv -pedantic -Wall -Wno-long-long" \
>
>-D BUILD_SHARED_LIBS:BOOL=ON \
>
>Can you give a hint ?
>
>Best, Elek
>
>
>
>?
>
>On Fri, Nov 10, 2017 at 1:38 AM, Lofstead, Gerald F II <gflofst at sandia.gov> wrote:
>
>
>Hi Elek,
>?
>It looks to me like the HDF5 library was built assuming shared libraries only and you are trying to link just statically. Is that the case?
>?
>Best,
>?
>Jay
>?
>
>From: Trilinos-Users <trilinos-users-bounces at trilinos.org> on behalf of Eligiusz Postek <ewpostek at googlemail.com>
>Date: Thursday, November 9, 2017 at 5:35 PM
>To: "trilinos-users at trilinos.org" <trilinos-users at trilinos.org>
>Subject: [EXTERNAL] [Trilinos-Users] hdf5
>
>
>
>?
>
>
>Hello,
>
>Could it mean that the cray libs (hdf5) are incomplete ?
>
>Is there any other error ?
>
>best regards, Elek
>
>
>
>[ 34%] Building CXX object packages/teuchos/parameterlist/src/CMakeFiles/teuchosparameterlist.dir/Teuchos_StringInputStream.cpp.o
>/opt/cray/hdf5/1.8.16/GNU/5.1/lib/libhdf5.a(H5PL.o): In function `H5PL_load':
>H5PL.c:(.text+0x634): warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
>/opt/cray/hdf5/1.8.16/GNU/5.1/lib/libhdf5_hl.a(H5LT.o): In function `H5LT_dtype_to_text':
>H5LT.c:(.text+0x2871): undefined reference to `H5Tget_tag'
>/opt/cray/hdf5/1.8.16/GNU/5.1/lib/libhdf5_hl.a(H5LTparse.o): In function `H5LTyyparse':
>H5LTparse.c:(.text+0x108d): undefined reference to `H5Tset_tag'
>collect2: error: ld returned 1 exit status
>make[2]: *** [packages/seacas/applications/nem_slice/nem_slice] Error 1
> ?
> ?
>
>
>
>------------------------------
>
>Message: 8
>Date: Wed, 15 Nov 2017 08:01:29 +0100
>From: Eligiusz Postek <ewpostek at googlemail.com>
>To: trilinos-users at trilinos.org
>Subject: [Trilinos-Users] static_warning
>Message-ID:
> <CA+maUnpr1H6gMy4e6+rLj9CCuwJA_i+CwmLRhG_v77NcCVFQuQ at mail.gmail.com>
>Content-Type: text/plain; charset="utf-8"
>
>Dear Colleagues,
>I obtain the libraries and I install them.
>However, I get the warning for static libraries.
>I have -D BUILD_SHARED_LIBS:BOOL=OFF .
>Once I add the option above ON I get comment
>the system does not support it. Not sure, but
>it seems to be specific for cray.
>
>Could you me how to get rid of the warning ?
>Possible it affects building the Peridigm I do.
>
>best regrds, Elek
>
>There are much more this kind of warnings.
>
>[ 13%] Building Fortran object
>packages/seacas/libraries/suplib/CMakeFiles/suplib.dir/ffexst.f.o
>../../libraries/aprepro_lib/libaprepro_lib.a(apr_getline_int.c.o): In
>function `gl_set_home_dir':
>apr_getline_int.c:(.text+0x4a0e): warning: Using 'getpwnam' in statically
>linked applications requires at runtime the shared libraries from the glibc
>version used for linking
>-------------- next part --------------
>An HTML attachment was scrubbed...
>URL: <https://trilinos.org/pipermail/trilinos-users/attachments/20171115/f7db79ed/attachment-0001.html>
>
>------------------------------
>
>Message: 9
>Date: Tue, 21 Nov 2017 15:52:31 +0000
>From: <Nigel.Overmars at dlr.de>
>To: <trilinos-users at trilinos.org>
>Subject: [Trilinos-Users] Transposing a Tpetra::MultiVector
>Message-ID:
> <DD07DFB16C113A47B9FAE7783DA77F9584386C at dlrexmbx02.intra.dlr.de>
>Content-Type: text/plain; charset="us-ascii"
>
>Dear all,
>
>For two suitable MultiVector's mvec1 and mvec2 and two scalars alpha and beta I want to implement
>mvec2 = alpha * mvec1 ^T + beta * mvec2
>Is there a possible way to do this directly in Tpetra? I can (ab)use the multiply member function, but there must be a more canonical way. Even more so because this would mean that I have to make an allocation. The dimensions of the objects I am considering are rather low, but it is still something I wish to prevent.
>
>Best regards,
>Nigel Overmars
>-------------- next part --------------
>An HTML attachment was scrubbed...
>URL: <https://trilinos.org/pipermail/trilinos-users/attachments/20171121/9a6b6ec1/attachment-0001.html>
>
>------------------------------
>
>Message: 10
>Date: Sat, 2 Dec 2017 07:50:48 +0100
>From: Eligiusz Postek <ewpostek at googlemail.com>
>To: trilinos-users at trilinos.org
>Subject: [Trilinos-Users] with_boost
>Message-ID:
> <CA+maUnp6287bvQCa14X4yM69456EUEp7xkWc8cZj0eFLUU-dkg at mail.gmail.com>
>Content-Type: text/plain; charset="utf-8"
>
>Dear Colleagues,
>I encountered an error with boost (1.65, 1.64).
>I use gcc-5.
>This is Trilinos taken from github,
>could you give a hint,
>Best regards,
>Elek
>
>[ 97%] Building CXX object
>packages/stokhos/src/CMakeFiles/stokhos.dir/nox/NOX_Epetra_LinearSystem_SGGS.cpp.o
>In file included from /usr/include/boost/functional/hash/hash.hpp:477:0,
> from /usr/include/boost/functional/hash.hpp:6,
> from /usr/include/boost/unordered/unordered_map.hpp:17,
> from /usr/include/boost/unordered_map.hpp:16,
> from
>/users/work/ewpostek/libs_all/tri_x/trunk/packages/stokhos/src/epetra/Stokhos_AdaptivityManager.hpp:64,
> from
>/users/work/ewpostek/libs_all/tri_x/trunk/packages/stokhos/src/epetra/Stokhos_AdaptivityManager.cpp:44:
>/usr/include/boost/functional/hash/extensions.hpp:38:33: error:
>?template<class T, class A> std::size_t boost::hash_value? conflicts with a
>previous declaration
> std::size_t hash_value(std::list<T, A> const& v);
> ^
>/usr/include/boost/functional/hash/extensions.hpp:36:17: note: previous
>declaration ?namespace boost { }::hash_value?
> std::size_t hash_value(std::vector<T, A> const&);
>-------------- next part --------------
>An HTML attachment was scrubbed...
>URL: <https://trilinos.org/pipermail/trilinos-users/attachments/20171202/da40a9f7/attachment-0001.html>
>
>------------------------------
>
>Message: 11
>Date: Wed, 6 Dec 2017 17:25:41 +0100
>From: Deepak Garg <deepak.garg at ingv.it>
>To: Trilinos-users at trilinos.org
>Subject: [Trilinos-Users] BELOS GMRES
>Message-ID:
> <CAJFVxiCWVeZT5A0XincnU3LDW54wVK=thr55_eeWL7GHH3TOwA at mail.gmail.com>
>Content-Type: text/plain; charset="utf-8"
>
>Hi,
>
>I am using BELOS package of Trilinos (version 12.10.1) to solve my linear
>system in parallel arising in finite element method. I use GMRES method
>with ILU preconditioner following "BlockPrecGmresEpetraExFile.cpp" as an
>example to write a wrapper class in my code. The matrix vector system in my
>numerical problems are usually of size 1 million (entries in vector). The
>entries in system vary from 0.00001 to 10000000.
>
>I have three questions:
>
>1. What should be the optimized values for the GMRES parameters such as
>maxrestarts, maxiters and maxsubspace?
>
>2. Is ILU preconditioner the best one to use with GMRES? Are there other
>preconditioners which can scale the system better than ILU?
>
>3. Which is the fastest linear solver in BELOS package?
>
>Kind regards,
>Deepak
>-------------- next part --------------
>An HTML attachment was scrubbed...
>URL: <https://trilinos.org/pipermail/trilinos-users/attachments/20171206/64b48e4d/attachment.html>
>
>------------------------------
>
>Subject: Digest Footer
>
>_______________________________________________
>Trilinos-Users mailing list
>Trilinos-Users at trilinos.org
>https://trilinos.org/mailman/listinfo/trilinos-users
>
>
>------------------------------
>
>End of Trilinos-Users Digest, Vol 147, Issue 1
>**********************************************
More information about the Trilinos-Users
mailing list