[Trilinos-Users] MKL / ACML
Alireza Nejadmalayeri
Alireza.Nejadmalayeri at Colorado.EDU
Fri Dec 10 22:00:07 MST 2010
Thank you all so much for the hints and suggestions.
I have tried some different possibilities; Ross and Jonathan's suggestions work and Kelly's approach *partially* works (compiled but returns segmentation fault while running)
Let me summarize as follows:
1)
-D BLAS_LIBRARY_DIRS:PATH=$MKL_HOME/lib/em64t \
-D LAPACK_LIBRARY_DIRS:PATH=$MKL_HOME/lib/em64t \
Result: configuration will find /usr/lib64/liblapack.so and /usr/lib64/libblas.so
But these are not part of MKL
2)
-D BLAS_LIBRARY_DIRS:PATH=$MKL_HOME/lib/em64t \
-D LAPACK_LIBRARY_DIRS:PATH=$MKL_HOME/lib/em64t \
-D LAPACK_LIBRARY_NAMES:STRING="libmkl_intel_lp64;libmkl_intel_thread;libmkl_core;libiomp5" \
-D BLAS_LIBRARY_NAMES:STRING="libmkl_intel_lp64;libmkl_intel_thread;libmkl_core;libiomp5" \
OR
-D BLAS_LIBRARY_DIRS:PATH=$MKL_HOME/lib/em64t \
-D LAPACK_LIBRARY_DIRS:PATH=$MKL_HOME/lib/em64t \
-D LAPACK_LIBRARY_NAMES:STRING="lmkl_intel_lp64;lmkl_intel_thread;lmkl_core;liomp5" \
-D BLAS_LIBRARY_NAMES:STRING="lmkl_intel_lp64;lmkl_intel_thread;lmkl_core;liomp5" \
i.e., LIBRARY_NAMES set the same as MKL environment variable
Result: configuration fails. LAPACK and BLAS can not be found
3)
With Natarajan's script, I got ld: cannot find -lmkl_intel_lp64 while compiling my code.
4)
Based on what Kelly recommended:
I used the TPL_BLAS_LIBRARIES:STRING and TPL_LAPACK_LIBRARIES:STRING
With this script, I could configure successfully, built the Trilinos successfully, and compiled my code using both cmake and makefile.
BUT, both exe files return "Segmentation fault"!?
Perhaps, when pthread comes first, the link is broken?!
5)
Jonathan and Ross suggestion:
-D BLAS_LIBRARY_DIRS:PATH=$MKL_HOME/lib/em64t \
-D LAPACK_LIBRARY_DIRS:PATH=$MKL_HOME/lib/em64t \
-D LAPACK_LIBRARY_NAMES:STRING="mkl_intel_lp64;mkl_intel_thread;mkl_core;iomp5" \
-D BLAS_LIBRARY_NAMES:STRING="mkl_intel_lp64;mkl_intel_thread;mkl_core;iomp5" \
This one is working.
Thank you,
alireza
----- Original Message -----
From: "Jonathan Hu" <jhu at sandia.gov>
To: <trilinos-users at software.sandia.gov>
Sent: Friday, December 10, 2010 9:07 AM
Subject: Re: [Trilinos-Users] MKL / ACML
> All,
>
> Ross' recommendation should work. I've seen the "-l-L" when I
> use the Trilinos makefile.export system. Once I started specifying the
> blas/lapack as is recommended in Trilinos10CMakeQuickstart.txt, this
> error went away. Please try the following in your cmake script:
>
> \
> -D
> BLAS_LIBRARY_DIRS:STRING="/usr/local/intel/Compiler/11.1/064/mkl/lib/em64t;/usr/local/intel/Compiler/11.1/064/lib/intel64;/usr/lib64"
> \
> -D BLAS_LIBRARY_NAMES:STRING="mkl_intel_lp64; mkl_intel_thread;
> mkl_core; iomp5; pthread" \
> -D
> LAPACK_LIBRARY_DIRS:STRING="/usr/local/intel/Compiler/11.1/064/mkl/lib/em64t;/usr/local/intel/Compiler/11.1/064/lib/intel64;/usr/lib64"
> \
> -D LAPACK_LIBRARY_NAMES:STRING="mkl_intel_lp64; mkl_intel_thread;
> mkl_core; iomp5; pthread" \
>
>
> I've been using this for day-to-day compilation against Trilinos dev.
>
> Regards,
> Jonathan
>
> trilinos-users-request at software.sandia.gov wrote on 12/10/2010 08:40 AM:
>>
>> Message: 2
>> Date: Fri, 10 Dec 2010 09:30:08 -0700
>> From: "Bartlett, Roscoe A"<rabartl at sandia.gov>
>> Subject: Re: [Trilinos-Users] MKL / ACML
>> To: "Meyer, Hal"<hxm at sandia.gov>, Tommy Binford<tbinford at gmail.com>,
>> Alireza Nejadmalayeri<Alireza.Nejadmalayeri at colorado.edu>
>> Cc: "trilinos-users at software.sandia.gov"
>> <trilinos-users at software.sandia.gov>
>> Message-ID:
>> <9C5EDABC60AD90488D506008E9277E4A20BAA1C457 at ES02SNLNT.srn.sandia.gov>
>> Content-Type: text/plain; charset="iso-8859-1"
>>
>> Hello guys,
>>
>> Have you seen the documentation on setup in TPLs at:
>>
>> http://trilinos.sandia.gov/Trilinos10CMakeQuickstart.txt
>>
>> ?
>>
>> I think that setting:
>>
>> TPL_BLAS_LIBRARIES="-L$(DEPEND_INSTALL)/blas -lblas"
>>
>> is not supported in general given the way that CMake works. Where did you find this approach documented? Does CMake claim to support this approach in its find_library(...) function? From looking at the documentation at:
>>
>> http://www.cmake.org/cmake/help/cmake-2-8-docs.html#command:find_library
>>
>> it does not really mention this at all.
>>
>> Instead I would suggest:
>>
>> -DTPL_BLAS_LIBRARIES="$(DEPEND_INSTALL)/blas/libblas.a"
>>
>> or whatever the full path to the BLAS library(s) is or:
>>
>> -DBLAS_LIBRARY_DIRS:PATH=$(DEPEND_INSTALL)/blas
>>
>> Actually, the last approach is really the right way to change the library search path.
>>
>> Please give BLAS_LIBRARY_DIRS a try and see if that solves your problem. If not, please let us know because it should. That is really the right way to do this.
>>
>> Cheers,
>>
>> - Ross
>>
>>
>>
>>
>>> -----Original Message-----
>>> From: trilinos-users-bounces at software.sandia.gov [mailto:trilinos-
>>> users-bounces at software.sandia.gov] On Behalf Of Meyer, Hal
>>> Sent: Friday, December 10, 2010 8:34 AM
>>> To: Tommy Binford; Alireza Nejadmalayeri
>>> Cc: trilinos-users at software.sandia.gov
>>> Subject: Re: [Trilinos-Users] MKL / ACML
>>>
>>> I am seeing the same problem on a gnu/linux system where I have built
>>> blas and lapack from source. The error is in Makefile.export.Trilinos.
>>>
>>> The location of my blas/lapack are defined below:
>>>
>>> BLAS_LIB := -L$(DEPEND_INSTALL)/blas -lblas
>>> LAPACK_LIB := -L$(DEPEND_INSTALL)/lapack -llapack
>>>
>>> The cmake option is set here:
>>>
>>> cd $(TRILINOS_BUILD_DIR)&& ($(MODULE_COMMANDS) $(CMAKE) \
>>> ......
>>> -D TPL_BLAS_LIBRARIES:STRING="$(BLAS_LIB)" \
>>> -D TPL_LAPACK_LIBRARIES:STRING="$(LAPACK_LIB)" \
>>> $(TRILINOS_SRC_DIR))
>>>
>>> Trilinos builds without error, but the link fails with the following
>>> error.
>>>
>>> /usr/bin/ld: cannot find -l-
>>> L/scratch1/hxm/aleph/build/tsali/aleph.2472/user/machines/tsali/lapack
>>>
>>> In my Makefile I am including:
>>>
>>> -include $(TRILINOS_DIR)/include/Makefile.export.Trilinos
>>>
>>> When I look at the include file, the error is in that file.
>>>
>>> grep -i lapack Makefile.export.*
>>> Makefile.export.Trilinos:Trilinos_TPL_LIBRARIES=
>>> /scratch1/hxm/aleph/build/tsali/aleph.2472/user/machines/tsali/exodus/l
>>> ib/libnetcdf.a -l-
>>> L/scratch1/hxm/aleph/build/tsali/aleph.2472/user/machines/tsali/lapack
>>> -llapack -l-
>>> L/scratch1/hxm/aleph/build/tsali/aleph.2472/user/machines/tsali/blas -
>>> lblas
>>>
>>> Note the -l-L after libnetcdf.a. This file is created by trilinos.
>>>
>>> The work around I started using was to specify the full path, not "-
>>> L... -l ...", but I would rather use the "-L.. -l.." format.
>>>
>>> I hope this is helpful.
>>> Hal
>>>
>>> _______________________________________
>>> From: trilinos-users-bounces at software.sandia.gov [trilinos-users-
>>> bounces at software.sandia.gov] On Behalf Of Tommy Binford
>>> [tbinford at gmail.com]
>>> Sent: Friday, December 10, 2010 07:21
>>> To: Alireza Nejadmalayeri
>>> Cc: trilinos-users at software.sandia.gov
>>> Subject: Re: [Trilinos-Users] MKL / ACML
>>>
>>> I'm not sure because I've never seen the -l-L option, but I think you
>>> might have a problem in your makefile prepending a "-l" onto your
>>> library paths. My guess is ld doesn't know what
>>>
>>> -l-L/opt/intel/mkl/10.2.5.035/lib/em64t
>>>
>>> means. It should probably be
>>>
>>> -L/opt/intel/mkl/10.2.5.035/lib/em64t
>>>
>>> This also happens on
>>>
>>> -l-L/opt/intel/mkl/10.2.5.035/lib/em64t -lmkl_intel_lp64
>>>
>>> Hope this helps.
>>>
>>> TLB
>>>
>>> On Thu, Dec 9, 2010 at 11:52 PM, Alireza Nejadmalayeri
>>> <Alireza.Nejadmalayeri at colorado.edu> wrote:
>>>> Thanks, Chris. I have tried something similar. It works as long as I
>>> use
>>>> cmake. But if I compile my code using makefile, I'll get:
>>>>
>>>> /apps/rhel5/mpich2-1.3.1/64/nemesis-intel-11.1.072/bin/mpicxx -c -g -
>>> O0 -g
>>>> -I/usr/rmt_share/scratch95/a/alireza/The_Trilions_Project/trilinos-
>>> 10.6.1__Install_mpi_wrapper_debug_MKL2/include
>>>> -DMYAPP_EPETRA main.cpp
>>>> icpc: command line warning #10120: overriding '-O3' with '-O0'
>>>> /usr/bin/ar cr libmyappLib.a main.o
>>>> /apps/rhel5/mpich2-1.3.1/64/nemesis-intel-11.1.072/bin/mpicxx -g -O0
>>> -g
>>>> -I/usr/rmt_share/scratch95/a/alireza/The_Trilions_Project/trilinos-
>>> 10.6.1__Install_mpi_wrapper_debug_MKL2/include
>>>> -DMYAPP_EPETRA
>>>> -L/usr/rmt_share/scratch95/a/alireza/The_Trilions_Project/trilinos-
>>> 10.6.1__Install_mpi_wrapper_debug_MKL2/lib
>>>> libmyappLib.a -laztecoo -ltpetrainout -ltpetra -ltriutils -lepetra
>>>> -lkokkoslinalg -lkokkosnodeapi -lkokkos -ltpi -lteuchos
>>>> -l-L/opt/intel/mkl/10.2.5.035/lib/em64t -lmkl_intel_lp64 -lmkl_core
>>>> -lmkl_sequential -l-L/opt/intel/mkl/10.2.5.035/lib/em64t -
>>> lmkl_intel_lp64
>>>> -lmkl_core -lmkl_sequential -o MyApp.exe
>>>> icpc: command line warning #10120: overriding '-O3' with '-O0'
>>>> ld: cannot find -l-L/opt/intel/mkl/10.2.5.035/lib/em64t
>>>>
>>>> Any thoughts/suggestions?
>>>>
>>>> Thank you,
>>>> alireza
>>>>
>>>>
>>>> ----- Original Message -----
>>>> From: "Baker, Christopher G."<bakercg at ornl.gov>
>>>> To:<Alireza.Nejadmalayeri at colorado.edu>;
>>>> <trilinos-users at software.sandia.gov>
>>>> Sent: Thursday, December 09, 2010 6:16 PM
>>>> Subject: RE: [Trilinos-Users] MKL / ACML
>>>>> The following will link in the sequential BLAS:
>>>>>
>>>>> set SEQMKL="-L${MKLROOT}/lib/em64t -lmkl_intel_lp64 -lmkl_core
>>>>> -lmkl_sequential"
>>>>>
>>>>>
>>>>> cmake ......
>>>>> ? -D
>>> TPL_BLAS_LIBRARIES:STRING="${SEQMKL}"???????????????????????????? \
>>>>> ? -D
>>> TPL_LAPACK_LIBRARIES:STRING="${SEQMKL}"?????????????????????????? \
>>>>>
>>>>>
>>>>> ?-----Original Message-----
>>>>> From: Alireza Nejadmalayeri
>>> [mailto:Alireza.Nejadmalayeri at colorado.edu]
>>>>> Sent: Thursday, December 09, 2010 06:39 PM Eastern Standard Time
>>>>> To: trilinos-users at software.sandia.gov
>>>>> Subject: [Trilinos-Users] MKL / ACML
>>>>>
>>>>> Hello,
>>>>>
>>>>> I wonder if anyone has built Trilinos with Intel MKL or ACML
>>> libraries
>>>>> (for BLAS and LAPACK). I would like to have a sample configure
>>> script since
>>>>> mine is causing various problems.
>>>>>
>>>>> Thank you,
>>>>> alireza
>>>>>
>>>>>
>>>>>
>>>>>
>>>> _______________________________________________
>
> _______________________________________________
> Trilinos-Users mailing list
> Trilinos-Users at software.sandia.gov
> http://software.sandia.gov/mailman/listinfo/trilinos-users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://software.sandia.gov/pipermail/trilinos-users/attachments/20101210/04f13f0f/attachment-0001.html
More information about the Trilinos-Users
mailing list