[Trilinos-Users] [EXTERNAL] Anasazi BlockKrylovSchur solver Error in F^H M F == I
Andris Freimanis
andris.freimanis_1 at rtu.lv
Wed Sep 20 09:07:28 EDT 2017
Dear, Heidi,
I experimented a bit with different GMRES settings, preconditioners and
tolerances and finally got both Belos and Anasazi solvers to converge.
At first I tried using tolerance between 1e-10 and 1e-14, but Belos
solver would converge and show loss of accuracy, Anasazi wouldn't
converge. However, Belos converged and didn't lose accuracy when the
tolerance was 1e-8 or looser. This alone helped only for a small 240x240
problem. A larger 75e4x75e4 problem still wouldn't converge, so I
changed solver to flexible GMRES and preconditioner from ILU to ILUT
with larger level of fill (around 60.0 seemed best) and larger drop
tolerance (1e-3). This helped and both solvers converged.
So thanks, Heidi and Alicia, for all your help.
Best regards,
Andris Freimanis
**
On 2017.09.18. 20:07, Thornquist, Heidi K wrote:
> Hi Andris,
>
> I'm glad to hear that the solver converged with the direct solver and
> I understand that this is not ideal for larger eigensystems. However,
> at least we have verified that a simple comparison to scipy is
> correct. The convergence tolerance for Belos is relative, so anything
> more than machine precision is unnecessary. I believe if you place
> the tolerance in the 1e-10 – 1e-14 range then Belos will converge and
> so will Anasazi. Once that happens you will want to play with
> loosening the tolerance of the iterative solver relative to the
> eigensolver.
>
> In practice, I have found that if you are using a static tolerance
> then you need the iterative solver to be 1-2 orders of magnitude
> tighter (smaller) than the requested tolerance for the eigensolver.
> Think of it this way, the block Krylov Schur method in Anasazi is
> building a subspace that approximates the eigenvalues/eigenvectors of
> the inexact operator that is provided by Belos. If the operator
> provided by Belos is less accurate than the requested tolerance of the
> eigenpairs from Anasazi, then the subspace generated by Anasazi will
> have problems converging. This relates to how the inexactness impacts
> the generation of the Krylov subspace in Anasazi's BKS solver.
>
> Hope this helps.
>
> Thanks,
> Heidi
>
> --
>
> Heidi K. Thornquist
> Electrical Models & Simulation
> Sandia National Laboratories
> Albuquerque, NM 87185-1323
>
>
> From: Trilinos-Users <trilinos-users-bounces at trilinos.org
> <mailto:trilinos-users-bounces at trilinos.org>> on behalf of Andris
> Freimanis <andris.freimanis_1 at rtu.lv <mailto:andris.freimanis_1 at rtu.lv>>
> Date: Friday, September 15, 2017 at 2:12 AM
> To: "Thornquist, Heidi K" <hkthorn at sandia.gov
> <mailto:hkthorn at sandia.gov>>, "Klinvex, Alicia Marie"
> <amklinv at sandia.gov <mailto:amklinv at sandia.gov>>
> Cc: Trilinos mailing list <trilinos-users at trilinos.org
> <mailto:trilinos-users at trilinos.org>>
> Subject: Re: [Trilinos-Users] [EXTERNAL] Anasazi BlockKrylovSchur
> solver Error in F^H M F == I
>
> Dear, Heidi, Alicia,
>
> yes, eigs in scipy uses direct LU solver. I tried Amesos KLU from that
> example, fortunately solver converged and the results were the same as
> from scipy's eigs. However, I should have mentioned it earlier, this
> 240x240 system is only a test case. The size of matrices will
> be 10e6x10e6 and larger, so I'll need an iterative solver.
>
> I changed linear solver's tolerance from 1e-3 to 1e-300. Belos shows
> that it converged, however, there's also a message about a loss of
> accuracy.
>
>
> Belos::StatusTestGeneralOutput: Passed
> (Num calls,Mod test,State test): (197, 1, Passed)
> Passed.......OR Combination ->
> OK...........Number of Iterations = 196 < 1000
> Converged....(2-Norm Res Vec) / (2-Norm Prec Res0)
> residual [ 0 ] = 2.94889e-301 < 1e-300
>
> Belos::BlockGmresSolMgr::solve(): Warning! Solver has experienced a
> loss of accuracy!
> Anasazi::BlockKrylovSchurSolMgr::solve() caught unexpected exception
> from Anasazi::BlockKrylovSchur::iterate() at iteration 1
> /usr/local/trilinosDebug/include/AnasaziEpetraAdapter.hpp:1401:
>
> Throw number = 1
>
> Throw test that evaluated to true: ret != 0
>
> Anasazi::OperatorTraits<double,Epetra_Multivector,Epetra_Operator>::Apply():
> Error in Epetra_Operator::Apply(). Code -1
> Anasazi::BlockKrylovSchurSolMgr::solve() returning Unconverged with no
> solutions.
>
> Best regards,
>
> Andris
>
>
>
> On 2017.09.14. 18:15, Thornquist, Heidi K wrote:
>> Hi Andris,
>>
>> If the eigs in scipy is like the one in Matlab, the shift-invert
>> transformation is solved using a direct solver and not an iterative
>> solver. Like Alicia mentioned, this can affect convergence. Given
>> the size of the system, it would be better suited for using Amesos as
>> the linear solver. There is an example in Anasazi that shows how to
>> do this:
>>
>> Trilinos/packages/anasazi/epetra/example/BlockKrylovSchur/BlockKrylovSchurEpetraExGenAmesos.cpp
>>
>>
>> Thanks,
>> Heidi
>>
>> --
>>
>> Heidi K. Thornquist
>> Electrical Models & Simulation
>> Sandia National Laboratories
>> Albuquerque, NM 87185-1323
>>
>>
>> From: Trilinos-Users <trilinos-users-bounces at trilinos.org
>> <mailto:trilinos-users-bounces at trilinos.org>> on behalf of "Klinvex,
>> Alicia Marie" <amklinv at sandia.gov <mailto:amklinv at sandia.gov>>
>> Date: Thursday, September 14, 2017 at 8:31 AM
>> To: Andris Freimanis <andris.freimanis_1 at rtu.lv
>> <mailto:andris.freimanis_1 at rtu.lv>>
>> Cc: Trilinos mailing list <trilinos-users at trilinos.org
>> <mailto:trilinos-users at trilinos.org>>
>> Subject: Re: [Trilinos-Users] [EXTERNAL] Anasazi BlockKrylovSchur
>> solver Error in F^H M F == I
>>
>> Hello Andris,
>>
>> Your linear solver is converging; the eigensolver is not. You said
>> this problem was being solved correctly by scipi. It’s possible that
>> the eigensolver is failing to converge because your linear systems
>> are not being solved accurately enough. (Block Krylov Schur is
>> sensitive to that.) Do you know how scipi is solving the linear
>> systems?
>>
>> Best wishes,
>>
>> Alicia
>>
>> *From:*Andris Freimanis [mailto:andris.freimanis_1 at rtu.lv]
>> *Sent:* Thursday, September 14, 2017 9:20 AM
>> *To:* Klinvex, Alicia Marie <amklinv at sandia.gov
>> <mailto:amklinv at sandia.gov>>
>> *Subject:* Re: [EXTERNAL] [Trilinos-Users] Anasazi BlockKrylovSchur
>> solver Error in F^H M F == I
>>
>> Dear, Alicia,
>>
>> I increased the number of blocks to 48 and to 239 (matrix size - 1),
>> sadly solver still didn't converge. It's interesting that residual is
>> less than the tolerance, but it shows no convergence.
>>
>> ================================================================================
>>
>> - StatusTestOutput: Failed
>> (Num calls,Mod test,State test): (1, 1, Passed Failed Undefined )
>> - StatusTestCombo: Failed
>> - StatusTestWithOrdering: Undefined
>> Quorum: 1
>> Auxiliary values: [empty]
>> - StatusTestResNorm: Failed
>> (Tolerance,WhichNorm,Scaled,Quorum):
>> (1.000000e-03,RITZRES_2NORM,true,1)
>> Which vectors: [empty]
>>
>> *******************************************************
>> ***** Belos Iterative Solver: Block Gmres
>> ***** Maximum Iterations: 1000
>> ***** Block Size: 1
>> ***** Residual Test:
>> ***** Test 1 : Belos::StatusTestImpResNorm<>: (2-Norm Res Vec) /
>> (2-Norm Prec Res0), tol = 0.001
>> *******************************************************
>> Iter 82, [ 1] : 7.842584e-04
>> Belos::BlockGmresSolMgr::solve(): Warning! Solver has experienced a
>> loss of accuracy!
>> Passed.......OR Combination ->
>> OK...........Number of Iterations = 82 < 1000
>> Converged....(2-Norm Res Vec) / (2-Norm Prec Res0)
>> residual [ 0 ] = 7.842584e-04 < 1.000000e-03
>>
>> ================================================================================
>>
>> On 2017.09.14. 16:09, Klinvex, Alicia Marie wrote:
>>
>> Hello Andris,
>>
>> Can you try increasing your number of blocks? You allowed GMRES
>> to store a subspace of size 24, and you allowed it 10 restarts,
>> which is why it did not attempt 1000 iterations.
>>
>> Best wishes,
>> Alicia
>>
>> *From:*Andris Freimanis [mailto:andris.freimanis_1 at rtu.lv]
>> *Sent:* Thursday, September 14, 2017 9:03 AM
>> *To:* Klinvex, Alicia Marie <amklinv at sandia.gov>
>> <mailto:amklinv at sandia.gov>; Trilinos mailing list
>> <trilinos-users at trilinos.org> <mailto:trilinos-users at trilinos.org>
>> *Subject:* RE: [EXTERNAL] [Trilinos-Users] Anasazi
>> BlockKrylovSchur solver Error in F^H M F == I
>>
>> Dear, Alicia,
>>
>> thank you for your response.
>>
>> M is a part of a generalized eigenvalue problem A*x = lambda*M*x.
>> Since A is real non-hermitian I'm using Block Krylov Schur
>> eigensolver with shift-invert transformation as shown in
>> BlockKrylovSchurEpetraExGenBelos.cpp example. And block GMRES
>> linear solver with ILU preconditioner for the inverted part.
>>
>> Linear solver parameters are:
>> Block size 1
>> Number of blocks 24
>> Max iterations 1000 (if I understand the output correctly,
>> only 264 were used)
>> Max restarts 10
>> Tolerance 1e-3
>> ILU preconditioner parameters were:
>> Drop tolerance 1e-9
>> Level-of-fill 1
>> Schwartz: combine mode "Add"
>>
>> Scipy's eigs function, which uses Arpack, solved this problem
>> easily. So most likely I have made some mistake with Trilinos
>> solvers.
>>
>> Best regards,
>>
>> Andris
>>
>> On 2017.09.14. 15:06, Klinvex, Alicia Marie wrote:
>>
>> Hello Andris,
>>
>> I’m sorry to hear you’re having trouble with Anasazi. The
>> “error” you’re getting isn’t an error in the programming
>> sense. It’s an error in the mathematical sense. You turned
>> on debugging output, so it’s printing information that’s
>> useful to the developers. That message is telling us that
>> F^H M F is approximately equal to I in that the error F^H M F
>> - I is small.
>>
>> Can you tell us about the linear solver you’re using? What
>> are the tolerance and maximum number of iterations? Also, is
>> M a preconditioner, or is M part of the generalized
>> eigenvalue problem Ax = \lambda M x?
>>
>> Best wishes,
>>
>> Alicia
>>
>> *From:*Trilinos-Users
>> [mailto:trilinos-users-bounces at trilinos.org] *On Behalf Of
>> *Andris Freimanis
>> *Sent:* Thursday, September 14, 2017 1:18 AM
>> *To:* Trilinos mailing list <trilinos-users at trilinos.org>
>> <mailto:trilinos-users at trilinos.org>
>> *Subject:* [EXTERNAL] [Trilinos-Users] Anasazi
>> BlockKrylovSchur solver Error in F^H M F == I
>>
>> Hello,
>>
>> I'm trying to calculate few smallest eigenvalues of a linear
>> system, where A - nonsymmetric sparse and M - sparse
>> symmetric positive-definite, using Anasazi's Block Krylov
>> Schur solver with shift-invert transformation that is solved
>> by Belos's block GMRES solver. Both test matrices are
>> 240x240 in size.
>>
>> Unfortunately, I'm getting this error: >> Error in F^H M F
>> == I : 4.44e-16 (copied the full error message below), and
>> can't find info on this in neither Anasazi nor Belos examples
>> or documentation. Can you please take a look at the error
>> message below?
>>
>> This problem is similar to the one here:
>> https://trilinos.org/pipermail/trilinos-users/2015-December/005270.html.
>> However, the suggestion to use shift-invert doesn't help as
>> I'm already doing that.
>>
>> Best regards,
>>
>> Andris Freimanis
>>
>> Debugging checks: iteration 0: after initialize()
>> >> Error in F^H M F == I : 4.44e-16
>>
>> ================================================================================
>>
>> BlockKrylovSchur Solver Status
>>
>> The solver is initialized.
>> The number of iterations performed is 0
>> The block size is 1
>> The number of blocks is 24
>> The current basis size is 0
>> The number of auxiliary vectors is 0
>> The number of operations Op*x is 0
>>
>> CURRENT RITZ VALUES
>> [ NONE COMPUTED ]
>>
>> ================================================================================
>>
>> - StatusTestOutput: Failed
>> (Num calls,Mod test,State test): (1, 1, Passed Failed
>> Undefined )
>> - StatusTestCombo: Failed
>> - StatusTestWithOrdering: Undefined
>> Quorum: 5
>> Auxiliary values: [empty]
>> - StatusTestResNorm: Failed
>> (Tolerance,WhichNorm,Scaled,Quorum):
>> (1.000000e-03,RITZRES_2NORM,true,5)
>> Which vectors: [empty]
>> Performing restart number 1 of 10
>>
>> Performing restart number 2 of 10
>>
>> Performing restart number 3 of 10
>>
>> Performing restart number 4 of 10
>>
>> Performing restart number 5 of 10
>>
>> Performing restart number 6 of 10
>>
>> Performing restart number 7 of 10
>>
>> Performing restart number 8 of 10
>>
>> Performing restart number 9 of 10
>>
>> Performing restart number 10 of 10
>>
>> Failed.......OR Combination ->
>> OK...........Number of Iterations = 264 < 1000
>> Unconverged..(2-Norm Res Vec) / (2-Norm Prec Res0)
>> residual [ 0 ] = 2.855202e-01 > 1.000000e-03
>>
>> ================================================================================
>>
>> TimeMonitor results over 1 processor
>>
>> Timer Name Global time (num calls)
>> --------------------------------------------------------------------------------
>> Anasazi: BlockKrylovSchur::Computing Ritz vectors 0 (0)
>> Anasazi: BlockKrylovSchur::Computing Schur form 0 (0)
>> Anasazi: BlockKrylovSchur::Operation Op*x 0 (1)
>> Anasazi: BlockKrylovSchur::Orthogonalization 0.0001295 (1)
>> Anasazi: BlockKrylovSchur::Sorting Ritz values 0 (0)
>> Anasazi: BlockKrylovSchur::Sorting Schur form 0 (0)
>> Anasazi: BlockKrylovSchurSolMgr restarting 0 (0)
>> Anasazi: BlockKrylovSchurSolMgr::solve() 0 (1)
>> Belos: BlockGmresSolMgr total solve time 0.03066 (1)
>> Belos: Operation Op*x 0.005605 (275)
>> --------------------------------------------------------------------------------
>> Belos: Operation Prec*x 0.007409 (275)
>> Belos: Orthogonalization 0.01177 (275)
>> Epetra_CrsMatrix::Multiply(TransA,X,Y) 0.005537
>> (280)
>> Epetra_CrsMatrix::Solve(Upper,Trans,UnitDiag,X,Y) 0.006862
>> (550)
>> Ifpack_ILU::ApplyInverse 0.007275
>> (275)
>> Ifpack_ILU::ApplyInverse - Solve 0.007074
>> (275)
>> Ifpack_ILU::Compute 0.002538
>> (1)
>> Ifpack_ILU::ComputeSetup
>> 0.0008707 (1)
>> Ifpack_ILU::Initialize
>> 0.0004559 (1)
>> ================================================================================
>> Anasazi::BlockKrylovSchurSolMgr::solve() caught unexpected
>> exception from Anasazi::BlockKrylovSchur::iterate() at
>> iteration 1
>> /usr/local/trilinos/include/AnasaziEpetraAdapter.hpp:1401:
>>
>> Throw number = 1
>>
>> Throw test that evaluated to true: ret != 0
>>
>> Anasazi::OperatorTraits<double,Epetra_Multivector,Epetra_Operator>::Apply():
>> Error in Epetra_Operator::Apply(). Code -1
>> Anasazi::BlockKrylovSchurSolMgr::solve() returning
>> Unconverged with no solutions.
>>
>> --
>>
>> Andris Freimanis
>>
>> PhD student
>>
>>
>>
>> Riga Technical university
>>
>> Institute of transporation
>>
>> engineering
>>
>>
>>
>> ??psalas iela 6a, Riga, Latvia, LV1048
>>
>> www.rtu.lv <http://www.rtu.lv>
>>
>>
>>
>>
>> --
>>
>> Andris Freimanis
>>
>> PhD student
>>
>>
>>
>> Riga Technical university
>>
>> Institute of transporation
>>
>> engineering
>>
>>
>>
>> ??psalas iela 6a, Riga, Latvia, LV1048
>>
>> www.rtu.lv <http://www.rtu.lv>
>>
>>
>>
>> --
>> Andris Freimanis
>> PhD student
>> Riga Technical university
>> Institute of transporation
>> engineering
>> ??psalas iela 6a, Riga, Latvia, LV1048
>> www.rtu.lv <http://www.rtu.lv>
>
> --
> Andris Freimanis
> PhD student
>
> Riga Technical university
> Institute of transporation
> engineering
>
> ??psalas iela 6a, Riga, Latvia, LV1048
> www.rtu.lv
--
Andris Freimanis
PhD student
Riga Technical university
Institute of transporation
engineering
Ķīpsalas iela 6a, Riga, Latvia, LV1048
www.rtu.lv
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://trilinos.org/pipermail/trilinos-users/attachments/20170920/099d59ec/attachment.html>
More information about the Trilinos-Users
mailing list