[Trilinos-Users] ilut setting

Heroux, Michael A maherou at sandia.gov
Mon Sep 23 10:13:37 MDT 2013


Ross' approach is a reasonable path.  AztecOO does have a SetPreconditioner, ConstructPrecondition, and DestroyPreconditioner method.  But it does not have an ApplyPreconditioner.  It would be very straightforward to create such a method using ConstructPreconditioner as the starting point, but I hesitate add code to AztecOO  if the suggested approach that Ross outlined works for you.

Mike

From: <Bartlett>, Ross Bartlett <bartlettra at ornl.gov<mailto:bartlettra at ornl.gov>>
Date: Monday, September 23, 2013 10:55 AM
To: Eric Cyr <eccyr at sandia.gov<mailto:eccyr at sandia.gov>>, Hoang Giang Bui <hgbk2008 at gmail.com<mailto:hgbk2008 at gmail.com>>
Cc: "trilinos-users at software.sandia.gov<mailto:trilinos-users at software.sandia.gov>" <trilinos-users at software.sandia.gov<mailto:trilinos-users at software.sandia.gov>>
Subject: Re: [Trilinos-Users] [EXTERNAL] Re: ilut setting

I think NOX uses AztecOO as a precoditioner by doing a single GMRES iteration.  Roger Pawlowski would know more.

Stratimikos does not directly support such a usage but you could use Stratimikos and Thyra to construct such a thing.  You would use one Stratikmikos linear solver builder to create an AztecOO LinearOpWtihSolveBase object with GMRES and a single iteration then you would wrap this within a Thyra::DefaultInverseLinearOp object (see http://trilinos.sandia.gov/packages/docs/dev/packages/thyra/doc/html/classThyra_1_1DefaultInverseLinearOp.html ).  You would then use another Stratimikos linear solver build to create a Thyra::LinearOpWithSolveFactoryBase object (i.e. Belos) and then create a new LinearOpWithSolveBase object where you set the AztecOO GMRESS solver set that as a preconditioner using Thyra::LinearOpWithSolveFactoryBase::initializePreconditionedOp() (see http://trilinos.sandia.gov/packages/docs/dev/packages/thyra/doc/html/classThyra_1_1LinearOpWithSolveFactoryBase.html#a3c0e4b3df20383b09eb0f4b671576783 ).

This is only a few lines of code once you have the two Stratimikos::DefaultLinearSolverBuilder objects created.

-Ross

From: trilinos-users-bounces at software.sandia.gov<mailto:trilinos-users-bounces at software.sandia.gov> [mailto:trilinos-users-bounces at software.sandia.gov] On Behalf Of Cyr, Eric C
Sent: Monday, September 23, 2013 11:02 AM
To: Hoang Giang Bui
Cc: trilinos-users at software.sandia.gov<mailto:trilinos-users at software.sandia.gov>
Subject: Re: [Trilinos-Users] [EXTERNAL] Re: ilut setting

Bui

So to clarify you would like to use only the preconditioners within AztecOO? Furthermore you want to apply this repeatedly with out rebuilding at each iteration?

I'm not sure if only the preconditioners from AztecOO can be used, can someone else answer this?

Eric

On Sep 23, 2013, at 5:28 AM, "Hoang Giang Bui" <hgbk2008 at gmail.com<mailto:hgbk2008 at gmail.com>> wrote:
I think the problem lies in line 1044 of Thyra_AztecOOLinearOpWithSolveFactory.cpp. The ConstructPreconditioner try to set AZ_pre_calc to AZ_reuse so the method failed as the previous message showed. However, I'm not sure that uncomment the next line will do the thing correctly. Because I tried to uncomment and it can pass the AZ_pre_calc error but the preconditioner take so long to compute. I think the Stratimikos's AztecOO interface try to run several iteration steps to compute the approximation instead of applying the preconditioner directly. This is not what I want, I think there should be option to apply AztecOO's preconditioner directly instead of trial solving steps. This will lead to recomputing the preconditioner at every iteration steps of outer loop.

Bui

This is the parameter list I used for my test:
<ParameterList name="AztecOO">
    <Parameter name="Type" type="string" value="AztecOO"/>
    <ParameterList name="Forward Solve">
        <ParameterList name="AztecOO Settings">
          <Parameter name="Aztec Solver" type="string" value="BiCGStab"/>
          <Parameter name="Aztec Preconditioner" type="string" value="ilut"/>
          <Parameter name="Overlap" type="int" value="1"/>
          <Parameter name="Graph Fill" type="int" value="0"/>
          <Parameter name="Drop Tolerance" type="double" value="0.0"/>
          <Parameter name="Fill Factor" type="double" value="10.0"/>
          <Parameter name="Steps" type="int" value="3"/>
          <Parameter name="Polynomial Order" type="int" value="3"/>
          <Parameter name="RCM Reordering" type="string" value="Enabled"/>
          <Parameter name="Orthogonalization" type="string" value="Classical"/>
          <Parameter name="Size of Krylov Subspace" type="int" value="1000"/>
          <Parameter name="Convergence Test" type="string" value="r0"/>
          <Parameter name="Ill-Conditioning Threshold" type="double" value="1e+11"/>
          <Parameter name="Output Frequency" type="int" value="1"/>
        </ParameterList>
        <Parameter name="Max Iterations" type="int" value="1"/>
        <Parameter name="Tolerance" type="double" value="1e-9"/>
    </ParameterList>
    <Parameter name="Output Every RHS" type="bool" value="false"/>
    <ParameterList name="VerboseObject">
        <Parameter name="Output File" type="string" value="none"/>
        <Parameter name="Verbosity Level" type="string" value="none"/>
    </ParameterList>
  </ParameterList>


On 09/20/2013 06:04 PM, Cyr, Eric C wrote:
I don't know why. I've never used the native Aztec preconditioners and always use IFPACK (though not the ILUT). Its possible (likely even) that the default AztecOO parameters specified in Stratimikos don't satisfy the requirements of Aztec as stated in the error text you sent below. I'd look at AZ_pre_calc and try to see if its set to "reuse". If so I think thats the problem.

Eric

From: Hoang Giang Bui <hgbk2008 at gmail.com<mailto:hgbk2008 at gmail.com>>
Date: Friday, September 20, 2013 3:04 AM
To: Eric Cyr <eccyr at sandia.gov<mailto:eccyr at sandia.gov>>
Subject: Re: [EXTERNAL] Re: [Trilinos-Users] ilut setting


Hi again,

I realized that this was not correct since Thyra::LinearOpBase and Epetra_Operator are not incompatible. I fixed that using Thyra::block1x1 and it seems working. However with this setting

<ParameterList name="AztecOO">
    <Parameter name="Type" type="string" value="AztecOO"/>
    <ParameterList name="Forward Solve">
        <ParameterList name="AztecOO Settings">
          <Parameter name="Aztec Solver" type="string" value="BiCGStab"/>
          <Parameter name="Aztec Preconditioner" type="string" value="ilut"/>
          <Parameter name="Overlap" type="int" value="1"/>
          <Parameter name="Graph Fill" type="int" value="0"/>
          <Parameter name="Drop Tolerance" type="double" value="0.0"/>
          <Parameter name="Fill Factor" type="double" value="10.0"/>
          <Parameter name="Steps" type="int" value="3"/>
          <Parameter name="Polynomial Order" type="int" value="3"/>
          <Parameter name="RCM Reordering" type="string" value="Enabled"/>
          <Parameter name="Orthogonalization" type="string" value="Classical"/>
          <Parameter name="Size of Krylov Subspace" type="int" value="1000"/>
          <Parameter name="Convergence Test" type="string" value="r0"/>
          <Parameter name="Ill-Conditioning Threshold" type="double" value="1e+11"/>
          <Parameter name="Output Frequency" type="int" value="1"/>
        </ParameterList>
        <Parameter name="Max Iterations" type="int" value="1"/>
        <Parameter name="Tolerance" type="double" value="1e-9"/>
    </ParameterList>
    <Parameter name="Output Every RHS" type="bool" value="true"/>
    <ParameterList name="VerboseObject">
        <Parameter name="Output File" type="string" value="none"/>
        <Parameter name="Verbosity Level" type="string" value="default"/>
    </ParameterList>
  </ParameterList>



generates a runtime error:

        *******************************************************
        ***** Problem: Epetra::CrsMatrix
        ***** Preconditioned BICGSTAB solution
        ***** Thyra::DefaultInverseLinearOp<double>{lows=Thyra::AztecOOLinearOpWithSolve{fwdOp=Thyra::EpetraLinearOp{op='Epetra_CrsMatrix',rangeDim=41443,domainDim=41443}},fwdSolveCriteria=DEFAULT,adjSolveCriteria=DEFAULT}
        ***** No scaling
        *******************************************************

                iter:    0           residual = 1.000000e+00

         *******************************************************
         ***** Problem: Epetra::CrsMatrix
         ***** Preconditioned BICGSTAB solution
         ***** ILUT( fill-in = 1.000e+01, drop = 0.000e+00)
          ***** with overlap = 1
         ***** No scaling
         ***** NOTE: convergence VARIES when the total number of
         *****       processors is changed.
         *******************************************************

 Error:    Did not find previous factorization (requested
     by setting options[AZ_pre_calc] to AZ_reuse).
     To find this factorization, the following
     parameters must match the previous factorization:
          1) Total number of unknowns.
          2) options[AZ_overlap]
          3) options[AZ_scaling]
          4) options[AZ_precond]
          5) options[AZ_reorder]
          6) options[AZ_type_overlap]
          7) options[AZ_subdomain_solve]
          8) options[AZ_graph_fill]
          9) params[AZ_ilut_fill]
         10) params[AZ_drop]
         11) data_org[AZ_name]

Do you know why ?

Bui




On 09/20/2013 12:19 AM, Hoang Giang Bui wrote:

Hi Eric
I manage to know all the AztecOO settings by reading Stratimikos. However, it still didn't work as I expected. Even with the setting "Aztec Solver" as "LU" doesn't work as well. That's kind of strange. I want to ask if I did program correctly or not. In my case for the first step there are no pressure, so matrix has just one block. To build preconditioner using Teko, I invoke:

Teuchos::RCP<const Teko::InverseFactory> inverseA = mInverseLibrary->getInverseFactory("AztecOO");

Teko::LinearOp opA = Thyra::epetraLinearOp(blkExtractor.GetBlock(0, 0));

Teko::LinearOp inv_rA = Teko::buildInverse(*inverseA, opA); //build the preconditioner using Teko InverseFactory

Teuchos::RCP<const Epetra_Operator> opPrec = Teuchos::rcp_dynamic_cast<const Epetra_Operator>(inv_rA);

Teuchos::RCP<Epetra_Operator> mPrec = Teuchos::rcp_const_cast<Epetra_Operator>(opPrec);
.............

aztec_solver.SetPrecOperator(*mPrec);
Do you think this is correct?
Bui




On Thu, Sep 19, 2013 at 8:14 PM, Cyr, Eric C <eccyr at sandia.gov<mailto:eccyr at sandia.gov>> wrote:
Yeah, thats what you want. Instead of using the string "Ifpack" when you
call the sub solve, use "AztecOO". Teko is "smart" enough to know that a
solver can be used as a preconditioner. (You can also use Amesos, if you
want a direct solver) If you'd like to know how to specify a
preconditioner for Azteco when used as a sub solve let me know.

Eric

On 9/19/13 12:04 PM, "Hoang Giang Bui" <hgbk2008 at gmail.com<mailto:hgbk2008 at gmail.com>> wrote:

>Hi Eric
>
>Yes, that what I currently use. For example in attached file. (The
>AztecOO section is irrelevant)
>
>Bui
>
>On 09/19/2013 07:45 PM, Cyr, Eric C wrote:
>> Are you using the Teko parameter list, and defining an InverseFactory
>> through it?
>>
>> Eric
>>
>> On 9/19/13 11:08 AM, "Hoang Giang Bui" <hgbk2008 at gmail.com<mailto:hgbk2008 at gmail.com>> wrote:
>>
>>> Dear Eric
>>>
>>> I think Stratimikos support wrapping AztecOO as solver but not
>>> preconditioner. Do you have a working script that allows AztecOO as
>>> preconditioner ?
>>>
>>> Bui
>>>
>>>
>>> On 09/19/2013 03:42 PM, Cyr, Eric C wrote:
>>>> It is already an inverse factory in Teko. AztecOO is wrapped by
>>>> Stratimikos. Anything in Stratimikos is accessible to Teko (This is
>>>>how
>>>> you access Ifpack for instance). If you go to the Stratimikos page and
>>>> look up the AztecOO parameters you should be able to modify those to
>>>>be
>>>> accessible by Teko. Let me know if you need more help.
>>>>
>>>> Regarding the ILUT in Ifpack. I have to say that I don't use it and
>>>> think
>>>> there may be problems with its implementation. The ILU in Ifpack is
>>>> solid
>>>> and we get a lot of use out it.
>>>>
>>>> Eric
>>>>
>>>> On 9/19/13 4:43 AM, "Hoang Giang Bui" <hgbk2008 at gmail.com<mailto:hgbk2008 at gmail.com>> wrote:
>>>>
>>>>> That's fine. I can use AztecOO with ilut since it works well in my
>>>>> case.
>>>>> In fact, I used Ifpack as sub-preconditioner for my block
>>>>> preconditioner. This block preconditioner doesn't work since
>>>>> Ifpack_ILUT
>>>>> failed. It is not sure that AztecOO ILUT will be success if it's
>>>>> applied
>>>>> as sub-preconditioners though. However I wonder if there is a way to
>>>>> introduce AztecOO as an InverseFactory in Teko?
>>>>>
>>>>> Best regards
>>>>> Giang Bui
>>>>>
>>>>>
>>>>>
>>>>> On 09/18/2013 07:16 PM, Erik Boman wrote:
>>>>>> Unfortunately, the incomplete factorization parameters are not well
>>>>>> documented so looking at the source code is best. Yes, Ifpack_ILUT
>>>>>> interprets "ilut level-of-fill" to be the ratio of nonzeros in the
>>>>>> preconditioner relative to the original matrix, so a value of 1.0
>>>>>>or a
>>>>>> bit higher is reasonable. (Note: 1.0 in Ifpack roughly corresponds
>>>>>>to
>>>>>> 0 in AztecOO.) If you set it to 50, you will in most cases compute a
>>>>>> complete factorization (but in a very slow way!). As a sanity check,
>>>>>> you can try using Amesos (by default, KLU) as your subdomain solver.
>>>>>>
>>>>>> There are several other parameters in Ifpack that may be helpful.
>>>>>>For
>>>>>> example, the absolute and relative threshold parameters may make
>>>>>>your
>>>>>> preconditioner more robust. Confusingly, these prescribe a diagonal
>>>>>> perturbation (aka Manteuffel shift).
>>>>>>
>>>>>> Erik
>>>>>>
>>>>>> Hoang Giang Bui wrote:
>>>>>>> Hi Chris
>>>>>>>
>>>>>>> I didn't see the range of value of ilut level of fill in Ifpack
>>>>>>>user
>>>>>>> guide yet. But looking in the code of Ifpack_ILUT, I'm quite sure
>>>>>>> that the level of fill indicates the number of nonzeros in the
>>>>>>> preconditioner compare to initial nonzeros. Therefore its value can
>>>>>>> be bigger than 1.
>>>>>>>
>>>>>>> Giang Bui
>>>>>>>
>>>>>>>
>>>>>>> On 09/18/2013 04:43 AM, Chris Jackson wrote:
>>>>>>>> I don¹t know if this applies to the one you are talking about,
>>>>>>>>but I
>>>>>>>> recall that with one of the ILUT preconditioners the fill value is
>>>>>>>> expected to be between 0.0 and 1.0.  1.0 meant essentially we were
>>>>>>>> finding the inverse of the matrix.  Before we figured that out we
>>>>>>>> were always astounded that BiCGStab was converging in 2
>>>>>>>>iterationsŠ
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Chris Jackson
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> *From:* trilinos-users-bounces at software.sandia.gov<mailto:trilinos-users-bounces at software.sandia.gov>
>>>>>>>> [mailto:trilinos-users-bounces at software.sandia.gov<mailto:trilinos-users-bounces at software.sandia.gov>] *On Behalf Of
>>>>>>>> *Hoang Giang Bui
>>>>>>>> *Sent:* Tuesday, September 17, 2013 6:28 PM
>>>>>>>> *To:* Erik Boman
>>>>>>>> *Cc:* trilinos-users at software.sandia.gov<mailto:trilinos-users at software.sandia.gov>
>>>>>>>> *Subject:* Re: [Trilinos-Users] ilut setting
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Hi Erik
>>>>>>>>
>>>>>>>> I tried to increase "fact: ilut level-of-fill" to 50.0 and it
>>>>>>>>still
>>>>>>>> doesn't converge. In fact, with this value the preconditioner take
>>>>>>>> very long to compute. Is there other option we can set for Ifpack
>>>>>>>> ILUT?
>>>>>>>>
>>>>>>>> Giang Bui
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Sep 17, 2013 at 8:01 PM, Erik Boman <egboman at sandia.gov<mailto:egboman at sandia.gov>
>>>>>>>> <mailto:egboman at sandia.gov<mailto:egboman at sandia.gov>>> wrote:
>>>>>>>>
>>>>>>>> Giang Bui,
>>>>>>>>
>>>>>>>> You will not get exactly the same answer because the ILUT
>>>>>>>> implementations are different in AztecOO and Ifpack. Also, the
>>>>>>>> meaning of "fill" may be slightly different. Try increase the
>>>>>>>>value
>>>>>>>> of  "fact: ilut level-of-fill" in the Ifpack parameter to see if
>>>>>>>>you
>>>>>>>> can make it converge.
>>>>>>>>
>>>>>>>> Erik
>>>>>>>>
>>>>>>>> Hoang Giang Bui wrote:
>>>>>>>>
>>>>>>>> Dear Trilinos developers,
>>>>>>>>
>>>>>>>> Is there difference between 2 settings:
>>>>>>>>
>>>>>>>> solver_parameters.set("AZ_solver", "AZ_bicgstab")
>>>>>>>> solver_parameters.set("AZ_kspace", 1000)
>>>>>>>> solver_parameters.set("AZ_output", 100)
>>>>>>>> solver_parameters.set("AZ_precond", "AZ_dom_decomp")
>>>>>>>> solver_parameters.set("AZ_subdomain_solve", "AZ_ilut")
>>>>>>>> solver_parameters.set("AZ_drop", 0.0)
>>>>>>>> solver_parameters.set("AZ_ilut_fill", 10.0)
>>>>>>>> aztec_solver.SetParameters(solver_parameter, true);
>>>>>>>>
>>>>>>>> And:
>>>>>>>> solver_parameters.set("AZ_solver", "AZ_bicgstab")
>>>>>>>> solver_parameters.set("AZ_kspace", 1000)
>>>>>>>> solver_parameters.set("AZ_output", 100)
>>>>>>>> solver_parameters.set("AZ_precond", "AZ_none")
>>>>>>>> aztec_solver.SetParameters(solver_parameter, true);
>>>>>>>> IFPreconditionerType = "ILUT"
>>>>>>>> preconditioner_parameters.set("fact: ilut level-of-fill", 10.0);
>>>>>>>> preconditioner_parameters.set("fact: drop tolerance", 0.0);
>>>>>>>> overlap_level = 1
>>>>>>>> mPrec = Teuchos::rcp(Factory.Create(IFPreconditionerType, &rA,
>>>>>>>> overlap_level));
>>>>>>>> mPrec->SetParameters(preconditioner_parameters);
>>>>>>>> aztec_solver.SetPrecOperator(&*(mpPrec->GetOperator()))
>>>>>>>>
>>>>>>>> I thought it all resulted in Additive-Schwartz preconditioner with
>>>>>>>> ILUT as local subdomain preconditioner. But it gives different
>>>>>>>> performance. The first setting works well but the second failed to
>>>>>>>> converge.
>>>>>>>>
>>>>>>>> Best regards
>>>>>>>> Giang Bui
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Trilinos-Users mailing list
>>>>>>>> Trilinos-Users at software.sandia.gov<mailto:Trilinos-Users at software.sandia.gov>
>>>>>>>> <mailto:Trilinos-Users at software.sandia.gov<mailto:Trilinos-Users at software.sandia.gov>>
>>>>>>>> http://software.sandia.gov/mailman/listinfo/trilinos-users
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> With Best Regards !
>>>>>>>> Giang Bui
>>>>>>>>
>>>>>>>> To learn and to excel
>>>>>>>>
>>>>> _______________________________________________
>>>>> Trilinos-Users mailing list
>>>>> Trilinos-Users at software.sandia.gov<mailto:Trilinos-Users at software.sandia.gov>
>>>>> http://software.sandia.gov/mailman/listinfo/trilinos-users
>



--
With Best Regards !
Giang Bui
To learn and to excel


-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://software.sandia.gov/pipermail/trilinos-users/attachments/20130923/6a80c753/attachment-0001.html 


More information about the Trilinos-Users mailing list