[Trilinos-Users] Floating-point arithmetic is non-associative

[Hoemmen, Mark]

On 6/15/11 10:01 AM, "Conjeepuram Subramanian, Natarajan" <C.S.Natarajan at bp.com> wrote:
> Maybe I don't understand this issue correctly but wouldn't
> guaranteeing consistency mean floating point arithmetic is commutative?

In the previous message string, please replace "commutative" with "associative."  (Non-associativity of floating-point arithmetic is the cause of different results when varying the number of processors.)  Floating-point addition and multiplication are commutative: a + b = b + a, and a*b = b*a.  

Associativity relates to reduction trees: a reduction tree is a complete parenthesization of a sum.  Changing the shape of the reduction tree changes the placement of parentheses in the sum.  Other than completely serializing all reductions, the only way to guarantee bitwise repeatable results is to use so-called "distillation" algorithms.  I have not seen any parallel implementations of those.  They would be quite slow in any case.  A faster way to make bitwise identical results more likely (not guaranteed!) is to compute all sums in extended-precision arithmetic.  

mfh