2009 · 22 citations · 6 references
ComputingComputer EngineeringComputer ScienceComputer ArchitectureParallel ComputingAlgorithm AnalysisData ParallelismNumerical AnalysisNumerical Linear AlgebraParallel ComputationParallel Computing SystemsComplexity Of Numerical AlgorithmsFloating-point ArithmeticMagnitudes Of NumbersMultiply-accumulate Units
Floating-point operations, as defined in the IEEE-754 standard, are not associative. The ordering of large numbers of operations (such as summa-tions) that deal with operands of substantially dif-ferent magnitudes can significantly affect the final result. On massively multi-threaded systems, the non-deterministic nature of how machine floating-point operations are interleaved, combined with the fact that intermediate values have to be rounded or truncated to fit in the available precision leads to non-deterministic numerical error propagation. We have investigated on a Cray XMT system the effect of non-deterministic error propagation by observing the convergence rate of a conjugate gradient calcula-tion used as part of a Power State Estimation (PSE) application. As a possible mitigation strategy, we have explored quadruple precision accumulation, as well as a deterministic parallel tree scheme. The tree based approach has consistently outperformed the quadruple precision approach due to an improved convergence rate. As a consequence, we motivate the need for compile time mechanisms that enable enforcement of parallel deterministic operations on the Cray XMT. ∗This work was funded by the Center for Adaptive Super-
6
A Regular Layout for Parallel Adders
Brent, Kung · IEEE Transactions on Computers · 1982 · 1.1K citations
Portals 3.0: protocol building blocks for low overhead communication
Ron Brightwell, R. Riesen, B. Lawry et al. · 2002 · 82 citations