Parallel and Distributed Computation:
Numerical Methods

Contents, Preface, Ordering, Home

This highly acclaimed work, first published by Prentice Hall in 1989, is a comprehensive and theoretically sound treatment of parallel and distributed numerical methods. It focuses on algorithms that are naturally suited for massive parallelization, and it explores the fundamental convergence, rate of convergence, communication, and synchronization issues associated with such algorithms.

This is an extensive book, which aside from its focus on parallel and distributed algorithms, contains a wealth of material on a broad variety of computation and optimization topics. It is an excellent supplement to several of our other books, including Convex Optimization Algorithms (Athena Scientific, 2015), Nonlinear Programming (Athena Scientific, 1999), Dynamic Programming and Optimal Control (Athena Scientific, 2012), Neuro-Dynamic Programming (Athena Scientific, 1996), and Network Optimization (Athena Scientific, 1998).

"This major contribution to the literature belongs on the bookshelf of every scientist with an interest in computational science, directly beside Knuth's three volumes and Numerical Recipes..."
Anna Nagurney, University of Massachusetts, in the International Journal of Supercomputer Applications

"This major work of exceptional scholarship summarizes more than three decades of research into general-purpose algorithms for solving systems of equations and optimization problems."
W. F. Smyth, in Computing Reviews

"This book marks an important landmark in the theory of distributed systems and I highly recommend it to students and practicing engineers in the fields of operations research and computer science, as well as to mathematicians interested in numerical methods."
Lorne G. Mason, in IEEE Communications Magazine

Among its special features, the book:

• Quantifies the performance of parallel algorithms, including the limitations imposed by the communication and synchronization penalties.

• Provides a comprehensive convergence analysis of asynchronous methods and a comparison with their asynchronous counterparts.

• Covers direct and iterative algorithms for linear and nonlinear systems of equations and variational inequalities.

• Describes optimization methods for nonlinear programming, shortest paths, dynamic programming, network flows, and large-scale decomposition.

• Includes extensive research material on optimization methods, asynchronous algorithm convergence, asynchronous communication network protocols, and others.

• Supplements the text material with many exercises, with full solutions that are internet-accessible.

• Contains a lot of material not found in any other book.

The authors are Professors of Electrical Engineering and Computer Science at the Massachusetts Institute of Technology, and members of the US National Academy of Engineering. They have been awarded several prestigious awards, including the INFORMS John von Neumann Theory Prize, for the contributions of the research monographs "Parallel and Distributed Computation" and "Neuro-Dynamic Programming".

Free download:

• Entire book including solutions to nearly all exercises.