parallelism

COMPUTER ORGANIZATION AND ARCHITECTURE DESIGNING FOR PERFORMANCE NINTH EDITION

Traditionally, the computer has been viewed as a sequential machine. Most computer
programming languages require the programmer to specify algorithms as sequences
of instructions. Processors execute programs by executing machine instructions in
a sequence and one at a time. Each instruction is executed in a sequence of opera-
tions (fetch instruction, fetch operands, perform operation, store results).
This view of the computer has never been entirely true. At the micro-operation
level, multiple control signals are generated at the same time. Instruction pipelining,
at least to the extent of overlapping fetch and execute operations, has been around
for a long time. Both of these are examples of performing functions in parallel. This
approach is taken further with superscalar organization, which exploits instruction-
level parallelism. With a superscalar machine, there are multiple execution units
within a single processor, and these may execute multiple instructions from the
same program in parallel.
As computer technology has evolved, and as the cost of computer hardware
has dropped, computer designers have sought more and more opportunities for par-
allelism, usually to enhance performance and, in some cases, to increase availability.
After an overview, this chapter looks at some of the most prominent approaches
to parallel organization. First, we examine symmetric multiprocessors (SMPs), one
of the earliest and still the most common example of parallel organization. In an
SMP organization, multiple processors share a common memory. This organization
raises the issue of cache coherence, to which a separate section is devoted. Next,
the chapter examines multithreaded processors and chip multiprocessors. Then
we describe clusters, which consist of multiple independent computers organized
in a cooperative fashion. Clusters have become increasingly common to support
workloads that are beyond the capacity of a single SMP. Another approach to the
use of multiple processors that we examine is that of nonuniform memory access
(NUMA) machines. The NUMA approach is relatively new and not yet proven in
the marketplace, but is often considered as an alternative to the SMP or cluster
approach. Finally, this chapter looks at hardware organizational approaches to vec-
tor computation. These approaches optimize the ALU for processing vectors or
arrays of floating-point numbers. They are common on the class of systems known
as supercomputers.