Synchronization usually involves waiting by at least one task, and can therefore cause a parallel application's wall clock execution time to increase. The coordination of parallel tasks in real time, very often associated with communications. There are several ways this can be accomplished, such as through a shared memory bus or over a network. Parallel tasks typically need to exchange data. As a programming model, tasks can only logically "see" local machine memory and must use communications to access memory on other machines where other tasks are executing. In hardware, refers to network based memory access for physical memory that is not common. Shared memory hardware architecture where multiple processors share a single address space and have equal access to all resources - memory, disk, etc. In a programming sense, it describes a model where parallel tasks all have the same "picture" of memory and can directly address and access the same logical memory locations regardless of where the physical memory actually exists. Shared Memoryĭescribes a computer architecture where all processors have direct access to common physical memory. Pipeliningīreaking a task into steps performed by different processor units, with inputs streaming through, much like an assembly line a type of parallel computing.
A parallel program consists of multiple tasks running on multiple processors. A task is typically a program or program-like set of instructions that is executed by a processor.
Parallel computers still follow this basic design, just multiplied in units.
Traditionally, software has been written for serial computation:
Overview What Is Parallel Computing? Serial Computing References are included for further self-study.
The tutorial concludes with several examples of how to parallelize several simple problems. These topics are followed by a series of practical discussions on a number of the complex issues related to designing and running parallel programs. The topics of parallel memory architectures and programming models are then explored. The tutorial begins with a discussion on parallel computing - what it is and how it's used, followed by a discussion on concepts and terminology associated with parallel computing. It is not intended to cover Parallel Programming in depth, as this would require significantly more time. As such, it covers just the very basics of parallel computing, and is intended for someone who is just becoming acquainted with the subject and who is planning to attend one or more of the other tutorials in this workshop. It is intended to provide only a brief overview of the extensive and broad topic of Parallel Computing, as a lead-in for the tutorials that follow it. This is the first tutorial in the "Livermore Computing Getting Started" workshop.