CMPUT 481/681 - Parallel and Distributed Systems

Overview

This course provides a graduate-level introduction to parallel programming and parallel and distributed systems. Both shared-memory parallel computers and distributed-memory multicomputers (e.g., clusters) will be studied. Aspects of the practice and research issues in parallelism will be covered.

We will cover fundamental and current research topics in the design, implementation, and evaluation of parallel and distributed systems. Our focus will be on the systems software and parallel programming systems, but some hardware issues will also be covered. Topics will include parallel algorithms, parallelization strategies, virtual machines, and operating system support.

Approximately 2/3 of the course will be on the practical and hands-on aspects of parallel programming. The other 1/3 of the course will be on current research issues. A number of research papers will be discussed and critically evaluated.

Objectives

• Understand why we should or should not use parallel and distributed systems
• Understand speedup and Amdahl's Law
• Study various hardware (multiprocessors, networks of workstations, clusters) and software (threads and shared memory, processes and message passing, distributed shared memory and distributed shared data) architectures
• Study parallel algorithms
  • Concurrency and synchronization (review)
  • Data and work partitioning
  • Common parallelization strategies
  • Granularity
  • Load balancing
  • Examples: parallel search, parallel sorting, etc.
• Study shared-memory programming (Threads)
  • Pthreads
  • Locks & semaphores
• Study distributed-memory programming (message passing & MapReduce)
  • MPI
  • MapReduce and Hadoop

Course Work

• Assignments
• Final Exam (Take-home)
• Group project with report