Lecture 2

Learning objectives

After this class, you should be able to:

1. Define the time taken by a parallel program.
2. Define the communication cost model that we will be using in this course.
3. Derive the time complexity of different reduction implementations using our communication cost model.
4. Given a simple parallel algorithm, derive its time complexity.
5. Explain the difference between data parallelism and task parallelism.
6. Explain the differences between shared memory and distributed memory programming.
7. Explain the differences between concurrent, distributed, and parallel programming.

Reading assignment

1. Lec-2 slides.
2. Chapter 1.
3. Review the basic idea behind caches from your computer organization course.

Exercises and review questions

• Questions on current lecture's material
  1. (Post on the discussion board) We discussed an analogy between distributed memory programming and people communicating with each other when they work in a group. Give an example of a human activity that is analogous to shared memory programming.

• Questions on next lecture's material
  1. What are the sizes of the different types of caches on linprog?