Learning objectives

After this class, you should be able to:

1. Given data for the following operations: (i) reduction, (ii) broadcast, (iii) gather, and (iv) scatter, show the messages sent at each step and the state of each processor at the end of each step, for the algorithms discussed in class for these operation.
2. Derive the time complexity for each of the above operations.
3. Explain the idea behind divide and conquor and pipelining, which are used to develop efficient parallel algorithms.
4. Define the terms (i) speedup, (ii) efficiency, and (iii) work.
5. Given the time on a single processor, the number of processors, and the time taken by an algorithm on those processors, compute the speedup, efficiency, and work.

Reading assignment

1. Handout on Parallel algorithms: Slides 1 - 20.
2. Handout on Parallel algorithms: Slides 21 - 30.

Exercises and review questions

• Questions on current lecture's material
  1. Given eight processors with data 1, 3, 2, 4, 5, 3, 1, 8 respectively, show the messages sent at each step of the all-gather operation, and the data present in each processor at the end of each step.
  2. Give an efficient algorithm for the scatter operation, and derive its time complexity.
  3. Show a pipelined scheme for performing n > > P broadcasts on P processors, with the time complexity mentioned in the Parallel algorithms handout.
• Questions on next lecture's material
  1. Given the following sequence of data: 1, 3, 2, 4, give all the prefixes. You can use a sequential algorithm to compute them.
  2. Given the following linear recurrence: xi = i xi-1 + (i+1) xi-2, compute (sequentially) x4, if x0 = x1 = 1.
  3. Show any 3 x 3 matrix, any vector with three components, and the result of multiplying the matrix by the vector.