FSU Seal - 1851

    COURSE SYLLABUS

    CIS 5930-04 Parallel Computing
    Spring 2007

Prerequisites:

You should be comfortable programming in C, and have good knowledge of undergraduate level algorithms, data structures, and computer architecture. No knowledge of parallel computing is required.

Class Schedule:

Activity Day Time Location
Lecture TR 12:30 pm - 1:45 pm LOV 103

Contact Information:

Instructor: Ashok Srinivasan
Office hours: T 3:30 pm - 4:30 pm, F 2:30 pm - 3:30 pm. I am also usually available in my office, and you can feel free to meet me in the afternoons, except before class. Alternatively, you may schedule an appointment, either by email or by phone.
Office: 169, Love Building
Phone: 644-0559
Email: asriniva AT cs.fsu.edu

Course Material:

Required Text Other Material
Computer Accounts

Course Rationale:

This is an introductory course on parallel computing, meant for graduate students in Computer Science. This course will teach practical aspects of parallel computing, so that you will be able to effectively use parallel machines. It will be particularly useful for for those who plan to perform research on parallel computing. It should also be useful for those who want to learn programming multicore processors.

Course Description:

In a parallel computation, multiple processors work together to solve a given problem. These are exciting times in parallel computing. The largest parallel machine has over a hundred thousand processors, and it is believed that machines with over ten thousand processors will be commonly available by the end of the decade. Furthermore, with most chip manufacturers moving toward multicore processors, most machines will soon be parallel ones. It is, therefore, essential to learn to use parallel machines effectively.

While parallel machines provide enormous raw computational power, it is often not easy to make effective use of all this power. The problems encountered in making effective use of a large number of machines are similar to those encountered in making a group of people work together. (i) People may spend much of their time talking to each other, instead of doing useful work. Communication between processors is quite expensive, compared with the CPU speed. So we need to pay attention to minimizing the amount of communication; otherwise much of the time will be spent on inter-processor communication, rather than on useful work. (ii) In a group, a few people may do much of the work, while the others relax. Similarly, in a computation, the work load on different processors may differ. In order to make effective use of the parallel machine, we want to keep the work load balanced on all processors. (iii) It may be difficult to decompose a problem so that people can work on different parts simultaneously. For example, consider someone who want to have dinner cooked, eat it, and then have the dishes washed. It is not easy to speed up this process by hiring someone to cook, and another person to wash the dishes, because the three tasks are sequential; the food needs to be cooked before it is eaten, and the food needs to be eaten before the dishes are washed. Similar problems occur in parallel computations too, and sequential parts of the computation can reduce the effectiveness of parallelization substantially. This course will describe different techniques used to solve the above problems, in order to develop efficient parallel algorithms for a variety of problems. We will also pay much attention to practical aspects of implementing parallel code that actually yields good performance on real parallel machines.

Learning Objectives:

At the end of this course, you should be able to accomplish the objectives given below.

Your Responsibilities:

Deadlines and Instructions

Following the same professional guidelines that you will encounter at work, there are strict deadlines, and instructions that must be followed. Please read instructions carefully, and schedule your activities so that you submit assignments well in time. You should check your garnet email account and the class web page regularly, and note other announcements, on-line and in class.

Class Participation

I will ask you questions in class: (i) review questions on the previous lecture, and (ii) questions on the material currently being discussed, in order for me to obtain feedback on how well you understand the material. You should be prepared to answer these questions, and should also participate by asking questions, suggesting ideas, and performing in-class assignments that I give. Of course, you cannot participate in class unless you attend it!

Group Project

You will have one group project. You should participate in your group's activities and make a fair contribution to the efforts of your group. You can learn more about working in groups at: www.cs.fsu.edu/~asriniva/courses/parco07/groupwork.html. Roughly half of your grade on the group project will be based on the performance of the whole group. The other half will be based on your contribution. However, if your contribution was unacceptably small, then you may be assigned a grade as low as zero. Note that the group project constitutes an important component of your grade. Your work should be of sufficient quality and quantity for it to be accepted in at least a mediocre conference, such as Europar or ICCS. You may want to read some best papers from conferences such as IPDPS or SC, to get an idea of good quality work. While your work may differ from the best ones in quantity, the quality of your work and presentation should be high.

Reading Assignments

After each lecture, you will be given a reading assignment pertaining to that lecture. You should read these, and also practice writing code. New material builds on the old ones. So, if you have trouble with some material, please get help through the discussion board on Blackboard, or from me, before the next class. You should also peruse the material for the next lecture, and be prepared to answer questions on it, which I will provide in advance. I expect that you will need to spend between one and two hours studying, for each lecture. The programming assignments, project, and exams will consume additional time. The following learning components are important, and you may want to verify if you do satisfactorily on these, after studying the material.

Assignments

You will have two programming assignments in this course, and you will have around ten days to work on each one. The assignments will be announced on the Blackboard course web site under "Assignments". Parallel programming assignments are substantially more difficult than sequential programming assignments, and require substantially more time and effort. Please start working on the assignments as soon as they are announced, if you wish to complete them!

Course Calendar:

Week Lecture Chapter Assignments
1 9 Jan Introduction -- Chapter 1
11 Jan Parallel architectures -- Chapter 2
2 16 Jan Parallel algorithm design -- Chapter 3 (to be continued)
18 Jan Parallel algorithm design -- Chapter 3 (completed)
3 23 Jan Shared memory programming -- Chapter 17 (to be continued)
25 Jan Shared memory programming -- Chapter 17 (to be continued)
4 30 Jan Shared memory programming -- Chapter 17 (completed)
1 Feb Message passing programming -- Chapter 4
5 6 Feb Combining MPI and OpenMP -- Chapter 18. Programming assignment 1 announced 6 Feb.
8 Feb Debugging MPI programs -- Appendix C
6 13 Feb Floyd's algorithm -- Chapter 6 Project groups and topics due 16 Feb.
15 Feb Performance analysis -- Chapter 7 (to be continued)
7 20 Feb Performance analysis -- Chapter 7 (completed) Assignment 1 due 20 Feb.
22 Feb Matrix-vector multiplication -- Chapter 8 (to be continued)
8 27 Feb Midterm 1 Midterm this week!
1 Mar Matrix-vector multiplication -- Chapter 8 (completed)
9 6 Mar Spring break -- no class.
8 Mar Spring break -- no class.
10 13 Mar Matrix multiplication -- Chapter 11 Project progress reports due 16 Mar.
15 Mar Finite difference methods -- Chapter 13
11 20 Mar Sorting -- Chapter 14 Programming assignment 2 announced 23 Mar.
22 Mar Combinatorial search -- Chapter 16 (to be continued)
12 27 Mar External memory sorting
29 Mar Heterogeneous architectures
13 3 Apr Combinatorial search -- Chapter 16 (to be continued)
5 Apr Combinatorial search -- Chapter 16 (completed)
14 10 Apr Load balancing Midterm this week!
12 Apr Midterm 2
15 17 Apr Project presentations 1 Assignment 2 due 16 Apr.
Projects due 20 Apr.
19 Apr Project presentations 2

Grading Criteria:

Your overall grade will be based on your performance in (i) midterms, (ii) group project, (iii) class participation, and (iv) assignments, with weights as given in Table 1. Exams consist of two midterms. Class participation consists of answering questions correctly in class, and other positive contributions, discussed in greater detail at www.cs.fsu.edu/~asriniva/courses/parco07/classparticipation.html. Assignments consist of two programming projects.

Your average on the midterms should be at least 80% for you to get a course grade of B or better. If you meet this constraint, then the final grade will be determined using Table 2.

    Table 1: Course Points
    Item Weight
    Class Participation 10
    Midterms 40
    Assignments 30
    Project 20
    Table 2: Letter Grades
    Points Grade
    92.0 - 100.0 A
    90.0 - 91.9 A-
    88.0 - 89.9 B+
    82.0 - 87.9 B
    80.0 - 81.9 B-
    0 - 79.9 F - C+

NOTE: You must earn a weighted average of 80% in the midterm exams to be awarded a course grade of B or better. For example, if you obtain a total of 89%, but an exam average of only 78%, then you will not get a B+. Instead, you will get a B-, because that is the highest grade for which you will be eligible without meeting the exam cutoff.

Programming Assignment Assessment

You must understand your assignment work. If you are asked to explain your work, and you are unable to do so, you may be assigned a grade of zero.

Course Policies:

Attendance Policy

The university requires attendance in all classes, and it is also important to your learning. The attendance record may be provided to deans who request it. If your grade is just a little below the cutoff for a higher grade, your attendance will be one of the factors that we consider in deciding whether to "bump" you up to the higher grade. Missing three or fewer lectures will be considered good attendance. In rare cases, such as medical needs or jury duty, absences may be excused with appropriate documentation. You should let me know in advance, when possible, and submit the documentation I seek. You should make up for any materials missed due to absences.

Missed Exam Policy

A missed exam will be recorded as a grade of zero. We will follow the university rules regarding missed final exams (see http://registrar.fsu.edu/dir_class/fall/exam_schedule.htm), for all the exams.

Late Assignment Policy

In order to enable us to provide timely solutions to assignments, we have the following policy regarding submission of late assignments.

Grade of 'I' Policy

The grade of 'I' will be assigned only under the following exceptional circumstances:

Professional Ethics

You will gain confidence in your ability to design and implement algorithms only when you write the code yourself. On the other hand, one does learn a lot through discussions with ones peers. In order to balance these two goals, I give below a list of things that you may, and may not, do.

Things you may not do: You should not copy code from others. This includes directly copying the files, replacing variable names in their code with different names, altering indentation, or making other modifications to others' code, and submitting it as your own. (You may also wish to note that many of the modifications that make codes look very different in a higher level language, yield lower level representations that are very close, and are easy to detect.) Furthermore, you should take steps to ensure that others cannot copy code from you -- in particular, you should have all permissions on assignment files and directories set off for others.

Things you may do: You may discuss specific problems related to use of the computer, useful utilities, and some good programming practices, with others. For example, you may ask others about how to submit your homework, or how to use the debugger or text editor. Honor Code: Students are expected to uphold the academic honor code published in "The Florida State University Bulletin" and the "Student Handbook". Please read the provisions of the Academic Honor Code: http://dof.fsu.edu/honorpolicy.htm.

Plagiarism:

Plagiarism is "representing another's work or any part thereof, be it published or unpublished, as ones own. For example, plagiarism includes failure to use quotation marks or other conventional markings around material quoted from any source" (Florida State University General Bulletin 1998-1999, p. 69). Failure to document material properly, that is, to indicate that the material came from another source, is also considered a form of plagiarism. Copying someone else's program, and turning it in as if it were your own work, is also considered plagiarism.

SYLLABUS CHANGE POLICY:

This syllabus is a guide for the course and is subject to change with advanced notice.

Last modified: 30 Apr 2007