COMP 633 High-Performance Computing (Fall 2007)

$ Revised: Sun Dec 9 2007 by prins@cs.unc.edu

COMP 633
High-Performance Computing
http://www.cs.unc.edu/~prins/Classes/633/

Fall 2007 (Aug 21 - Dec 5)
TTh 9:30 - 10:45 SN 011

Instructor: Jan Prins, SN 355, Tel: 962-1913, prins@cs.unc.edu
Office hours: by appt.

TA: Catalin Constantin SN 024, Tel: 843-7417, catalinc@cs.unc.edu
Office hours: MW 10 - 2

Secretary: Madelyn Mann
SN 353, Tel: 962-1922, mann@cs.unc.edu

Overview

This is an introductory graduate course covering several aspects of high-performance computing, primarily focused on parallel computing. Upon completion, you should

be able to design and analyze parallel algorithms for a variety of problems and computational models,
be familiar with the fundamentals of the architecture and systems software of high-performance parallel computing systems, and
have experience with the implementation of parallel applications on actual high-performance computing systems, and be able to measure, tune, and report on their performance.

Additional course information including a syllabus for the course can be found in the course overview.

Announcements

Copies of all the lecture notes are available online.
The final exam for this class can be individually completed during any 3 hour slot within the following time intervals:
- Tuesday December 11, 9:00 AM - 5:00 PM.
- Wednesday December 12, 8:30 AM - 11:30 AM.
Pickup your exam from the front desk and return it three hours later. You may use all materials from class in completing your exam. You may not use computers other than to read your notes, or use the calculator. You may not discuss the exam with anyone else before noon December 12th. As always, you will be asked to pledge that you have followed these rules. In case you run into difficulties, contact me (962-1913) or Madelyn Mann (962-1922). (I may not be able to take your call Tuesday morning.)
The midterm exam will be held in class on Tue Oct 23 and will cover material from the first 13 lectures. You may bring any course notes, readings, and assignments with you for use during the exam. However, you can't use a computer during the exam.
Please request an account on topsail for use in programming assignments. Please adapt this template with your details to make the body of a mail message sent to research@unc.edu. Please do this now rather than when the first assignment comes due, since the process may take some time. Topsail information page.

Reading Assignments

(for Tue Nov 20) Read Kumar et al. Ch 3 (Basic Communication Operations)
(for Thu Nov 15) Read Kumar et al. Secns 2.3 - 2.7 (Interconnection Networks).
(for Tue Nov 13) Read Foster, Chapter 8, Overview of MPI.
(for Thu Nov 1) Read Skillicorn et al., Questions and Answers about BSP.
(for Tue Oct 16) Read Nyland et al. "Chapter 1 Fast N-Body Simulation with CUDA"
(for Thu Oct 4) Read Patterson & Hennesey Ch. 8, sections 8.3 - 8.6 (shared memory, synchronization primitives in shared memory, and memory consistency models).
(for Tue Oct 2) Read Andrews Ch. 5 section 5.1 (monitors) and section 5.4 (Case Study: Java).
(For Thu Sep 27) Read Andrews Chapter 3 (Locks and Barriers) in Multithreaded, Parallel and Distributed Programming.
(For Thu Sep 13) Read "Getting started with OpenMP" (Chapter 2) in Parallel Programming in OpenMP.
(For Tue Sep 11) Read overview of Memory Hierarchy in Cache-based Systems.
(For Thu Sep 6) Complete PRAM Handout.
(For Thu Aug 30) PRAM Handout, sections 3.4, 3.6.
(For Tue Aug 28) PRAM handout, sections 3.2, 3.3, 3.5.
(For Thu Aug 23) Read PRAM Handout secns 1, 2, 3.1.

Written and Programming Assignments

(Mon Nov 14) Written assignment #3 - due Tue Dec 4 at start of class
(Wed Nov 2)Programming Assignment #2 -due Tue Dec 4 at start of class.
(Thu Oct 25) Programming Project - Topic discussed and approved by Fri Nov 2 and final project submission due Wed Dec 5.
(Thu Oct 4) Written Assignment #2 - due Tue Oct 16.
(Thu Sep 13) Programming Assignment #1 - Updated Assignment . Extended due date Tue Oct 2.
(Thu Aug 23) Written Assignment #1 - due Tue Sep 11.

On-line Handouts

(some material local-access only)

Software

OpenMP

OpenMP reference: Specification of OpenMP binding for C/C++.
Topsail-specific material
- Getting started on Topsail
- A directory with the sample diffusion program discussed in class.
- Command lines for the compilation and execution of programs on topsail.unc.edu
  1. C compilation to create sequential program (compiler ignores OpenMP directives and does not link with OpenMP runtime library):
    icc -o prog prog.c
  2. C compilation to create parallel program (OpenMP directives honored and program linked with OpenMP runtime library)
    icc -openmp -o prog prog.c
  3. Batch-scheduled execution of the parallel program with 8 processors:
    bsub -q 32cpu -x -n 8 -o result%J env OMP_NUM_TREADS=8 ./prog

Cilk

This Cilk reference manual refers to a slightly older revision of the Cilk system, but is accurate with respect to the language definition.
Currently there is no platform in the department running Cilk, but implementations exist for shared memory multiprocessor linux platforms, and can be installed.

Java

Java threads reference material
- Java 6 Documentation and SDK
- Class Thread API description.
- Java threads and locks specification (not easy going, unfortunately).
- Some background on recent efforts to update the Java memory model to yield a better specification for Java concurrency.
Java threads execute in parallel on the following CS machines

Linux/ia32
java 1.6.0 (Java 2 runtime) in /usr/java/bin/ on seca (4 proc), and swan (2 proc * 2-way hyperthreading).

MPI

MPI reference material
- Comprehensive site organizing all reference and tutorial information on MPI.
- A short overview of MPI.
Sample command lines for the compilation and execution of C/MPI programs on linux Cluster Topsail (C++ and Fortran programs can also use MPI).
- To specify compilation and runtime environment:
  topsail% module load hpc/mvapich-intel
- To compile and link with the MPI library (be sure to #include mpi.h in your programs):
  topsail% mpicc -o prog prog.c
- To submit job to run using 64 processors:
  topsail% bsub -q 128cpu -x -n 64 -o result%J -a mvapich mpirun ./prog

Bibliography

This list will evolve throughout the semester. Specific reading assignments are listed above.

PRAM Algorithms, S. Chatterjee, J. Prins, course notes, 2007.
Memory Hierarchy in Cache-Based Systems, R. v.d. Pas, Sun Microsystems, 2003.
Parallel Programming in OpenMP, R. Chandra et al., Morgan-Kaufmann, 2001.
Multithreaded, Parallel and Distributed Programming, G. Andrews, Addison-Wesley, 2000.
Computer Architecture: A Quantitative Approach 2nd ed, D. Patterson, J. Hennessy, Morgan-Kaufmann 1996.
"Fast N-Body Simulation with CUDA", L. Nyland, M. Harris, J. Prins, in GPU Gems 3, H Nguyen, ed., Prentice-Hall 2007.
"Questions and Answers about BSP", D. Skillicorn, J. Hill, and W. McColl, Scientific Programming 6, 1997.
Designing and Building Parallel Programs, I. Foster, Addison-Wesley, 1995. Online text.
Introduction to Parallel Computing: Design and Analysis of Algorithms, V. Kumar, A. Grama, A. Gupta, G. Karypis, Benjamin-Cummings, 1994.

This page is maintained by prins@cs.unc.edu. Send mail if you find problems.