Parallel Computing Seminar (Spring 1996)

Announcements

• Lizy Kurian John will speak on "Improving the Memory Access Performance of Programs", Thursday April 4 at 10.30 in SN011.

Handouts

• HO 01: Course Overview

Reading Assignments

Jan 17

Culler et al., "LogP: Towards a Realistic Model of Parallel Computation"
Culler et al., "Fast Parallel Sorting under LogP: from Theory to Practice"

Jan 31

Valiant, "A bridging model for parallel computation"
Bisseling and McColl, "Scientific Computing on Bulk Synchronous Parallel Architectures"

Feb 7

Alpern et al., "Modeling Parallel Computers as Memory Hierarchies",
Alpern et al., "Space-Limited Procedures: A Methodology for Portable High-Performance",

Feb 19

Dai, Gupta, et al., "EXTENT: A Portable Programming Environment for Designing and Implementing High Performance Block Recursive Algorithms"
Z. Li, J. Reif and S. Gupta "Synthesizing Efficient Out-of-Core Programs for Block Recursive Algorithms using Block-Cyclic Data Distributions"

Feb 26

Blelloch et al., "Implementation of a Nested Data-Parallel Language"
Riely et al., "Provably Correct Vectorization of Nested Parallel Programs"

Mar 4

Spring Break

Mar 11

Rau & Fisher, "Instruction Level Parallelism: Overview, History and Perspective".
(Presented by Lei Wang)

Mar 13

No reading assignment. Class will introduce various approaches to object-based concurrency.

Mar 18

A. Chien, "Concurrent Aggregates".
(Presented by Hye-Chung Kum)

Mar 25

Chandy, "Compositional C++".

Apr 1

Chien, Reddy, Plevyak & Dolby, "ICC++ -- A C++ Dialect for High Performance Parallel Computing".
(Presented by Rik Faith)

Apr 3

No reading assignment. Class will be an introduction to Vector Architectures.

Apr 8

Blelloch, "Scan Model of Computation".
(Presented by Tom White)

Apr 10

Gibbons et al., "Efficient Low-Contention Parallel Algorithms",
Blelloch et al., "Accounting for Memory Bank Contention and Delay in High-Bandwidth Multiprocessors",
(Presented by Peter Nee)

Apr 15

Wulf & McKee, "Hitting the Memory Wall: Implications of the Obvious"
(presented by Michael North)

Apr 17

Saulsbury, Pong & Nowatzyk, "Missing the Memory Wall: The Case for Processor/Memory Integration"

Apr 22,24

(fast N-body computations)
(presented by Lars Nyland)

Reading List

Parallel Computing Models and Algorithm Development

1. Scan model
   • G. Blelloch, "Scan Model of Parallel Computation", IEEE, 1989.

2. LogP
   • D. Culler, R. Karp, D. Patterson, A. Sahay, K. Schauser, E. Santos, R. Subramonian, T. von Eicken, "LogP: Towards a Realistic Model of Parallel Computation", PPoPP, ACM, 1993.
   • D. Culler, A. Dusseau, R. Martin, K. Schauser, "Fast Parallel Sorting under LogP: from Theory to Practice", Portability and Performance for Parallel Processing, Hey T. and J. Ferrante ed., John Wiley and Sons, 1994.

3. BSP
   • Leslie G Valiant, "A Bridging Model for Parallel Computation", CACM 33 (8), Aug 1990.
   • R Bisseling and W. McColl, "Scientific Computing on Bulk Synchronous Parallel Architectures", TR 836, Dept. of Mathematics, Rijksuniversiteit Utrecht, NL, 1994.
   • Gerbessiotis and Valiant, "Direct Bulk-Synchronous Parallel Algorithms", JPDC 22, 1994.

4. Memory-hierarchy models
   • B. Alpern, L. Carter and J. Ferrante, "Modeling Parallel Computers as Memory Hierarchies", First Intnl Conference on Massively Parallel Programming Models, IEEE, 1993.
   • B. Alpern, L. Carter, and J. Ferrante, "Space-Limited Procedures: A Methodology for Portable High-Performance", Second Intnl Conference on Massively Parallel Programming Models, IEEE, 1995.
   • B. Alpern and L. Carter, "Towards a Model for Portable Parallel Performance: Exposing the Memory Hierarchy", Portability and Performance for Parallel Processing, Hey T. and J. Ferrante ed., John Wiley and Sons, 1994.
   • B. Alpern, L. Carter, E. Feig, and T. Selker, "The Uniform Memory Hierarchy Model of Computation", Algorithmica, 1994.

5. Tensor-product techniques
   • D. L. Dai, S. K. S. Gupta, S. D. Kaushik, J. H. Lu, R. V. Singh, C.-H. Huang, P. Sadayappan and R. W. Johnson, "EXTENT: A Portable Programming Environment for Designing and Implementing High Performance Block Recursive Algorithms", Supercomputing '94 Proceeedings, ACM, 1994.
   • Z. Li, J. Reif, Synthesizing Efficient Out-of-Core Programs for Block Recursive Algorithms using Block-Cyclic Data Distributions, Technical Report, Dept. of Computer Science, Duke Univ., 1996.
   • S. D. Kaushik, S. Sharma, C.-H. Huang, J. R. Johnson, R. W. Johnson, and P. Sadayappan, "An Algebraic Theory for Modeling Direct Interconnection Networks", J. Info. Science and Engineering (to appear).

Parallel Programming Models

1. Nested Data-Parallel model
   • G. E. Blelloch, S. Chatterjee, J. C. Hardwick, J. Sipelstein and M. Zagha, "Implementation of a Portable Nested Data-Parallel Language", Fourth ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (PPoPP), ACM, 1993.
   • J. Riely, J. Prins, P. Iyer, "Provably Correct Vectorization of Nested-Parallel Programs", Second Intnl Conference on Massively Parallel Programming Models, IEEE, 1995.
   • G. Blelloch, J. Greiner, "A Provable Time and Space Efficient Implementation of NESL", to appear in International Conference on Functional Programming, ACM 1996.
   • D. Gannon et al., "pC++", Scientific Programming, 1993.

2. Concurrent Object-Oriented model
   • L. Kale, S. Krishnan, "Charm++ : A Parallel Object-Oriented Language based on C++".
   • A. Chien, U. Reddy, J. Plevyak, J. Dolby, ICC++ -- A C++ Dialect for High Performance Parallel Computing, Second International Symposium on Object Technologies for Advanced Software, (ISOTAS '96), 1996.
   • A. Chien, et al., "Concurrent Aggregates", JPDC 25 (2), 1995.
   • Mentat (Grimshaw)

3. Coordination model
   • M. Chandy, "Compositional C++", in Designing and Building Parallel Programs, I. Foster, ed., Addison-Wesley, 1995.

4. Parallelism from Sequential Programs
   • S. Rau, J. Fisher, "Instruction Level Parallelism: Overview, History and Perspective", Journal of Supercomputing 7, 1993.

Parallel Algorithms

1. Fast N-body algorithms
   • Salmon and Warren, ...
   • J.P. Singh et al., "Load Balance and Locality in Adaptive N-body Simulations", JPDC 27 (2), 1995.

2. Sorting algorithms
   • D. Helman, D. Bader, J. JaJa, "Parallel Algorithms for Personalized Communication and Sorting with an Experimental Study", Technical Report, Univ. of Maryland CS Dept.

Parallel Architecture Issues

1. Bounded Contention Shared Memory
   • G. Blelloch, , P. Gibbons, Y. Matias, M. Zagha, "Accounting for Memory Bank Contention and Delay in High-Bandwidth Multiprocessors", Proc. 7th SPAA, ACM, 1995.
   • P. Gibbons, Y. Matias, V. Ramachandran, "Efficient Low-Contention Parallel Algorithms", Proc. 6th SPAA, ACM, 1994.

2. Processors and Memory
   • W. Wulf, S. McKee, "Hitting the Memory Wall: Implications of the Obvious", CAN 23 (1), ACM, 1995.
   • D. Burger. J. Goodman, A. Kagi, "Quantifying Memory Bandwidth Limitations of Current and Future Microprocessors", Proc. 23rd ISCA, ACM, 1996.
   • A. Saulsbury, F. Pong, A. Nowatzyk, "Missing the Memory Wall: The Case for Processor/Memory Integration" Proc. 23rd ISCA, ACM, 1996.