SPRING SEMESTER 2011

Assessment Pattern

Unit(s) of Assessment	Weighting Towards Module Mark( %)
Exam:   4 hour practical examination	50
Coursework (individual):   1 piece of assigned coursework, due in week 6	25
Coursework (individual):   1 piece of assigned coursework, due in week 10	25
Qualifying Condition(s)    A weighted aggregate of 40% is required to pass the module.

Module Overview

The course introduces concepts of parallel computing by considering different architectures that support this, and working through different categories of examples. The implementation of such solutions and their subsequent analysis gives practical experience and an understanding of the difficulties involved.

Good programming skills; prior knowledge of C/C++ is helpful, as these are the languages used throughout the module. Students who do not have a C/C++ background are strongly encouraged to take a prior course on C++.

The module aims to develop the student’s ability to think clearly about the relationship between a problem abstraction and architectural implementation details. We focus on the techniques for the development of solutions of scientific computing problems on parallel architectures. A number of case studies are considered to illustrate facets of the subject. This course will enable you to gain experience in building parallel solutions for scientific computing problems. Most of the course will use C and C++ to access the parallel computing libraries of the various architectures.

Module Content

 

The lesson schedule and contents below is subject to change, following revisions based on earlier deivery.   Lesson 1: Introduction   ·          Motivation (why parallelize?)   ·          Types of parallelism:   -          Implicit vs Explicit   -          Floyd Taxonomy   ·          Example architectures:   -          SMP/multicore; Cell BE; GPGPU;   -          clusters; grids; clouds;   ·          Problem bottlenecks/classification:   -          compute-bound   -          I/O-bound (or memory-bound)   ·          Programming models:   -          shared memory (e.g. threads)   -          message-passing (e.g. MPI)   -          vectorization   ·          Introduce systems used in class:   -          Intel Core 2 Duo E8400 @3.00GHz (duck/swan labs)   -          Intel Xeon E3113 @3.00GHz (ps3 cluster head node)   -          nVidia 9400GT (duck/swan labs)   -          Cell BE (ps3 cluster)   Lesson 2: Threading   ·          Thread basics   -          What are threads?   -          Threads and processes   -          Advantages of thread programming   -          Common models for programming with threads   ·          POSIX threads   -          Creation and termination   -          Synchronization primitives   -          Joining and detaching   -          Mutexes   Lesson 3: SIMD Extensions   ·          Introducing SIMD Extensions   -          Vector vs Scalar processing   -          Vector data types   -          Byte order   -          Programming interfaces   ·          SIMD Programming with GCC   -          Rudimentary example   -          Vector reorganization   -          Data type conversion   -          Elimination of conditional branches   Lesson 4: Cell BE Programming (part 1)   ·          The Cell Broadband Engine   -          Cell BE Architecture   -          Cell BE Programming Model   ·          SPE Programming   -          A simple SPE program   -          Using DMA transfer   Lesson 5: Cell BE Programming (part 2)   ·          Parallel SPE Programming   -          SIMD programming on the SPE   -          Using Multiple SPEs   ·          Advanced Cell Programming   -          Communication between PPE and SPE   ·          (mailboxes, signal notification registers)   Lesson 6: Cell BE Programming (part 3)   ·          Advanced Cell Programming   -          Effective Utilization of DMA Transfer   -          Differences in PPE and SPE Data Representation   -          Vector Data Alignment   -          Scalar Operations on SPE   -          SPE Program Embedding   Lesson 7: GPU Programming (part 1)   ·          Introduction to GPU Programming   -          GPUs and GPU Architecture   -          Introduction to CUDA   ·          GPU Programming with CUDA   -          A simple CUDA example   -          Blocks and thread hierarchy   -          Memory hierarchy   -          Device and host separation   -          Putting everything together   Lesson 8: GPU Programming (part 2)   ·          Optimizing Performance   ·          Advanced CUDA Facilities   -          Shared Memory   -          Texture Memory   -          Page-Locked Host Memory   -          Concurrent Execution   Lesson 9: Cluster Programming with MPI   ·          Introduction to Clusters   ·          Message-Passing Interface API   -          MPI Environment Management   -          Point-to-Point Communications   -          Collective Communications   Lesson 10: Exploiting Parallelism   ·          Introduction to Grids & Public Computing   ·          Parallelism at different levels   -          Granularity and level   -          Which way(s) to use?   -          System-level concerns   ·          Closure

APRIL 2011