Course Description
This course covers general issues of parallel and distributed
processing, which include system, algorithm, programming, performance
evaluation, and application of parallel computers.
Approximately two third of the course will be devoted to
basic concepts and techniques, and the remaining third will
be on programming and assorted current topics in parallel
processing, including cloud computing, data intensive (big data) systems, and GPGPU. We will not follow any particular text
through out the entire class. Instead, we will use two
texts as the general guideline of the lecture, one
for basic concepts and another one for programming
skill. Three reference books are also listed.
Assigned readings may give from time to time to complement the lectures.
Homework assignments will include question/answer execises and programming
assignments.
For the last part of the course we will read
an assortment of papers and work on different term projects.
Topics include:
- The Scope of Parallel Computing
- Parallel Architecture: Basic issues and its influence on programming
- Performance Evaluation
- Algorithm Design and Analysis: case study
- Parallel Programming
- Network, communication, and distributed processing
- System Consideration: Partition, Communication, Scheduling and OS issues
- Data Access and I/O
- Cluster and Distributed Network Computing
- Other Current Trend of Parallel Processing
Course Materials
-
Recommended
- Ananth Grama, Anshul Gupta, and George Karypis, Vipin Kumar
- Introduction to Parallel Computing (2nd Edition)
- Addison-Wesley, 2003 (see here for additional resources related to the text).
- W. Gropp, et. al.,
- Using MPI Tow -Volume Set
- MIT Press, 1999 (see here for additional resources related to the text).
- Introduction to Parallel Computing (2nd Edition)
-
References
- Kai Hwang,Jack Dongarra, and Geoffrey C. Fox
- Distributed and Cloud Computing: From Parallel Processing to the Internet of Things
- Morgan Kaufmann Publishers, 2011 (see here for additional resources related to this text.)
- Ian Foster and Carl Kesselman
- The Grid 2: Blueprint for a New Computing Infrastructure
- Morgan Kaufmann Publishers, 2003 (see here for additional resources related to the text.)
- David Kirk, and Wen-Wei Hwu
- Programming Massively Parallel Processors
- Morgan Kaufmann Publishers (2nd Edition), 2012 (see here for additional resources related to the text.)
- Distributed and Cloud Computing: From Parallel Processing to the Internet of Things