Parallel Programming using Message Passing Interface (MPI) (Part of HPC @ IU Workshop, Indiana University) Warning: These notes are based on the decommissioned AVIDD cluster and may contain information, that's not applicable any more! Click here for a printable version of this document

---

Working on the AVIDD Cluster

• Introduction - What will I learn in this workshop?

• Initial Setup
  • SSH onto AVIDD
  • Working Directory Tree for Tutorial Examples
  • Comment about additions to your ~/.soft file
  • Grab Interactive Compute Nodes using qsub -I

Message Passing Interface (MPI) - Simple Programming Examples

• My First MPI Program
• How did the HelloWorld program compile? Makefile

• Basic Concepts in Parallel Programming
  • Parallel Programming Paradigms
  • Hardware Architectures used by Parallel Computers

• RoundRobin - Simple Message Passing Illustrated
• The RoundRobin program -- Explanation of MPI function calls

• Data Decomposition

• A Naturally Parallelizable Computation
• Computing PI using Serial Code
• Computing PI using Parallel Code

• The 2-D Diffusion Equation
• Serial Numerical Solution of the Diffusion Equation
  • Results of serialDiffusion
• Parallelizing the Numerical Solution of the Diffusion Equation
• Parallelized Solution for Diffusion Equation

MPI I/O - Parallel File Access

• Simple MPI I/O and Error Handling
  • What is MPI I/O
  • Program: mpi_io.c
  • The mpi_io Program Explained

Getting Ready for Graphical Applications from Compute Nodes

• Tweaking Security Settings
  • Tweaking Linux firewall to open graphical applications
  • Using Windows and XLiveCD (To run AVIDD-based graphical apps. on a Windows computer)

Profiling Parallel Code using Intel Trace Analyzer/Collector (ITA/C)

• Profiling roundRobin using Intel Trace Analyzer (ITA)
  • ITA Initial Timeline
  • ITA Detailed Timeline
  • ITA Activity Chart

• Profiling parallelPi using ITA

• More on Tracing parallelPi - Using Intel Trace Collector (ITC) Instrumentation
  • Using ITC instrumentation
  • Adding ITC instrumentation to the parallelPi source code
• Viewing parallelPi's ITC Instrumented trace

• Profiling parallelDiffusion using Intel Trace Analyzer (ITA)
  • Recall parallelDiffusion Output
  • The ITA Profile

Debugging Parallel Code using TotalView

• Introducing TotalView: Debugging roundRobin
  • Preparing for a TotalView parallel debugging session
  • Starting the TotalView parallel debugging session
  • Exploring TotalView

• Debugging parallelDiffusion using TotalView
  • Recall: Preparing for a TotalView parallel debugging session
  • Starting the TotalView parallel debugging session
  • Finding the bug in parallelDiffusion

• Concluding Remarks

Using MPICH (instead of LAM) to run MPI code (Appendix)

• Compiling and Running Parallel jobs using MPICH on AVIDD

More Documentation!

• Useful Links to Documentation