Bug 5525

First of all, I am not referring to the mpiexec command of MPICH2 here. If you
enter 'mpiexec' in Google, the following page comes up first, and this is what
we need: http://www.osc.edu/~pw/mpiexec/index.php

mpiexec is the recommended/comfortable way to run MPI jobs within PBS/TORQUE.
It implements many MPI-version-specific protocols for starting MPI tasks (makin
g it easy to switch from one MPI implementation to another) and supports
correct accounting of CPU time for parallel jobs. Its mode of use is that you
submit a PBS job, specifying your multi-node requirements as usual, and then
within the job script you run mpiexec as shown below:

/opt/mpiexec/bin/mpiexec -np 4 -comm mpich-p4 /path/to/your/mpiapp

If you want to switch from MPICH/Gigabit Ethernet to MVAPICH/InfiniBand:

/opt/mpiexec/bin/mpiexec -np 4 -comm mpich-ib /path/to/your/mpiapp

This is a lot simpler than the rsh/mpirun approach used by the current (4.0.5)
PBS adapter. There is no machinefile to care about. Also, you don't need to
enable password-less rsh access from the front machine to the execution nodes
or between nodes, provided that they share a file system with the front machine
(as is required by GRAM's file staging anyway). Finally, the interface exposed
by mpiexec (as can be seen above) is much simpler than the mpirun/rsh
interface, which would simplify the implementation of the PBS adapter.