Previous: Installation of Nero2d, Up: Installation on GNU/Linux

2.1.3 Parallel Nero2d

If you have a parallel machine, and you want to run some large simulations or benchmark Nero2d for your machine (it would be greatly appreciated if the results of these benchmarks are sent to us) you can chose to compile a parallel version of Nero2d. For communication, Nero2d relies on the MPI of Message Passing Interface.

If you do not have MPI installed on your system, you can try use the following free MPI implementations: MPICH, LAM-MPI and Open-MPI. If you use MPICH, try using version 2.0 or higher, as we noticed that its performance is considerably higher. We recommend LAM-MPI is you are using an Gigabit Ethernet interconnect (as we do), because its performance is the best of the three. We recommend for Gigabit Ethernet that so-called jumbo frames are disabled (MTU=1500). Open-MPI is promising new MPI implementation with many nice options and features (like support for multiple Gigabit Ethernet connections to a single machine). At the time of writing, it is however still a bit slower than its LAM-MPI counterpart.

The three MPI implementations can be freely downloaded

After installation, add the directory to the MPI compiler scripts to the PATH environment (you will need the C++ MPI compiler, usually called mpic++, mpicxx or mpiCC). The configure script will try to detect the correct one. To enable MPI support in Nero2d add the --enable-mpi flag to the configure script. 

         ./configure --enable-mpi

As is previously described (see Tutorial), Nero2d requires two input files, namely the config.dat script and the input geometry file (.igf). In a parallel environment, it is required that all processes have access to these files so they can be read in parallel. Most likely, some sort of network file system is mounted on each machine, so all the machines see the same files. If this is not the case, you must make sure that the two input files are locally available on each machine in the same path. Make sure that all input files are identical! If this requirement is not met, Nero2d will crash once it starts its iterations.

We have noticed that when using a network file system, it might take a few seconds before the changes made to a file on one machine propagate to the other machines (most likely some buffering issues). Future versions of Nero2d will communicate a fingerprint of the input files to make sure that they are all identical.

If you want to do some serious benchmarking, we suggest using the benchmarks which come along with Nero2d and are located in /share/benchmark. You might want to read the README file first.

Nero2d can also be compiled with MPE support. Using the MPE library, bottlenecks can be detected in a post-mortem fashion. The MPE library is included in MPICH, but can be downloaded separately www-unix.mcs.anl.gov/perfvis and used in combination with LAM-MPI and Open-MPI. To enable support for MPE in Nero2d, use the --enable-mpe flag 

     ./configure --enable-mpi --enable-mpe

To launch a parallel job, use the mpirun tool 

     mpirun -np xx ./nero2d inputfile.igf

Change xx by the number of processes you want to use. To specify which machines to use, you most likely want to create a so-called machinefile which contains their names or ip-numbers. We refer to the user's guide of your MPI implementation of choice.