OpenMP with gcc over network?

Question

Does anyone know how to run OpenMP with gcc on two, three or more computers (identical computers with identical settings) that are connected together by router?
At the moment I have ssh connection between each node (with each node having keygen ID of all others).... so no password is needed for any ssh connection request.
Last edited by kdar (2013-03-20 17:31:55)

Answer 1

It is important to remember that OpenMP uses shared memory as a means of communication between multiple threads. Generally speaking, OpenMP cannot be used to parallelize an application over multiple machines (because they do not share the same memory address space). I seem to recall reading about some research going into virtual shared memory systems for clusters, which would allow an threaded application to be run as if it was on a single machine. Unfortunately, I can't seem to find the article at the moment.
Anyway, for distributed memory systems (e.g. clusters), the Message Passing Interface (MPI) is the de-facto standard. There are a number of different MPI implementations, with Open MPI and MPICH being popular choices. The main advantage of MPI is that it defines a portable standard for developing parallel applications. In theory, an MPI based parallel application will run on anything from a single multi-core desktop PC, to a Beowulf UNIX/Linux cluster, to a top500 supercomputer. Some of the better MPI implementations will even optimize the communication based on the particular environment you are running in. The main disadvantage of MPI is that it generally requires one to use Domain Decomposition Methods (DDM) to efficiently parallelize their application. While DDM is very powerful, it does carry some important trade-offs: (1) it cannot be easily retrofitted to existing serial applications, and (2) it can impact the numerical stability and accuracy of your simulation. This is why there has been a fair amount of research going into hybrid MPI/OpenMP applications; where MPI is used for coarse grained parallelizm, and OpenMP for the fine grained parallelizm.
In your case, it sounds like you're trying to run a computation on commodity PC's over a home/office LAN? In this case, you should look into the possibility of configuring these machines to function as a small Beowulf cluster. Then running MPI and MPI/OpenMP applications will be very straightforward. Of course, before you invest in doing this, you'll want to estimate the parallel efficiency you would get with this configuration. Unless you're using high speed networking equipment, it may turn out that your application doesn't run significantly faster on two machines than one. For sure, it won't be twice as fast. (I'm assuming that the motivation for doing this is to make your application run faster; i.e. you're not memory bound.)
I hope this helps.
EDIT: I also seem to recall reading about tools for Grid Computing. The main idea behind Grid Computing is being able to create ad-hoc clusters out of an arbitrary collection of networked computers (either via the Internet or LAN). So, perhaps do some Googling with "grid computing", and see if that turns up anything that helps.
Last edited by bsilbaugh (2013-03-23 00:24:32)