The receive described in Section 
can be started
whether or not a matching send has been posted.
That version of receive is blocking.
blocking
It returns only after the receive buffer contains the newly received
message. A receive could complete before the matching send
has completed (of course, it can complete only after the matching send
has started).
The send operation described in Section
can be started whether or not a
matching receive has been posted. That version of send
is blocking.
It does not return until the message data
and envelope have been safely stored away so that the sender is
free to access and overwrite the send buffer.
The send call is also potentially non-local.
non-local
The message might be copied directly into the matching receive buffer,
or it might be copied into a temporary system buffer. In the first case, the
send call will not complete until a matching receive call occurs, and so,
if the
sending process is single-threaded, then it will be blocked until this time.
In the
second case, the send call may return ahead of the matching receive call,
allowing a single-threaded process to continue with its computation.
The MPI implementation may make either of these
choices. It might
block the sender or it might buffer the data.
Message buffering
decouples the send and receive operations. A blocking send might
complete as soon
as the message was buffered, even if no matching receive has been executed
by the receiver. On the other hand, message buffering can be expensive,
as it entails additional memory-to-memory copying, and it requires the
allocation of memory for buffering. The choice of the right amount of buffer
space to allocate for communication and of the buffering policy to use is
application and implementation dependent. Therefore, MPI
offers the choice of
several communication modes that allow one to control the choice of the
communication protocol. Modes are described in
communication modesmodescommunication protocol
protocol, communication
Section . The choice of a buffering policy for the
standard mode send described in
standard modemode, standard
Section is left to the implementation.
In any case, lack of buffer space will not cause a standard send
call to fail, but will merely cause it to block. In well-constructed
programs, this results in a useful throttle effect.
throttle effect
Consider a situation where a producer repeatedly produces
new values and sends them to a consumer. Assume that the producer produces
new values faster than the consumer can consume them.
If standard sends are used, then the producer will be automatically throttled,
as its send operations will block when buffer space is unavailable.
In ill-constructed programs, blocking may lead to a deadlock situation, where all processes are deadlock blocked, and no progress occurs. Such programs may complete when sufficient buffer space is available, but will fail on systems that do less buffering, or when data sets (and message sizes) are increased. Since any system will run out of buffer resources as message sizes are increased, and some implementations may want to provide little buffering, MPI takes the position that safe programs safe program do not rely on system buffering, and will complete correctly irrespective of the buffer allocation policy used by MPI. Buffering may change the buffering performance of a safe program, but it doesn't affect the result of the program.
MPI does not enforce a safe programming style. Users are free to take advantage of knowledge of the buffering policy of an implementation in order to relax the safety requirements, though doing so will lessen the portability of the program.
The following examples illustrate safe programming issues.
Example 2.7 An exchange of messages. CALL MPI_COMM_RANK(comm, rank, ierr) IF (rank.EQ.0) THEN CALL MPI_SEND(sendbuf, count, MPI_REAL, 1, tag, comm, ierr) CALL MPI_RECV(recvbuf, count, MPI_REAL, 1, tag, comm, status, ierr) ELSE IF (rank.EQ.1) THEN CALL MPI_RECV(recvbuf, count, MPI_REAL, 0, tag, comm, status, ierr) CALL MPI_SEND(sendbuf, count, MPI_REAL, 0, tag, comm, ierr) END IF This program succeeds even if no buffer space data is available. The program is safe and will always complete correctly. Example 2.8 An attempt to exchange messages. CALL MPI_COMM_RANK(comm, rank, ierr) IF (rank.EQ.0) THEN CALL MPI_RECV(recvbuf, count, MPI_REAL, 1, tag, comm, status, ierr) CALL MPI_SEND(sendbuf, count, MPI_REAL, 1, tag, comm, ierr) ELSE IF (rank.EQ.1) THEN CALL MPI_RECV(recvbuf, count, MPI_REAL, 0, tag, comm, status, ierr) CALL MPI_SEND(sendbuf, count, MPI_REAL, 0, tag, comm, ierr) END IF The receive operation of the first process must complete before its send, and can complete only if the matching send of the second processor is executed. The receive operation of the second process must complete before its send and can complete only if the matching send of the first process is executed . This program will always deadlock. Example 2.9 An exchange that relies on buffering. CALL MPI_COMM_RANK(comm, rank, ierr) IF (rank.EQ.0) THEN CALL MPI_SEND(sendbuf, count, MPI_REAL, 1, tag, comm, ierr) CALL MPI_RECV(recvbuf, count, MPI_REAL, 1, tag, comm, status, ierr) ELSE IF (rank.EQ.1) THEN CALL MPI_SEND(sendbuf, count, MPI_REAL, 0, tag, comm, ierr) CALL MPI_RECV(recvbuf, count, MPI_REAL, 0, tag, comm, status, ierr) END IF The message sent by each process must be copied somewhere before the send operation returns and receive operation starts. For the program to complete ,it is necessary that atleast one of the two messages to be buffered. Thus this program will succeed only if the communication system will buffer atleast count words of data.Otherwise, the program will deadlock. The success of this program will depend on the amount of buffer space available in a particular implementation ,on the buffer allocation policy used , and on other concurrent communication occuring in the system. This program is unsafe.
