The pvmd uses a wait context (waitc) to hold state when a thread of operation must be interrupted. The pvmd is not truly multithreaded but performs operations concurrently. For example, when a pvmd gets a syscall from a task and must interact with another pvmd, it doesn't block while waiting for the other pvmd to respond. It saves state in a waitc and returns immediately to the work() loop. When the reply arrives, the pvmd uses the information stashed in the waitc to complete the syscall and reply to the task. Waitcs are serial numbered, and the number is sent in the message header along with the request and returned with the reply.
For many operations, the TIDs and kind of wait are the only information saved. The struct waitc includes a few extra fields to handle most of the remaining cases, and a pointer, wa_spec, to a block of extra data for special cases-the spawn and host startup operations, which need to save struct waitc_spawn and struct waitc_add.
Sometimes
more than one phase of waiting is necessary-in
series, parallel,
or nested.
In the parallel case, a separate waitc is created for each foreign host.
The waitcs are
peered (linked in a list)
together to indicate they pertain to the same operation.
If a waitc has no peers,
its peer links point to itself.
Usually,
peered waitcs share data,
for example, wa_spec.
All existing parallel operations
are conjunctions;
a peer group is finished when every waitc
in the group is finished.
As replies arrive,
finished waitcs are collapsed out of the list and deleted.
When the
finished waitc is the only one left,
the operation is complete.
Figure 
shows single and peered waitcs
stored in
waitlist (the list
of all active waitcs).
When a host fails or a task exits, the pvmd searches waitlist for any blocked on this TID and terminates those operations. Waitcs from the dead host or task blocked on something else are not deleted; instead, their wa_tid fields are zeroed. This approach prevents the wait id's from being recycled while replies are still pending. Once the defunct waitcs are satisfied, they are silently discarded.
