Version of dpi_pipe_c_send() that can handle messages of unlimited size

From: Per Bojsen <bojsen_at_.....>
Date: Tue Mar 21 2006 - 19:45:29 PST
Hi,

I have enclosed an example implementation of a version of
dpi_pipe_c_send() that can handle messages of unlimited size.
By unlimited I refer to messages that are larger than the
current pipe buffer size.  For now I called the function
dpi_pipe_c_send_nl().  `nl' is for `no limit'.  It has the
exact same prototype as dpi_pipe_c_send() and can thus be
used as a direct replacement.  Internally, I chose to use
John's dpi_pipe_c_send() to transfer the individual segments
of the message.  The code also shows some possible optimizations
when one can exploit that the segment size is an integer
number of svBitVecVal words.  This happens when number of
elements in the segment multiplied by number of bytes per
segment is divisible by 4 (4 bytes per svBitVecVal word).
It turns out that if the pipe depth is 4 elements or more
one can always find such a nice segment size.  When the segment
size has this property, copying and extracting bit fields
out of the original svBitVecVal data buffer can be avoided.

The code for dpi_pipe_c_receive_nl() would follow a similar
pattern.  I can create this if there is any interest.

Note, that my code demonstrates that the functionality I
was calling for can be easily created at the application
layer.  Even though this is the case, I believe the interface
should support this at least for the blocking API since it is
fairly straightforward and it makes the interface more
intuitive.  This is also better aligned to the goal of
handling variable length messages.

I can live with the non-blocking API having messages limited
to the buffer size, although it would be preferable to have
it support segmentation/reassembly of messages as well.

I believe unlimited message size should only be supported
on the software side.  The reason is that on the hardware
side messages or message fragments are read into bit
vectors passed to the pipe functions.  These vectors are
fixed hardware resources that must be allocated.  Typically
they get implemented as flip flops and are thus expensive.
Typically much more expensive than the pipe buffers for the
same number of bits.  Therefore it is reasonable to assume
that most practical uses of the pipes API will use fairly
narrow data vectors (compared to average message size)
and it is not unreasonable to require pipe buffers to be
larger than the largest data vector used to access that
particular pipe.  In other words, it is not unreasonable
to require that pipe buffers are always large enough to
accomodate any calls from the hardware side and
segmentation/reassembly is thus not required on the hardware
side.

Brian can you add the proposal to change the current blocking
API to allow unlimited message sizes as an IM?  This may be
related to Shabtay's issue e.

Per



Received on Tue Mar 21 19:45:34 2006

This archive was generated by hypermail 2.1.8 : Tue Mar 21 2006 - 19:45:37 PST