Implementing the non-blocking pipes API

Hi,

in order to help the process of refining the pipes API along and to
improve the understanding of the semantics of the pipes API, I sat
down for a few hours and created a crude implementation of the
nob-blocking subset of the pipes API for DPI enabled simulators
based on John's latest distribution of 2/24/06.  I have enclosed the
current state of this implementation.  Some caveats:

  1) The implementation is not meant to be product grade.  It is a
     toy but is sufficiently rich to allow experimentation with the
     pipes API.

  2) No attempt has been made to make the implementation efficient.
     The goal was not performance but quick and simple implementation.

  3) The code has only been lightly tested.  I have not yet tested
     it with a simulator.  The enclosed test is C only.

  4) The implementation is based on my current understanding of the
     API and its semantics which is undoubtedly wrong in some areas.
     See below.

Note, it is fairly trivial to extend the implementation to include
the blocking API.  I think John has an example of how to implement
the blocking API for SystemC.  This should be usable with my
implementation of the non-blocking API bugs in the implementation
notwithstanding.

On my quest to implement the non-blocking API I encountered a
number of areas where I was unsure about the semantics of the API.
Here are some questions that was generated as a result of this:

  a) There are no calls on the C side to create a pipe.  Pipes are
     `created' from the HDL side?

  b) The description of dpi_pipe_c_try_receive() states that no data
     is transferred unless at least the indicated number of elements
     exist in the pipe.  However, the method also has the
     num_elements_read argument which seems to indicate it is possible
     to receive less than what was asked for.  I assume this comes
     into play during flushing, i.e., when a pipe is being flushed,
     the try_receive() calls may return less than what was requested,
     correct?

  c) In the dpi_pipe_c_try_receive(), is it the user's responsibility
     to ensure the memory area pointed to by data is large enough to
     receive the data requested?  Similarly for
     dpi_pipe_hdl_try_receive(), dpi_pipe_c_try_send(), and
     dpi_pipe_hdl_try_send().

  d) How are elements packed in the data array?  Are they byte packed
     or svBitVecVal (i.e., 32-bit word) packed?

  e) In dpi_pipe_c_try_receive() when flushing is going on, it is
     assume that fewer elements than requested may be returned.  What
     happens if the number of bytes currently in the buffer is not a
     multiple of the element size?  Will a partial element be returned
     or will the last few bytes (less than bytes_per_element) be left
     in the buffer despite flushing being in effect?

  f) In dpi_pipe_c_try_receive() and dpi_pipe_hdl_try_receive(), what
     is num_elements_read set to if no data transfer happens?  Is it
     set to 0 or is it not set at all?

  g) EOMs can coalesce if the consumer asks for a large enough chunk
     of data that will contain 2 or more messages sent by the producer
     with EOMs set.  Is this correct?

  h) Are input and output pipes fully symmetric?  Can they be
     implemented with shared try_send() and try_receive() functions?

  i) Message size limit: I think we decided to make it unlimited.
     This requires the implementation to dynamically resize buffering
     at least on the software side, correct?

  j) What are the exact conditions for calling the notify callback?

Since I didn't have the answers to the above questions I had to make
some assumptions during implementation:

  A1) Input and output pipes are symmetric.  They are
      implemented with shared try_send() and try_receive() functions.

  A2) EOM acts like a flag on the last byte of the
      message.  On the consumer side, whenever this byte is part of the
      data consumed EOM will be true.

  A3) num_bytes_read is set to 0 when no data transfer happens.

  A4) When the number of bytes available in a buffer is not a multiple
      of the element size and flushing is in progress and
      dpi_pipe_c_try_receive() is asking for more data than available,
      then the last element returned will be partial.  Otherwise the
      buffer will not flush completely.

  A5) Bytes are packed in svBitVecVal words in a little endian manner.
      Elements are byte packed both within calls and between calls.

  A6) No error checking is done on pipe context before creating pipes.

  A7) The notify callback is called on successful transfer of data and
      successfull flushing.

Some known issues exist:

  * Limitation: Currently, the buffer size is fixed and prevents any
    message larger than the buffer size to be sent.

  * Potential bug: The EOM handling does not look right in the output
    of the PipesNBTest1 test.  It looks like EOM is set on the first
    message received from the pipe rather than the last or perhaps
    some in the middle.  Add some code to track where we should expect
    to see EOM.

Also, I think assumption A7 is wrong.  I think you actually want
the callback to happen when blocking is required, rather than
when data is transferred.

There are some makefiles included with the code that are trivial
and should be easy to modify to your environment.  You will need
John's dpi_pipes.h include file to compile.  Also, I included
a stripped down version of svdpi.h and a dummy implementation of
svGetScope() and svSetScope() to allow testing without a
simulator.  If you want to use the code with a simulator you must
exclude these from the library and point to the svdpi.h supplied
with the simulator.

Please keep the caveats in mind when using this code.  Neither
Zaiq nor I are responsible for any mishaps that may occur due to
using this code :-)

Have fun,
Per