Per, I took a look at this model and took a slightly different approach that I think gets around some of the tricky parts you were dealing with. I've attached the new version (see attached file). Basically what I do is put in a small optimization where if the bytes_per_element is a multiple of sizeof(svBitVecVal) it means that all elements are word aligned and you can do it slightly more efficiently. But if that is not the case, then the code deals with the data transfer at byte granularity using the svPut/GetPartselBit() helper functions. Furthermore, I modified dpi_pipe_c_send() directly to handle unlimited buffer sizes. But it calls an internal dpi_pipe_c_send_to_fit() which is a verbatim copy of the original dpi_pipe_c_send() that I had. Symetric changes can be done for dpi_pipe_c_receive(). I also agree with your comments about not needing to support unlimited buffer size on the HDL side due to practical considerations. I verified that this compiled but have not yet had a chance to test it but it should be fairly easy. I also was thinking about another approach of putting byte level support inside the non-blocking call itself. The idea is that the non-blocking call would be given a byte index into the svBitVecVal array that it gets, and return the number of bytes it successfully did transfer. This returned value could then be used by the caller to bump the byte index for the next call in the loop, without changing the base svBitVecVal[] data pointer. This would allow for a more efficient implementation that would potentially reduce the amount of data copying required. The new prototype for the call would look something like this: int dpi_pipe_c_try_send( void *pipe_handle, // input: pipe handle int byte_offset, // input: byte offset within data array int bytes_per_element, // input: #bytes/element int num_elements, // input: #elements to be written const svBitVecVal *data, // input: data svBit eom ); // input: end-of-message marker flag Instead of return success (1) or fail (0) it would return # elements transferred (which on a full pipe would still return a 0 as before). The revised dpi_pipe_c_send() would then look something like this: void dpi_pipe_c_send( void *pipe_handle, // input: pipe handle int bytes_per_element, // input: #bits/element int num_elements, // input: #elements to be written const svBitVecVal *data, // input: data svBit eom ) // input: end-of-message marker flag { int byte_offset = 0, elements_sent; while( num_elements ){ elements_sent = dpi_pipe_c_try_send( pipe_handle, byte_offset, bytes_per_element, num_elements, data, eom ) * bytes_per_element; // if( pipe is full ) wait until OK to send more if( elements_sent == 0 ){ sc_event *ok_to_send = (sc_event *) dpi_pipe_get_notify_context( pipe_handle ); // if( notify ok_to_send context has not yet been set up ) ... if( ok_to_send == NULL ){ ok_to_send = new sc_event; dpi_pipe_set_notify_callback( pipe_handle, notify_ok_to_send_or_receive, ok_to_send ); } wait( *ok_to_send ); } else { byte_offset += elements_sent * bytes_per_element; num_elements -= elements_sent; } } } Almost as concise as the original - but now with unlimited size ! This is a little cleaner because now at the level of dpi_pipe_c_send(), you don't even care about querying buffer size. It's all handled within the NB API. I'll work this a little more and send out an update. -- johnS Per Bojsen wrote: > Hi, > > I have enclosed an example implementation of a version of > dpi_pipe_c_send() that can handle messages of unlimited size. > By unlimited I refer to messages that are larger than the > current pipe buffer size. For now I called the function > dpi_pipe_c_send_nl(). `nl' is for `no limit'. It has the > exact same prototype as dpi_pipe_c_send() and can thus be > used as a direct replacement. Internally, I chose to use > John's dpi_pipe_c_send() to transfer the individual segments > of the message. The code also shows some possible optimizations > when one can exploit that the segment size is an integer > number of svBitVecVal words. This happens when number of > elements in the segment multiplied by number of bytes per > segment is divisible by 4 (4 bytes per svBitVecVal word). > It turns out that if the pipe depth is 4 elements or more > one can always find such a nice segment size. When the segment > size has this property, copying and extracting bit fields > out of the original svBitVecVal data buffer can be avoided. > > The code for dpi_pipe_c_receive_nl() would follow a similar > pattern. I can create this if there is any interest. > > Note, that my code demonstrates that the functionality I > was calling for can be easily created at the application > layer. Even though this is the case, I believe the interface > should support this at least for the blocking API since it is > fairly straightforward and it makes the interface more > intuitive. This is also better aligned to the goal of > handling variable length messages. > > I can live with the non-blocking API having messages limited > to the buffer size, although it would be preferable to have > it support segmentation/reassembly of messages as well. > > I believe unlimited message size should only be supported > on the software side. The reason is that on the hardware > side messages or message fragments are read into bit > vectors passed to the pipe functions. These vectors are > fixed hardware resources that must be allocated. Typically > they get implemented as flip flops and are thus expensive. > Typically much more expensive than the pipe buffers for the > same number of bits. Therefore it is reasonable to assume > that most practical uses of the pipes API will use fairly > narrow data vectors (compared to average message size) > and it is not unreasonable to require pipe buffers to be > larger than the largest data vector used to access that > particular pipe. In other words, it is not unreasonable > to require that pipe buffers are always large enough to > accomodate any calls from the hardware side and > segmentation/reassembly is thus not required on the hardware > side. > > Brian can you add the proposal to change the current blocking > API to allow unlimited message sizes as an IM? This may be > related to Shabtay's issue e. > > Per > > > > ------------------------------------------------------------------------ > > /* > ** DPI blocking pipes for messages of arbitrary size. > ** > ** Copyright (C) 2006 Zaiq Technologies, Inc. All Rights Reserved. > ** > ** Author: Per Bojsen <per.bojsen@comcast.net> > ** > ** Filename: dpi_pipes_nobuflimit_sysc.cxx > ** > ** Created: Tue Mar 21 21:19:19 EST 2006 > ** > ** Note: This code is an example of DPI blocking pipes implementation > ** that allows messages to be any size, i.e., not limited by the > ** buffer size in the underlying non-blocking calls. > ** > ** $Id$ > ** > */ > > #include "svdpi.h" > #include "dpi_pipes.h" > > // NOTE: This is a newly proposed function. It is not yet included in > // dpi_pipes.h. > extern "C" int dpi_pipe_c_get_depth(void *pipe_handle); > > // > // dpi_pipe_c_send_nl() is based on John Stickley's version. The comment > // below is John's. > // > // This is the no limit blocking send function for the C endpoint of a > // transaction input pipe. It handles messages of any size regardless > // of the pipe buffer depth setting. Messages larger than the pipe > // buffer can handle will be segmented into chunks that are less than > // or equal to the pipe buffer size. Each of these segments will be > // sent with the dpi_pipe_c_send() function. > // > // This code shows how this function can be implemented at the application > // layer assuming we adopt the dpi_pipe_c_get_depth() function. Even though > // this is possible this does not necessarily imply that it is not desirable > // to include this functionality in the interface. > // > void > dpi_pipe_c_send_nl(void *pipe_handle, // in: pipe handle > int bytes_per_element, // in: #bytes/element > int num_elements, // in: #elems to write > const svBitVecVal *data, // in: data > svBit eom) // in: end-of-message > { > unsigned pipeDepth = dpi_pipe_c_get_depth(pipe_handle); > unsigned elementsRemaining = bytes_per_element; > > if (elementsRemaining <= pipeDepth) > { > // If transfer size is less than pipe depth > // dpi_pipe_c_send() can handle it directly. > dpi_pipe_c_send(pipe_handle, bytes_per_element, num_elements, data, eom); > } > else if (pipeDepth >= 4) > { > // If pipe depth is greater than or equal to 4 elements we can > // always come up with a transfer size that is an integer multiple > // of 4 elements. Integer multiples of 4 elements have the property > // that they will always consist of an integer number of svBitVecVal > // words, i.e., one can trivially get a pointer to the next segment of > // data to be transferred. > const svBitVecVal *dataP = data; > unsigned pipeDepthAdj = pipeDepth & ~0x3; // Round down to nearest > // multiple of 4. > unsigned bitVecValWordsPerSegment = (pipeDepthAdj / 4) * > bytes_per_element; > > // Transfer segments of the message until it is complete. Each > // segment is less than or equal to the pipe depth in size. Hence > // dpi_pipe_c_send() can handle each segment directly. > while (elementsRemaining > 0) > { > int elemsToTransfer = elementsRemaining < pipeDepthAdj ? > elementsRemaining : pipeDepthAdj; > int lastSegment = elementsRemaining < pipeDepthAdj ? 1 : 0; > > // Transfer segment. Take care to ensure the eom flag is only > // valid on the last transfer as it is tied to the last element > // of the message. > dpi_pipe_c_send(pipe_handle, bytes_per_element, elemsToTransfer, dataP, > lastSegment ? eom : 0); > > // Advance data pointer to the start of the next segment. > dataP += bitVecValWordsPerSegment; > elementsRemaining -= elemsToTransfer; > } > } > else > { > // This case is nasty. Since the pipe depth is less than 4 > // elements we cannoot guarantee that we can find a segment size > // that consists of an integer multiple of svBitVecVal words. We > // transfer segments of maximum size, i.e., pipe depth. The data > // is copied to a locally allocated buffer and realigned using the > // svGetPartSelBit() function. > > // Bytes per segment. > unsigned bytesPerSegment = pipeDepth * bytes_per_element; > > // Number of svBitVecVal words per segment. Note that we round up > // here as the last svBitVecVal word may be fractional. > unsigned bitVecValWordsPerSegment = > (bytesPerSegment + sizeof(svBitVecVal)) / sizeof(svBitVecVal); > unsigned bytesInLastWord = bytesPerSegment % sizeof(svBitVecVal); > > // Allocate buffer for segment data. Note in a real > // implementation this should be done somewhere else as part of > // initialization rather than on the fly. One good place to do > // this is in the data structures stored off of the pipe handle. > svBitVecVal *segmentData = new svBitVecVal[bitVecValWordsPerSegment]; > svBitVecVal *segmentDataP = segmentData; > > // Current bit offset in data buffer. Used by svGetPartSelBit() > // function. > unsigned bitOffset = 0; > > // Transfer segments of the message until it is complete. Each > // segment is less than or equal to the pipe depth in size. Hence > // dpi_pipe_c_send() can handle each segment directly. > while (elementsRemaining > 0) > { > int elemsToTransfer = elementsRemaining < pipeDepth ? > elementsRemaining : pipeDepth; > int lastSegment = elementsRemaining < pipeDepth ? 1 : 0; > unsigned i; > > // Recalculate bitVecValWordsPerSegment and bytesInLastWord for > // the last segment. Since this is the last segment and the > // loop will exit after this, we do not need to retain the > // original values of these variables. > if (elemsToTransfer < (int) pipeDepth) > { > bytesPerSegment = elemsToTransfer * bytes_per_element; > bitVecValWordsPerSegment = > (bytesPerSegment + sizeof(svBitVecVal)) / sizeof(svBitVecVal); > bytesInLastWord = bytesPerSegment % sizeof(svBitVecVal); > } > > // Copy all words of segment data except the last one which may > // be partial. > for (i = 0; > i < bitVecValWordsPerSegment - 1; > i++, bitOffset += 8 * sizeof(svBitVecVal)) > { > svGetPartselBit(segmentDataP++, data, bitOffset, > 8 * sizeof (svBitVecVal)); > } > > // Handle last word of segment data here. > svGetPartselBit(segmentDataP, data, bitOffset, 8 * bytesInLastWord); > > // Transfer segment. Take care to ensure the eom flag is only > // valid on the last transfer as it is tied to the last element > // of the message. > dpi_pipe_c_send(pipe_handle, bytes_per_element, elemsToTransfer, > segmentData, lastSegment ? eom : 0); > > // Keep track of elements transferred. > elementsRemaining -= elemsToTransfer; > } > > delete [] segmentData; > } > } /* dpi_pipe_c_send_nl */ > > /* > ** Local Variables: > ** tab-width: 8 > ** End: > */ > > /* > ** End of file dpi_pipes_nobuflimit_sysc.cxx. > */ -- This email may contain material that is confidential, privileged and/or attorney work product for the sole use of the intended recipient. Any review, reliance or distribution by others or forwarding without express permission /\ is strictly prohibited. If you are /\ | \ not the intended recipient please | \ / | contact the sender and delete / \ \ all copies. /\_/ K2 \_ \_ ______________________________/\/ \ \ John Stickley \ \ \ Mgr., Acceleration Methodologies \ \________________ Mentor Graphics - MED \_ 17 E. Cedar Place \ john_stickley@mentor.com Ramsey, NJ 07446 \ Phone: (201) 818-2585 ________________________________________________________________ //=========================================================================== // @(#) $Id: dpi_pipes_sysc.cxx,v 1.1 2006/02/21 04:20:06 jstickle Exp $ //=========================================================================== //--------------------------------------------------------------------------- // Mentor Graphics, Corp. // // (C) Copyright, Mentor Graphics, Corp. 2003-2006 // All Rights Reserved // Licensed Materials - Property of Mentor Graphics, Corp. // // No part of this file may be reproduced, stored in a retrieval system, // or transmitted in any form or by any means --- electronic, mechanical, // photocopying, recording, or otherwise --- without prior written permission // of Mentor Graphics, Corp. // // WARRANTY: // Use all material in this file at your own risk. Mentor Graphics, Corp. // makes no claims about any material contained in this file. //--------------------------------------------------------------------------- #include <vector> #include <systemc.h> #include "dpi_pipes.h" #include "svdpi.h" //--------------------------------------------------------------------------- // notify_ok_to_send_or_receive() johnS 2-12-06 // // This is a callback function that notifies the application that there // is room for at least 1 data element in an input pipe or least one data // element in an output pipe. // // This callback function assumes it has been given a context object // that is an sc_event that can be directly posted to. //--------------------------------------------------------------------------- static void notify_ok_to_send_or_receive( void *context ){ // input: notify context sc_event *me = (sc_event *)context; me->notify(); } //--------------------------------------------------------------------------- // dpi_pipe_c_send() johnS 1-26-06 // // This is the basic blocking send function for the C endpoint of a // transaction input pipe. It first calls the non-blocking send function // (dpi_pipe_c_try_send()). If it succeeds in sending on the first try, // it returns happily. // // Otherwise continues to wait on an sc_event until there is at least the // required number of data elements in the pipe at which point the call // to dpi_pipe_c_try_send() will succeed. //--------------------------------------------------------------------------- void dpi_pipe_c_send_to_fit( void *pipe_handle, // input: pipe handle int bytes_per_element, // input: #bits/element int num_elements, // input: #elements to be written const svBitVecVal *data, // input: data svBit eom ) // input: end-of-message marker flag { if( !dpi_pipe_c_try_send( pipe_handle, bytes_per_element, num_elements, data, eom ) ) { sc_event *ok_to_send = (sc_event *)dpi_pipe_get_notify_context( pipe_handle ); // if( notify ok_to_send context has not yet been set up ) ... if( ok_to_send == NULL ){ ok_to_send = new sc_event; dpi_pipe_set_notify_callback( pipe_handle, notify_ok_to_send_or_receive, ok_to_send ); } while( !dpi_pipe_c_try_send( pipe_handle, bytes_per_element, num_elements, data, eom ) ) wait( *ok_to_send ); } } void dpi_pipe_c_send( void *pipe_handle, // input: pipe handle int bytes_per_element, // input: #bits/element int num_elements, // input: #elements to be written const svBitVecVal *data, // input: data svBit eom ) // input: end-of-message marker flag { // dpi_pipe_c_get_depth() will always return a buffer size of at // least one element of size bytes_per_element. int pipe_depth = dpi_pipe_c_get_depth( pipe_handle, bytes_per_element ); // Operate more efficiently on whole words if bytes_per_element is a // multiple of 4 bytes. if( (bytes_per_element&3) == 0 ){ int elements_remaining = num_elements; while( elements_remaining ){ num_elements = elements_remaining > pipe_depth ? pipe_depth : elements_remaining; dpi_pipe_c_send_to_fit( pipe_handle, bytes_per_element, num_elements, data, eom ); elements_remaining -= num_elements; data += num_elements/4; } } // Otherwise use less efficient copying at byte granularity using // SV DPI helper functions. else { int bytes_per_buffer = pipe_depth * bytes_per_element; int bytes_remaining = num_elements * bytes_per_element; int i, num_bytes; int byte_index = 0; // Create a holder for a buffer's worth of elements rounded up // to the next word. This guarantees proper alignment of data // for each call to dpi_pipe_c_send_to_fit(). svBitVecVal *holder = new svBitVecVal[ (bytes_per_buffer-1)/4+1 ]; while( bytes_remaining ){ num_bytes = bytes_remaining > bytes_per_buffer ? bytes_per_buffer : bytes_remaining; for( i=0; i<num_bytes; i++ ){ svBitVecVal byte; svGetPartselBit( &byte, data, byte_index*8, 8 ); svPutPartselBit( holder, byte, i*8, 8 ); } dpi_pipe_c_send_to_fit( pipe_handle, bytes_per_element, num_bytes/bytes_per_element, holder, eom ); bytes_remaining -= num_bytes; byte_index += num_bytes; } delete [] holder; } } //--------------------------------------------------------------------------- // dpi_pipe_c_receive() johnS 1-26-06 // // This is the basic blocking receive function for the C endpoint of a // transaction output pipe. It first calls the non-blocking receive function // (dpi_pipe_c_try_receive()). If it succeeds in receiving on the first try, // it returns happily. // // Otherwise continues to wait on an sc_event until there is at least the // required number of data elements in the pipe at which point the call // to dpi_pipe_c_try_receive() will succeed. //--------------------------------------------------------------------------- void dpi_pipe_c_receive( void *pipe_handle, // input: pipe handle int bytes_per_element, // input: #bits/element int num_elements, // input: #elements to be read int *num_elements_read, // output: #elements actually read svBitVecVal *data, // output: data svBit *eom ) // output: end-of-message marker flag { if( !dpi_pipe_c_try_receive( pipe_handle, bytes_per_element, num_elements, num_elements_read, data, eom ) ) { sc_event *ok_to_receive = (sc_event *)dpi_pipe_get_notify_context( pipe_handle ); // if( notify ok_to_receive context has not yet been set up ) ... if( ok_to_receive == NULL ){ ok_to_receive = new sc_event; dpi_pipe_set_notify_callback( pipe_handle, notify_ok_to_send_or_receive, ok_to_receive ); } while( !dpi_pipe_c_try_receive( pipe_handle, bytes_per_element, num_elements, num_elements_read, data, eom ) ) wait( *ok_to_receive ); } } //--------------------------------------------------------------------------- // dpi_pipe_c_flush() johnS 1-26-06 //--------------------------------------------------------------------------- void dpi_pipe_c_flush( void *pipe_handle ) // input: pipe handle { sc_event *ok_to_send = (sc_event *)dpi_pipe_get_notify_context( pipe_handle ); if( ok_to_send == NULL ) return; while( !dpi_pipe_c_try_flush(pipe_handle) ) wait( *ok_to_send ); }Received on Thu Mar 30 20:48:04 2006
This archive was generated by hypermail 2.1.8 : Thu Mar 30 2006 - 20:48:06 PST