[sv-cc] RE: DPI-OO comments

Hi Arturo,

Please see my comments next to yours below:

From: owner-sv-cc@eda.org [mailto:owner-sv-cc@eda.org] On Behalf Of Arturo Salz
Sent: Tuesday, October 11, 2011 3:43 AM
To: SystemVerilog CC DWG
Subject: [sv-cc] DPI-OO comments

As I noted in an earlier message, I have posted a new version of the DPI-OO document with comments on the relevant sections. In this message I wanted to summarize the key issues and difficulties I see with the existing proposal.

We believe that the ability to manipulate types and objects across the C++/SV language boundary is a valuable addition to the language. And, if such a feature can be used to provide tighter interfacing between SV and other OO languages that is a good thing, which we wholeheartedly support. These are not all the issues, but the most salient, and, in my opinion, the, most in need of resolution.

- Asymmetry between C++ and SV. Generally, a proxy class is needed in both languages: a C++ proxy that handles method calls from C++ to an export SV object, and an SV proxy to handle method calls from SV to an imported C++ object. In the first case, the actual object is an SV class hence a proxy is needed in the C++ side; in the latter case, the actual object is a C++ class and the proxy is needed in the SV side. The proposal seems to only discuss C++ proxies, which makes the whole proposal very confusing.
[Vitaly] The proxies are the implementation artifact, not a part of the user view. They constitute the DPI-OO API, implemented in the object-oriented representation. An alternative could be to "flatten" the API and implement it in C. In that case there would be no use of the term "proxy" in the proposal.
Given the above, there is a need only for one language to implement the OO API and there is no need for SystemVerilog proxies. You probably got confused by the fact that C++ is the most obvious end user foreign language, and at the same time it's an implementation language. However, this coincidence of the languages has nothing to do with the design of the solution.
The end user is expected to operate with her/his own classes, not with the proxies.

- The proposal requires import proxies to be derived from class DPI-OO::ProxyT, and further states that proxies can be tool generated or written by users. Hence, users that wish to import a C++ class must write a proxy and ensure it is derived from DPI::ProxyT. It is confusing to call these proxies (as they are written by users), but the requirement that they must be derived from a common base class adds a level of complexity that become a barrier to integration of C++ classes. We believe that in order for DPI-OO to be practical and useful, it must provide first and foremost a strong and simple mechanism for interfacing existing C++ classes to SV. This proposal is somewhat skewed towards providing an interface between SV and other languages, but it significantly complicates the basic SV/C++ interface.
[Vitaly] I agree it's going to be a rather difficult task for the end users to write the proxies. That's the reason we expect that all vendors will automate this job to help the users.

- Memory management is not well specified. It is not clear from a user's perspective what guarantees the system must provide and what may lead to errors. There appears to be many unspecified behaviors that are not described.
[Vitaly] It will be helpful if you point to the specific behaviors that are not described. Please keep in mind, however, that we are developing a SystemVerilog LRM - hence it's not expected to describe the memory management aspects of other languages, including C++. Probably, separate documents to describe the C++/SystemC/e DPI-OO implementation would be helpful, but they are beyond our scope.

Likewise, it is not at all clear how an implementation manages the memory of objects that are distributed across different languages with widely different memory allocation strategies (garbage collection vs. explicit allocation). The proposal seems to imply that ProxyT maintains reference counts on a per-language basis (SV may be one of them).
[Vitaly] Not exactly. It's expected that the intermediate layer will keep the referencing between the import and export objects (not on a per-language basis). On the other side, the implementation of each proxy is specific and it depends on the export class language. The abstract class ProxyT, which is defined in Annex S, defines only the method definitions, not implementations. The implementations shall be vendor-specific.
 Presumably, the intermediate layer provides some mechanism to manage all the proxies, but it is not at all clear how this mechanism determines when an object is no longer referenced in other languages.
[Vitaly] Again, it's implementation-specific. SystemVerilog implementation is expected to decrement the reference count when an import object is garbage collected (see ProxyT::ClearReference()). Also, SystemVerilog is expected to implement ProxyT::ClearReferenceAtActual() for the SystemVerilog export class proxies. I would recommend to re-read the proposal. Those details are provided in the source code comments in Annex S.
The memory management per-language seems to add a degree of complexity that is not warranted - it seems unimportant whether a particular language has or doesn't have a reference to an object, all that matters is that no references to an import/export object remain.
[Vitaly] An import object can always be deallocated. Reference counting doesn't affect it. Deallocation of export objects shall depend on presence or absence of import objects but I don't understand why you call it memory management per language. The need for the language id originates from the fact that one export object can be referenced by few import objects (each in different language) at the same time.
 It is also unclear how a language that implements garbage collection can notify the intermediate layer that an object is no longer alive (it's garbage).
[Vitaly] We think it's completely feasible, once the language can distinguish between the native and DPI-OO classes.
All this is further complicated by the requirement of equivalence between objects and handles - it seems that to guarantee such equivalence some objects can never be deleted from the system.
[Vitaly] You are very welcome to suggest specific improvements to the spec. Currently it says (section H.9):

"An exported SystemVerilog object shall not be deallocated while a C++ proxy refers to it. The C++ proxy, in turn, should exist while at least one imported object in a foreign language refers to it.... When an imported SystemVerilog object is allocated, it shall call the C++ proxy's base class method RegisterReference(), which registers the reference from this imported object in the proxy. One proxy may have references from imported objects in multiple languages (one reference per language). .. While the imported SystemVerilog object persists, the C++ proxy (and the actual object in the foreign language) shall not be destroyed. When the SystemVerilog imported object is being garbage collected, the implementation shall call the C++ proxy's method ClearReference() to notify it that, from the SystemVerilog perspective, the proxy and the foreign actual object may be deallocated.

When a C++ proxy stops being referenced by any foreign language (that means, the last reference was removed by the ClearReference() method), the proxy is destroyed. Before being actually destroyed, the proxy shall notify the implementation in the exporting language by calling the method ClearReferenceAtActual(). This method is a pure virtual method of the base class ProxyT. Because its actual implementation depends on the implementation of the exporting language, ClearReferenceAtActual()shall be implemented for the specific C++ proxy class in the C++ implementation layer."

IMO, this is the most underspecified part of the proposal. Very late in Annex H, the proposal lists some methods in the ProxyT class, but how /when such methods are called by either side is a complete mystery to me - and to any user that would want to import their C++ class to SV.
[Vitaly] I cannot agree that Annex H is "very late". Section 35 is dedicated to the end user view. The API details are located in Annex H , exactly like the DPI-C details are located in that Annex (and the user is expected to use DPI-C). IMO the SV LRM shall contain only the specification of the API not the implementation details or implementation examples. You are welcome, of course, ask specific questions. BTW, additional information is in comments in the dpi_oo.h source code.

- The cross-language inheritance is seems to be a very ill-conceived feature. The motivation behind it seems unclear - especially since they lead to two distinct objects in the different languages and a containment strategy would provide the same functionality - and is much easier implement.
[Vitaly] The objects in different languages are distinct, according to the proposal. We are not suggesting to keep one underlying implementation, shared between different languages. That would not be feasible. Once we accept the fact that there are distinct objects we suggest that each language has its own "view" of the object. For the reference classes - the exporting language sees all interface methods of the class, its public data members and its public base classes. The importing language provides the user an import class. In that language, the user sees only the interface methods which were exported. Those interfaces may not be completely equivalent - and that's fine, IMO. Such an import "view" provides a subset of the export language interface and allows the user to add native language interfaces by deriving the class from a native language base class.

The dpi_initialize() scheme is an ugly hack for what seems to be a poorly-motivated feature and difficult for C++ users to implement.
[Vitaly] That's the best mechanism we found to enable constructors with arguments. I would like to return to discussion of the motivation (whether it's poor or not) after we complete the discussion of the inheritance model.

- The use of a common base class ProxyT seems overly intrusive and restrictive implementation. We believe this will be a big barrier to implementation by both users and implementors. All the functionality proposed for the ProxyT can be provided in a non-intrusive way via associative maps that contain the mapping between handles and pointers in all the intervening languages. The tables can be updated whenever an object crosses the language boundary - likewise the callbacks to notify the death of an object does not need to be a method but a call to the object manager. The use of associative tables is a common and effective technique to extend objects. We believe that any performance loss due to the lookup is more than offset by the ease of use provided by the separation of the two concerns: the actual class implementation and the inter-language object management and registration scheme.
[Vitaly] It looks like you are not distinguishing at this point between the intermediate and the end-user layers. Probably my comments above can help to close the gap between our and your understandings.

- The scheme for parameterized and template classes lacks scalability - requiring a post-elaboration scan of the entire SV design - and the specialization checking and error generation is not well defined. Some of the error conditions (such as missing specializations) in the C++ are generally detected by the linker or at run-time. It is not clear what are the responsibilities of a complaint SV tool.
[Vitaly] I provided some comments on this matter in the version I uploaded on the mantis item. On my side, I'll think what details shall be added concerning the error conditions and responsibilities of the SV tool.

- Copy classes can be useful but only if they are limited to the simplest types (packet or payload type of objects). The current proposal is generic and allows for copying of arbitrary data-structures, which requires detection of cycles, and may lead to high memory overhead and run-time degradation. It is also unclear whether inout copy classes are allowed, and, if they are, it is ambiguous regarding pointer/handle equivalences. Also we have a fundamental objection to copy classes being part of DPI-OO when they are in fact not an object-oriented solution - this is because it is not possible to define methods for a copy class.
[Vitaly] We are open to any improvement proposal on how to "narrow" the definition of which classes can be declared as DPI-OO copy classes. Pointer/handle equivalence requirement is unrelated to copy classes. They are copying the objects and no correspondence between them is kept. Concerning the need for the copy classes - we have a strong requirement to support efficient transaction-level methodology which de-facto already uses classes to represent big data containers. Changing the user code to call a method for setting/getting each field (or a part select of the field) does not scale.

- Inconsistent use of the "import class" and "export class" terminology that makes the text confusing to understand. It is unclear whether "importing class" is referring to a class that is imported into SV (as the term "import" is consistently used in DPI-C), or whether it is using the generic term to refer to a class that may imported into C++. Sometimes the intent can be gleaned from the text, but often it is hard to discern.
[Vitaly] I thought I already fixed all such occurrences of the inconsistent terminology based on the comments from Arnab. We are working to process all your detailed comments in the .doc file.

- There are many examples and text showing export classes (i.e., classes exported from SV to C++), but hardly any examples and descriptive text explaining import classes. Often, the normative text that applies to import classes is incomplete or just wrong.
[Vitaly] Please be more specific what examples and descriptive text are missing or wrong.

Vitaly

    Arturo

--
This message has been scanned for viruses and
dangerous content by MailScanner<http://www.mailscanner.info/>, and is
believed to be clean.
-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.