Subject: Re: Vote on Kevin's proposal
From: Stickley, John (john_stickley@mentorg.com)
Date: Fri Jan 03 2003 - 12:31:47 PST
Slight correction to previous e-mail:
> Dynamically loaded .so code:
>
> static int actual_foo_double( svHandle context, double arg ){
> return (*real_foo_double)( context, arg ); }
>
> static int actual_foo_int( svHandle context, int arg ){
> return (*real_foo_int)( context, arg ); }
>
> bootstrap(){
> svBind( (void *)&actual_foo_double, "foo_double" );
> svBind( (void *)&actual_foo_int, "foo_int" );
> }
should be,
Dynamically loaded .so code:
// Here are the real C++ foo functions:
static int foo( svHandle context, double arg ) {
fprintf(stderr,"Called (double)\n");
return arg * ((user_struct *)(svGetUserData(context))->dbl;
}
static int foo( svHandle context ,int arg ) {
fprintf(stderr,"Called (int)\n");
return arg * ((user_struct *)(svGetUserData(context))->data[0];
}
typedef int (*foo_double_type)(svContext context, double arg );
typedef int (*foo_int_type) (svContext context, int arg );
void bootstrap(){
// This assignments will pick correct variants of foo
// according to Stroustrup $r.13.3
foo_double_type actual_foo_double = &foo;
foo_int_type actual_foo_int = &foo;
svBind( (void *)actual_foo_double, "foo_double" );
svBind( (void *)actual_foo_int, "foo_int" );
}
This shows how my call foward bind scheme well even work for
overloaded C++ functions ! No statically linked C linkage
involved at all. Rather the C name in the SV declaration
is passed as a string to the svBind() function.
-- johnS
Stickley, John wrote:
> Kevin,
>
> Kevin Cameron x3251 wrote:
> >>From: "Stickley, John" <john_stickley@mentorg.com>
> >>
> >>Kevin,
> >>
> >>Kevin Cameron wrote:
> >>
> >>>>From: "Stickley, John" <john_stickley@mentorg.com>
> >>>>
> >>>>I vote "no" on accepting 1.1b in its current form
> >>>>mainly because,
> >>>>
> >>>>1. I don't think dynamic binding is needed in first version
> >>>> of the C calling interface.
> >>>
> >>>
> >>>Did you discuss that with the CVE/Seamless team?
> >>
> >>johnS:
> >>No I did not but I as I demonstrated below you can fairly
> >>easily create "function trays" that support a dynamic binding
> >>scheme even if the SV-C interface itself does not inherently
> >>support it. This scheme should work fine virtually any
> >>environment.
> >
> >
> > I'm still waiting for a full example that compiles and runs,
> > so that I can count the levels of indirection and setup your
> > approach requires.
> >
> >
> >>>
> >>>
> >>>
> >>>>2. Even if it were, a significantly simpler dynamic binding
> >>>> scheme can be realized than what Kevin proposes.
> >>>
> >>>
> >>>Not really. If you make it simpler in one place it'll get more
> >>>complicated somewhere else.
> >>>
> >>>
> >>>
> >>>>I would also like to say that I carefully went over Kevin's
> >>>>detailed examples on how the interface would be used.
> >>>>
> >>>>These examples demonstrate a complex technique for dynamically
> >>>>binding C++ functions where the code itself requires knowledge
> >>>>of the name mangling schemes of specific C++ compiler
> >>>>environments. This, in my view, is unacceptible.
> >>>
> >>>
> >>>You can take that issue to the ANSI committees. The point of
> >>>methodology in the proposal is it allows mixing libraries from
> >>>different compilers. You only need to write the name mangling
> >>>code once for any one compiler.
> >>>
> >>
> >>johnS:
> >>Still I think forcing a requirement to reverse-engineer
> >>every compiler that comes along is not a good idea.
> >>Plus you run the risk that older code libraries will
> >>break with new versions of compilers that have changed
> >>their mangling schemes. That's why I think avoiding
> >>name mangling knowledge altogether is an inherently
> >>good idea.
> >
> >
> > NB: there is no "requirement" to reverse engineer every
> > compiler, or any at all. You would only have to do it if
> > you want to build a completely generic linker for a particular
> > compiler. Vendors do not need to do it at all, only IP providers
> > or users building the shared libraries - and as your example
> > (below) points out you can get a routine handle without having
> > to generate mangled names if you know the routine in question.
> >
> > The point of my proposal is that: you can do it if you want to.
> >
> > Since the linker (locator) is part of the library you can mix
> > libraries from different compiliers and not suffer problems from
> > incompatible name mangling or C name clashes.
> >
> >
> >>>>Here are my other main objections to it:
> >>>>
> >>>>1. We don't need both a "call forward" "register locator"
> >>>> *and* a "call back" binding function. All we would need is
> >>>> a call forward binding function (i.e. svBind()).
> >>>
> >>>
> >>>You do if you want to link arbitrary C++ compilers code
> >>>dynamically.
> >>
> >>johnS:
> >>I think call forward from C would work quite nicely in
> >>a simplified dynamic binding scheme.
> >
> >
> > You need the locator if you don't have a a matching C name
> > in your library. If you only use the C name you cannot overload
> > calls the same way C++ and other languages do without generating
> > extra layers of indirection.
> >
> > If you have multiple libraries with different versions of
> > a particular routine, static linking with a single C name is
> > impossible, and using C++ is only possible if you stick with
> > one compiler.
> >
> > If SV dynamically loads a library then it could use call-forward
> > for locating functions, however, if you use static linking the
> > simulator cannot know which libraries it actually has unless the
> > libraries announce themselves somehow. The "register locator"
> > methodology avoids the simulator having to search for each library's
> > symbols in the simulation executable. The locator function is particular
> > to each library so they can't have the same C name in a static link.
> >
>
> johnS:
>
> I think you might have misunderstood what I meant by a
> call forward dynamic bind capability. In the scheme I
> had in mind, C symbols are not required.
>
> Let me see if I can elaborate a little more - here
> again, assuming dynamic binding is necessary -
> which I think is still questionable.
>
> The idea is this:
>
> 1. Use dlopen to dynamically load a C or C++ library.
>
> 2. Use dlsym to dynamically lookup the "bootstrap()"
> function of a pre-known name in that library.
>
> 3. Now the bootstrap() function would call a function that's
> a required part of the API called svBind() whose prototype
> looks something like this:
>
> void svBind( void *realFunctionPtr, const char *functionName );
>
> So, using our example below you would do something like
> this:
>
> SV declaration:
>
> extern dynamic context "foo_double" integer foo( real arg );
> extern dynamic context "foo_int" integer foo( integer arg );
>
> Dynamically loaded .so code:
>
> static int actual_foo_double( svHandle context, double arg ){
> return (*real_foo_double)( context, arg ); }
>
> static int actual_foo_int( svHandle context, int arg ){
> return (*real_foo_int)( context, arg ); }
>
> bootstrap(){
> svBind( (void *)&actual_foo_double, "foo_double" );
> svBind( (void *)&actual_foo_int, "foo_int" );
> }
>
> So, here, because I used a one way call forward dynamic bind
> function, I never actually had to have function symbols called
> "foo_double" and "foo_int" in my code. Nor did I have to have a
> "call forward" register locator *and* a "call back" binding
> function of the sort you have proposed.
>
> Rather, I'm just making a single forward call per bound
> function to inform the infrastructure what actual functions to
> use in place of the functions named in the SV declarations.
>
> This does imply that an extra "dynamic" attribute should be
> added to the SV extern declaration so that the SV compiler will
> know not to generate code that externally references the symbols.
>
> I think this scheme may be simpler to use than what
> you've proposed while still allowing true dynamic binding.
> However, I think it is still overkill for what we actually
> need.
>
> Also, in this scheme I don't show a way of querying
> argument types so that you can form mangled names as
> you've shown in your examples. However, I think a better
> way to do that would be through a VPI like abstract interface
> that lets you query the interface declarations of an
> extern'ed C function. From those query results you can form
> your compiler specific mangled names. This would be cleaner
> and more consistent than the awkward type specficiation strings
> that you've proposed.
>
> >>>
> >>>>2. The string parsing requirement for argument profiles
> >>>> is overly complicated and buys you very little. Plus
> >>>> it requires the code to have knowledge of C++ compiler
> >>>> name mangling schemes and even with this, there's no
> >>>> guarantee that the pointers to the functions being
> >>>> bound match the profiles of the extern definitions in
> >>>> the SV side. So really the type safety that it is
> >>>> attempting to create, falls well short of its goal.
> >>>
> >>>
> >>>There is a guarantee that if you have generated the type
> >>>strings properly, and the mangling is performed properly
> >>>the routine will match the profile. There is no other
> >>>proposal that offers any checking in this area.
> >>>
> >>
> >>johnS:
> >>"if you have generated the type strings properly"
> >>There's no guarantee of this ! Another proposal that
> >>offers checking in this area is compile time checking
> >>of functions against optionally generated headers.
> >>(see below).
> >
> >
> > The strings should be correct by construction, the same
> > as your header files. However there is no way of checking
> > that a user's code actually obeys the prototypes in the
> > headers. If you use the string information to generate
> > the appropriate C++ mangled name you are guaranteed that
> > the arguments match since the C++ compiler will generate
> > a (nearly) unique name for given set of arguments.
> >
> > Only C++ linking guarantees arguments match.
> >
> >
> >
> >>>>A static linking scheme that makes use of optional
> >>>>SV compiler generated headers provides superior type
> >>>>safety checking than Kevin's scheme does. The same
> >>>>overall goals of the example given by Kevin can
> >>>>alternately be accomplished using the static linking
> >>>>scheme.
> >>>
> >>>
> >>>Are mandating header generation?
> >>
> >>johnS:
> >>No, I continue to feel that this should be implementation
> >>optional. But I predict that most implementations will
> >>want to make use of it since it is easy to do.
> >>
> >>
> >>>That's not a link-time check, there is no guarantee that
> >>>the the code in the library matches correctly unless you
> >>>include all the code-generation rules.
> >>
> >>johnS:
> >>True. But this should be easy enough for users to do.
> >>They do it all the time (i.e. include header files)
> >>when using C functions from other libraries and APIs.
> >
> >
> > Like I said elsewhere, if you make it simpler in one place
> > it gets more complicated in another. I don't want to have
> > to run SystemVerilog to generate header files.
> >
> >
> >>>Only C++ combines the argument types with the C name of
> >>>the routine, so only C++ comes close to guaranteeing that
> >>>your calls match.
> >>
> >>johnS:
> >>ANSI C compilers also will check arguments of C functions
> >>against extern declarations in headers.
> >
> >
> > Not at link time, so pre-compiled libraries can be out-of-sync
> > without you knowing.
> >
> >
> >>>
> >>>>Here's how:
> >>>>
> >>>>1. In SV code declare extern functions to Kevin's
> >>>>"foo" double and "foo" int functions:
> >>>>
> >>>>SV declarations:
> >>>>----------------
> >>>>
> >>>> extern context "foo_double" integer foo( real arg );
> >>>> extern context "foo_int" integer foo( integer arg );
> >>>
> >>>
> >>>The "foo_double"/"foo_int" is extra over my proposal, the
> >>>locator works it out.
> >>
> >>johnS:
> >>Yes, a differentiator is needed in the name since
> >>the function names must have C linkage. In a sense we
> >>are requiring the user to provide explicit manual name
> >>mangling for this particular example.
> >>
> >>But, what's nice about doing it this way is that there
> >>are no pre-made assumptions about compiler name mangling
> >>schemes. Manual mangling in this example creates clearly
> >>defined names that must simply be matched on the C side.
> >>It is a simple mechanism and very easy for users to
> >>understand and use. No magic processing is needed by a
> >>locator function.
> >
> >
> > There are no assumptions about the name-mangling in my
> > scheme, it's left up to the creator of the library to
> > determine which routine gets linked, whether it ends up
> > as a call to C++, VHDL or Perl is transparent to SystemVerilog.
> >
> >
> >>>>In this case foo_double() and foo_int() are extern "C"
> >>>>linkage wrappers for the real overloaded C++ variants
> >>>>of these functions shown in Kevin's example.
> >>>>
> >>>>Now, on the C side, you can still use dlopen() and dlsym()
> >>>>to look up the functions as Kevin shows. But what you
> >>>>do is set pointers that the C wrappers call.
> >>>>
> >>>>Here you've created application level dynamic binding
> >>>>capability from static capability with very little
> >>>>added overhead and with no name mangling interpretation
> >>>>required.
> >>>>
> >>>>A better designed, more compact object oriented scheme
> >>>>could also be used but here I just used a rudimentry
> >>>>scheme to follow with the flow of Kevin's example.
> >>>>
> >>>>Optional SV compiler generated header sv_decls.h:
> >>>>-------------------------------------------------
> >>>>
> >>>> extern "C" {
> >>>> extern int foo_double( svHandle context, double arg );
> >>>> extern int foo_int ( svHandle context, int arg );
> >>>> };
> >>>>
> >>>>test.c:
> >>>>-------
> >>>>
> >>>> // Define pointers to the real C++ functions here:
> >>>> static int (*real_foo_double)( svHandle context, double arg );
> >>>> static int (*real_foo_int) ( svHandle context, int arg );
> >>>>
> >>>> // Define the wrappers here that will be directly called
> >>>> // from SV, which then call the real C++ functions.
> >>>> extern "C" {
> >>>> int foo_double( svHandle context, double arg ){
> >>>> return (*real_foo_double)( context, arg ); }
> >>>>
> >>>> int foo_int( svHandle context, int arg ){
> >>>> return (*real_foo_int)( context, arg ); }
> >>>>
> >>>> void (*bindMyFoos)( void *, void *); // "Locator" function
> pointer.
> >>>> };
> >>>>
> >>>> main(){
> >>>> // Dynamically load code library containing real functions:
> >>>> void *lib = dlopen("./liblib.so",RTLD_NOW); // Open
> dynamic library
> >>>>
> >>>> // Now look up and call "locator" function that, itself,
> >>>> // is an extern "C" function.
> >>>> bindMyFoos = dlsym(lib,"BindMyFoos");
> >>>> (*bindMyFoos)( (void *)&real_foo_double, (void
> *)&real_foo_int );
> >>>>
> >>>> ...
> >>>> }
> >>>
> >>>
> >>>Where did you tell SV to call "bindMyFoos" ?
> >>
> >>johnS:
> >>Right from the user's main() or anywhere else on the C side
> >>during the initialization phase as the example here shows.
> >
> >
> > The user wouldn't have a "main" in a real simulation, but I'm
> > sure you could stick the call in a constructor :-)
> >
> >
> >>>
> >>>>Dynamically loaded lib.cc:
> >>>>--------------------------
> >>>>
> >>>> // Here are the real C++ foo functions:
> >>>> int foo( svHandle context, double arg ) {
> >>>> fprintf(stderr,"Called (double)\n");
> >>>> return arg * ((user_struct *)(svGetUserData(context))->dbl;
> >>>> }
> >>>>
> >>>> int foo( svHandle context ,int arg ) {
> >>>> fprintf(stderr,"Called (int)\n");
> >>>> return arg * ((user_struct
> *)(svGetUserData(context))->data[0];
> >>>> }
> >>>>
> >>>> typedef int (*foo_double_type)(svContext context, double arg );
> >>>> typedef int (*foo_int_type) (svContext context, int arg );
> >>>>
> >>>> extern "C" {
> >>>> void BindMyFoos( void *real_foo_double, void *real_foo_int ){
> >>>> // This assignments will pick correct variants of foo
> >>>> // according to Stroustrup $r.13.3
> >>>
> >>>
> >>>Does that mean you didn't try it?
> >>
> >>johnS:
> >>Quite honestly I did not try it. However, since you raised the
> >>issue I created the following small program and verified that
> >>it indeed does work for at least GNU and Solaris SPRO compilers.
> >>Until now, I did not realize C++ supported this. Pretty cool !
> >>This can be a nice template for writing code to create function
> >>trays of dynamically bound function pointers.
> >>
> >>------------------------------ cut here ---------------------
> >>
> >>#include <stdio.h>
> >>#include <assert.h>
> >>
> >>typedef void *svContext;
> >>
> >>typedef int (*foo_double_type)(svContext context, double arg );
> >>typedef int (*foo_int_type) (svContext context, int arg );
> >>
> >>static foo_double_type real_foo_double;
> >>static foo_int_type real_foo_int;
> >>
> >>
> >>// Here are the real C++ foo functions:
> >>int foo( svContext context, double arg ) {
> >> printf("Called (double)\n");
> >> return (int)arg;
> >>}
> >>
> >>int foo( svContext context ,int arg ) {
> >> printf("Called (int)\n");
> >> return arg;
> >>}
> >>
> >>int main(){
> >>
> >> real_foo_double = &foo;
> >> real_foo_int = &foo;
> >>
> >> assert( (*real_foo_double)(NULL, 1.0) == 1 );
> >> assert( (*real_foo_int)(NULL, 2) == 2 );
> >>
> >> return 0;
> >>}
> >>
> >>------------------------------ cut here ---------------------
> >>
> >>
> >>>
> >>>> *((foo_double_type *)real_foo_double) = &foo;
> >>>> *( (foo_int_type *)real_foo_int) = &foo;
> >>>> }
> >>>> };
> >>>
> >
> > Thank's for spotting you can overload function assignments -
> > but it's all somewhat redundant if you gave the functions
> > different C names in SV anyway. You can just say:
> >
> > extern "C" int foo_double( svHandle context, double arg ) {
> > return foo(context,arg);
> > }
> >
> >
> >>>You don't have the data setup phase.
> >>
> >>johnS:
> >>I'm not sure what you mean by this. Please elaborate.
> >
> >
> > All your calls are not specific to the calling context, any
> > context dependent behavior has to be evaluated when the routine
> > is called.
> >
> > My scheme can interogate the context when deciding which routine
> > to use and set the context user data at that time so that calls
> > to the routine later do not have to interogate the context. In
> > my example I set the context user data differently for each call
> > when I do the linking.
> >
> > Your scheme can only bind one C routine for multiple instances
> > of a call, mine can bind a different routine for each instance.
> > Since module instances can vary substantially due to parameters
> > being different, my scheme offers performance benefits from being
> > able to move some context dependent decision making into the
> > locate/link phase.
>
> johnS:
> Any dynamic binding scheme can do this, including the one
> I described above.
> >
> >
> >
> >>>
> >>>
> >>>
> >>>>Note that this is just as efficient as Kevin's example with
> >>>>two added benefits:
> >>>>
> >>>>1. No need for knowledge of compiler specific mangling schemes
> >>>
> >>>
> >>>But the user needs to add information in the "extern" declaration
> >>>to compensate.
> >>
> >>johnS:
> >>Either the user or, more ideally, the SV compiler if it chooses
> >>to implement the optional header feature.
> >>
> >>
> >>>
> >>>>2. No dynamic binding needed in DirectC API. Can be handled
> >>>> completely at application layer if required.
> >>>
> >>>
> >>>But that didn't buy you anything, you still have no real argument
> >>>checking, and the same number of indirections. And as a downside
> >>>you are using more of the C linker name space which will lead to
> >>>more name clashes.
> >>
> >>johnS:
> >>I think in a typical application, name clashes can be kept to a
> >>minimum if good coding practices are used such as 3 letter prefixes
> >>for function names, etc. that we've all been using for years.
> >
> >
> > Why take the risk when you don't have to?
> >
> >
> >>>I suggest you redo your example with the missing bits added (how
> >>>myBindFoos gets declared to SV, and how the contexts get initialized),
> >>>and give us a version that compiles and runs like mine did.
> >>>
> >>
> >>johnS:
> >>bindMyFoos() doesn't need to get declared to SV. It is declared
> >>in and called from C. SV does not need to get involved.
> >>
> >>
> >>>Kev.
> >>
> >
> > Kev.
> >
> >
> >
> >>-- johnS
> >>
> >>
> >>>>-- johnS
> >>>>
> >>>>Warmke, Doug wrote:
> >>>>
> >>>>>Team,
> >>>>>
> >>>>>I believe today is the deadline for voting on Kevin's proposal.
> >>>>>Kevin requested an extension this week but there was no reply
> >>>>>from either Ghassan or Swapnajit.
> >>>>>
> >>>>>If there is no extension, my vote on the proposal is "no".
> >>>>>
> >>>>>I feel the complexity is too high and therefore it is not
> >>>>>in the spirit of the original DirectC requirements,
> >>>>>"convenience and performance". Also the proposal is making
> >>>>>too many assumptions regarding the rest of SV 3.1 which
> >>>>>may or may not come true.
> >>>>>
> >>>>>Thanks for all the work you did on it, Kevin.
> >>>>>I'm sorry I couldn't go with you on this one.
> >>>>>
> >>>>>Regards,
> >>>>>Doug Warmke
> >>>>
> __
> ______ | \
> ______________________/ \__ / \
> \ H Dome ___/ |
> John Stickley E | a __ ___/ / \____
> Principal Engineer l | l | \ /
> Verification Solutions Group | f | \/ ____
> Mentor Graphics Corp. - MED C \ -- / /
> 17 E. Cedar Place a \ __/ / /
> Ramsey, NJ 07446 p | / ___/
> | / /
> mailto:John_Stickley@mentor.com \ /
> Phone: (201)818-2585 \ /
> ---------
--This email may contain material that is confidential, privileged and/or attorney work product for the sole use of the intended recipient. Any review, reliance or distribution by others or forwarding without express permission is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. __ ______ | \ ______________________/ \__ / \ \ H Dome ___/ | John Stickley E | a __ ___/ / \____ Principal Engineer l | l | \ / Verification Solutions Group | f | \/ ____ Mentor Graphics Corp. - MED C \ -- / / 17 E. Cedar Place a \ __/ / / Ramsey, NJ 07446 p | / ___/ | / / mailto:John_Stickley@mentor.com \ / Phone: (201)818-2585 \ / ---------
This archive was generated by hypermail 2b28 : Fri Jan 03 2003 - 12:36:17 PST