Discussion about Coverage Donation

Subject: Discussion about Coverage Donation
From: Michael Rohleder (michael.rohleder@motorola.com)
Date: Thu Oct 17 2002 - 04:23:33 PDT

Hi all,

this is just an attempt to inject some more discussion about the coverage donation. I am unsure whether I understood everything, so it might also be a good cross-check to identify mismatches between my believes and the reality. This is a good chance for
Joao to clarify any wrong idea or silly statement I am making here due to some misunderstanding on my side.

In general I would like to thank Joao and his team for all his effort w.r.t. this work. I think this is an important topic we should cover within SystemVerilog soon. On the other side I am unsure whether the current donation is sufficient as a basis for
further work in this direction. Most of the comments in this email are targeted for getting clarification on this.

Joao, you might want to help me [and hopefully others] to form their opinion until friday.

Hope this starts a lively discussion ...

Best regards,
Michael

A) General comments:

A1) This is probably a discussion where we should also include sv-bc/ec: This donation implicitely defines also some new SV keywords/commands ($cm_coverage, $cm_get_coverage, $cm_get_limit). Can we and how do we want to handle such additions?

A2) Based upon our experience with PLI functions, I would like to hear some opinion about the abililty to start gathering coverage information from SystemVerilog _and_ from the C/C++ side. Since it is obvious that any instrumentation for gathering coverage
information will require some effort, there will be for sure a performance drawback -- as a result coverage instrumentation will be for sure not the default.
IMHO instrumentation from the SystemVerilog side is a must. The drawback is that a different instrumentation will require to recompile it, but this is very likely anyway the case. The corresponding coverage commands must not neccesarily be part of a
specific source module and thus affect the souce code. On the other side this would permit to include the instrumentation in the compilation process, which might be important for the appropriate optimizations.
Having the capability to start/stop the instrumentation from the C side might be a problem for this; without the knowledge which part of the design is to be instrumented during the compilation this might require some of the silly needs for user hints like
learnpli ... This is something I would like to avoid under any circumstance. Has this been considered when defining this API. Any experiences here??? What are EDA vendors thinking here???

A3) I have the feeling the Coverage API is very, very focused on FSM's. Although this might be an important aspect of coverage data, other coverage aspects are also having their merits. I am trying to show some examples for this afterwards.

A4) I do not like some of the used abstractions; especially in cases where these are completely tool dependent. An example for this is the basic definition of the coverage data: you get back some number [not clearly defined how this is being generated],
which must be interpreted relative to a limit [whose definition is fully tool dependent]. Why don't we just define that the returned coverage data is the raw number of hits [and define what a hit is for a specific coverage type] and have the capability to
identify the limit in a similar manner. Otherwise this is simply too much freedom.

A5) There are some implementation details, which are in my eyes far too specific for the actual implementation. There is a lot of features I am missing and some other features I believe are really limiting the usage of this API. Three examples for this
are: a) There seem to be no functionality available to determine the blocks in a specific module or instance [like this is available for FSM's] b) cm_fsm_get_statevar() obviously limits itself to FSM's having only one variable to store the internal state
c) state values are only made of 0/1's and their size is limited to 64 bit, I don't know how a state that is reflected by multiple values (don't care bits) or how the processing of erroneous transitions is to be handled. The definitions of
cm_fsm_get_state() at least seem to imply that there is exactly one value corresponding to one state. Although this might be good coding style, it is not necessarily the reality ...

Detailed issues:
B1) CM_SOURCE:
 - Like with FSM's I would like to know which basic blocks are part of a module/instance and enumerate them as well
 - When this is possilbe, it would be great to be able to identify a particular block
 - Instead of having just the capability to identify that a block is being executed at all, I would like to be able to identify how often it is being executed. Like in the SW world, this would permit to implement coverage tools _and_ profilers on top of a
simulation.
B2) CM_CONDITION
 - this coverage type is not well enough defined. I have some guess, but this could be nearly everything others. Please clarify.
B3) CM_FSM_TRANS, CM_FSM_STATES
 - what is a _legal_ transition or a _legal_ state. How are illegal ones defined ??? Don't tell me this is again tool specific ...
B4) CM_TOOGLE
 - If I understand this coverage metric than it tells me how many net/reg bits have been toggled.
Am I really interested in this? I think for n nets there are 2^n possible toggles, so I might want to know how many of them I have covered, or whether all of the n nets are covered, or how often a particular net has toggled (POWER measurement!). All in all
these are three metrics I would like to see where there is only one available ...
B5) CM_UEXPR
 - To be honest, even after your presentation and reading this stuff three times, I still don't get the usage of this stuff. There are several useful features I can think about here, but this would be plain guessing ...

B6) CM_MODULE and CM_HIER
 - Instead of relying on the capability to distinguish between module names and instance names, I would like to have four options here CM_MODULE_INST, CM_MODULE, CM_HIER_INST and CM_HIER or something like this. What happens, if module names and instance
names are non-unique ?

B7) It might be a good idea to state that cm_api_version() needs to be checked before using any other cm_ function. We should further state what are the differences between 32/64 version and how it is intended to enable upward compatible usages in later
releases. All this is missing.

B8) CM_PARTIAL
When I receive this return value, I would like to know for which part of the design I can not get coverage information. How would this be done?

B9) there is a cm_instance_id() function to determine the proper naming of an instance, but no corresponding cm_module_id() to do the same for a module name(). This seems not to be properly balanced ...

B10) This is just a matter of coding style and consistency, but there is a return value for errors (CM_ERROR), but this is not consistently used. See cm_instance_id() and cm_fsm_id() ...

Detailed comments:

C1) There is no notion how encrypted verilog is to be handled. Is is possible to obtain coverage information? To what extend? What information is restricted and which not?

C2) Is it possible to use $root to obtain coverage information for the full design ? Or do I need to supply the full list of modules and call the coverage request for each of them?

C3) I fully understand and honor that different tools might have different methods to identify a FSM. This might be one of the things that are used by tool vendors to differentiate between themself ...
On the other side I as a user would like to have some control capability to clearly identify a FSM or prohibit the detection of a FSM in form of a user hint. And I want to have some standardized form for doing this. There is no need to clutter your source
code with comments similar to
// synopsys coverage <statevar>=isFSM
// novas cover FSM <statevar>
// cadence coverage_hint FSM <statevar> yes
// mentor cov fsm enable <statevar>
One such method is more than enough. Therefore I would like to request some standard attribute to do so.

C4) Some special issues are not clearly defined in the Coverage API. Is there some order dependency of the commands? If I want to enable gathering of data on a specific instance tree and enable this tree, but afterwards disable a subtree within this tree
(intentionally), is this what I get. This needs to be clarified.

C5) It might be a good idea to provide some capability for registering standard user callbacks upon a state transition or when evaluating net toggles or when evaluating condition coverage (if it is what I understand here). Some coverage information might
be very dependent on such user specific criteria, which could then be covered in a standardized way.
I heard some statement from Joao that this might be possible by running some PLI function within the coverage code. Hmmm, I am not sure whether this is standardized (and thus permitted by all vendors) [or whether we should even permit this ... because
there are certain implications -- see above things like learnpli ]. IMHO I would like to have a clear distinction and a clearly defined methodology to do such things. 

--

NOTE:
The content of this message may contain personal views which are not neccessarily the views of Motorola, unless specifically stated.

The information contained in this email has been classified as:
Motorola General Business Information (x)
Motorola Internal Use Only            ( )
Motorola Confidential Proprietary     ( )

         ___________________________________________________
        |                                                   |
      _ | Michael Rohleder            Tel: +49-89-92103-259 | _
     / )| Software Technologist       Fax: +49-89-92103-680 |( \
    / / | Motorola, Semiconductor Products, ASA Methodology | \ \
  _( (_ |  _   Schatzbogen 7, D-81829 Munich, Germany    _  | _) )_
 (((\ \>|_/ >                                           < \_|</ /)))
 (\\\\ \_/ /    mailto:Michael.Rohleder@motorola.com     \ \_/ ////)
  \       /_______________________________________________\       /
   \    _/                                                 \_    /
   /   /                                                     \   \

*** This note may contain Motorola Confidential Proprietary or Motorola Internal Use Only Information and is intended to be reviewed by only the individual or organization named above. If you are not the intended recipient or an authorized representative
of the intended recipient, you are hereby notified that any review, dissemination or copying of this email and its attachments, if any, or the information contained herein is prohibited. If you have received this email in error, please immediately notify
the sender by return email and delete this email from your system. Thank you! ***

This archive was generated by hypermail 2b28 : Thu Oct 17 2002 - 04:27:27 PDT