Subject: Re: Coverage API requirements
From: Alain Raynaud (alain@tensilica.com)
Date: Mon Aug 19 2002 - 14:36:01 PDT
Joao,
Thanks for the detailed requirements. Looks good.
I have a couple of comments/questions:
1) should we add something about merging tests? Usually, one simulation
will not get you full coverage on everything, but your set of tests
(also called regression suite) hopefully will. From an customer point of
view, I want to query the "coverage results" of the entire suite and
find the weaknesses. I guess it would make sense to standardize such
queries, including ways to manipulate the test list, determine which
subset of tests would provide for "good enough" coverage and so on. On
the other hand, this contradicts the usual view that the API is running
"live" with a simulation which is only one test at a time.
2) coverage and assertions: in the long term, I am a proponent of
functional coverage, for which I believe assertions can be of great
help. In your requirements, I would therefore emphasize point 1) even
more: not only should the API be very similar for different kinds of
coverage, but could we make it independent of the underlying coverage
metrics? This way, I could query coverage results from any coverage
tool, whether it is line coverage, FSM coverage, or some new functional
coverage metric that we didn't even think of at the time this standard
was defined.
Alain Raynaud
Tensilica, Inc.
3255-6 Scott Blvd, Santa Clara, CA 95054
This archive was generated by hypermail 2b28 : Mon Aug 19 2002 - 14:37:58 PDT