Re: Streaming, batching, concurrent and alternating

Per,

Comments embedded ...

Bojsen, Per wrote:
> Hi Russ,
> 
>  > No. I prefer talking about deterministic vs. non-deterministic.
> Alternating
>  > is by definition deterministic, but "concurrent" can be either.
> 
> I'd like to point out that alternating execution does not guarantee
> determinism.  One example in SCE-MI 1.1 terms is when two output messages
> arrive at the same uclock.  In that case, the behavior of the test could
> depend on the order in which the output message callbacks are executed.
> This order is not defined by the standard.  You would have the same
> potential
> problem in the DPI world when two imported functions are called at the same
> simulation time.  Verilog semantics does not specify the order in which
> these
> two functions will be called.  So, even in alternating mode there is a
> burden
> on the user to ensure the deterministic behavior of his test.  I'd like to
> be able to state it is easy to do, but I've seen too many examples of people
> managing to create non-deterministic behavior in this case.

johnS:
Per you have to be very careful when deciding what determinism to
enforce here and where to draw the line.

True, Verilog naturally has race conditions. This has been a problem
for years. Two different simulators (or even the same simulator with
two different compiles) will give different results if there is
code that is written in such a way as to depend on races.

However, at the same time, there are very clear guidelines
(what we call "Cliff Cummings Guidelines") to avoid this scenario
in synchronous RTL style designs. People know how to do this
and it works.

That is what we're talking about when we speak of determinism in
the SCE-MI context. We need to insure that determinism is enforced
on the H/W side. Allowing a solution where a transaction can come
in on any non-deterministic clock cycle even if the HDL observes
Cliff Cummings Guidelines is simply not acceptable.

> 
>  > If "streaming" is implemented right, some concurrency can be achieved and
> it
>  > will still be deterministic.
> 
> Did you mean `if streaming is *defined* right' rather than *implemented*?
> In the Mentor proposal, the flush mechanism is what is required to create
> deterministic behavior at the test level, but it is up to the user to do it.
> If the user does not flush, he is open to non-deterministic behavior if he
> has
> feedback loops on the software side.

johnS:
I would say both "defined right" and "implemented right". It is very
easy to impose a requirement that the infrastructure prevents reactive
calls in the middle of a stream. But the nice thing about flush is
that it provides convenient sync points during which reactive interactions
can be made from a given thread.

This is easy to enforce at by the infrastructure and keep the user
out of trouble. It is very easy to word the standard so it requires this.

> 
>  > If a user uses those knobs and introduces the slightest indeterminism, he
> might
>  > as well just put the system in free running mode and forget about knobs.
> 
> No, I think you are overlooking the fact that although batching as Shabtay
> has described, especially on output ports, introduces the potential for
> non-determinsitc behavior, that does not mean that the test will behave
> non-deterministically.  I described one scenario where there is no feedback
> on the software side.  This would be a fully determinstic test even when
> output messages are batched.
> 
> Also, making the system free-run would drastically change the behavior of
> the testbench on the hardware side from run to run and vendor to vendor.
> All batching of output messages really do is to allow the infrastructure
> to delay the transmission of received output messages in order to optimize
> the usage of the communication bandwidth between software and hardware
> side.  The test runs will be repeatable from run to run although they may
> differ from vendor to vendor unless the test is written to avoid
> non-determinism.  You also have to this in a streaming scenario and even
> in a non-streaming/alternating mode as pointed out above.  Of course, it
> gets a lot easier in the latter case, although perhaps to the point where
> people ignore it and only realize they were not careful when they go to
> run their test on a different platform.
> 
>  > Wouldn't this be easier for a not fully reactive testbench?
> 
> No, it wouldn't because in a free-running testbench you can no longer
> control the timing of your transactions which is necessary for creating
> many test scenarios.

johnS:
We strongly disagree with this. You can create random timing by using
pseudo-random generated delays but still make it repeatable. This way, if
any failure is detected, it is repeatable with the same seed. This, in
our view is an absolute requirement.

I would challenge you to describe to me a test scenario that can only be
accomplished with non-deterministic behavior that could not just effectively
be done with pseudo-random delays that are deterministic and therefore
repeatable if bugs appear.

-- johnS
<eom>

> 
> Per
> 
> --
> Per Bojsen                                Email: <bojsen@zaiqtech.com>
> Zaiq Technologies, Inc.                   WWW:   http://www.zaiqtech.com
> 78 Dragon Ct.                             Tel:   781 721 8229
> Woburn, MA 01801                          Fax:   781 932 7488
> 

-- 

______________________________/\/            \     \
John Stickley                   \             \     \
Mgr., Acceleration Methodologies \             \________________
Mentor Graphics - MED             \_
17 E. Cedar Place                   \   john_stickley@mentor.com
Ramsey, NJ  07446                    \     Phone: (201) 818-2585
________________________________________________________________