The Myth of Latency

Frequently encountered comments:
“Throughput is meaningless. All that matters is latency”.
“How can I sell a product to a customer if I cannot insure a response time”?

Boolean Logic
Since Boolean logic is not logically determined and glitches it requires an external referent of completeness. At the circuit level this is the clock tic and all Boolean combinational circuits must propagate and stabilize well with the clock period. At this level, latency (critical path delay) is critical.

The state machine up
At the next level, however, the same designer builds a state machine with data conditional branches and data conditional iteration loops. A state machine for a variable length decoder might have several loops that may perform 1 to 16 iterations and conditional branches that may by bypass the loops altogether. A single transaction (video frame for instance) of the decoder may involve tens of thousands of decodes each taking 2 to 300 clock tics each with an average of 50 clock tics. Would the prudent engineer design the hardware fast enough to guarantee the a video frame is never missed. Or would he give himself a margin and target an average throughput of 100 clock tics which might be sufficient throughput to deliver 99.999% clean video frames and save cost by designing a system that is 1/3 as fast as it needs to be to guarantee absolute 100% latency response.

Modern processors have numerous wait states, speculative branches that cause pipeline flushes and memory caches. Programs run on these processors with data conditional branches and iteration loops which are controlled by multi tasking operating systems. Yet, with all this uncertainty of latency through multiple levels of average case throughput behavior we manage to build life critical systems such as airplane controllers and space shuttle controllers that deliver latency critical performance.

All NCL does is introduce another level of average case throughput behavior and remove the one level of absolute latency behavior which exists only because Boolean logic cannot work without an absolute latency referent. NCL can and does.