Latency

A.k.a. “Delay”. The time it takes for a request to result in a response. Latency is an important Metric for measuring Performance.

Latency is also an important Metric for User Experience. I.e.: a User feels that the system is responding “instantaneous” if the system can respond to User input within 100ms. If something takes longer than 10 seconds, the User will want to go and do something different. (For more information, see User Experience).

Latency not often is taken into account when designing or implementing systems or components. But it’s crucially important. Users may develop a negative association with the product, if the system is usually slow to respond. But more importantly, the chance of systems running out of resources increases exponentially as Latency increases beyond a certain threshold. This is also called a Suicide Mechanism: if latency increases, the number of requests at any point in time increases, which requires more system resources and decreases system performance, which makes the problems worse.

Things one can do to improve Latency, and things that one Should take into account, when designing or developing systems, are:

• Limit the number of hops: a component that is invoked by another component, Should NOT, in turn, invoke yet another component. Likewise a component that interacts with a Database Should have one interaction with the database, not many.
• Execute operations Concurrently instead of Sequentially.