Synchronization Costs

The Splash-2 report presents the time spent in synchronization in the different programs, and when programs have huge synchronization costs, their poor scalability (or efficiency) is attributed to the synchronization costs.

However, synchronization costs could be high due to two different reasons. Sometimes a program could have poor load-balance and therefore most of the processors might spend significant amount of time waiting at a barrier. Another scenario is that the program makes heavy use of locks and barriers in a fine-grained manner thus resulting in considerable overhead and poor speedups. In one of these cases, the high synchronization cost is a symptom of load imbalance, which is the actual reason for poor program performance. In the other case, the high synchronization cost results in poor program performance. While studying applications, we should separate one situation from the other.