Toward a unified, practical model of parallel computation: Incorporating the memory hierarchy

 

Abstract

Modeling communication memory performance is difficult since it requires estimation of the interaction of access patterns (software) with the memory hierarchy implementation (hardware). Micro-architecture enhancements such as blocking, prefetching, write buffering, non-blocking accesses and associativity are present (and varied) at each level of a multi-level cache hierarchy, complicating any attempt to estimate performance. Combining characterization of the access patterns of an application with a resulting architectural model is equally challenging. Moreover, coupling any such model with the implicit network-related parameters of the LogP model of parallel computation and finding an acceptable balance between simplicity and practicality exacerbates the problem.  Applications that require out-of-core accesses suffer severely from local memory communication performance delays. Message packing and transmission to the network buffer for such codes require a significant portion of time that is not estimated in models of point-to-point communication like LogP. Architectural advances that reduce the effects of network latency through pipelined communication and DMA accesses result in local memory-related delays encompassing a greater proportion of overall communication delay for many more applications. For asynchronous transfers the effects of local communication may encompass nearly all communication cost. There is increasing need to include local memory performance in models of parallel and distributed computation. We discuss these issues and extend the LogP model of parallel computation to incorporate local memory communication performance.

Short Bio of the Speaker

Back