Parallel file systems have become a common component of modern high-end computers to mask the ever-increasing gap between disk data access speed and CPU computing power.

Recognizing that an I/O request will not complete until all involved file servers in the parallel file system have completed their parts, we propose a server-side I/O coordination scheme for parallel file systems.

The basic idea is to coordinate file servers to serve one application at a time in order to reduce the completion time, and in the meantime maintain the server utilization and fairness. A window-wide coordination concept is introduced to serve our purpose.

The proposed scheme which you can download on this page, is implemented under the PVFS2 file system and MPI-IO environment.

Keywords: Parallel file system, Parallel I/O, I/O synchronization, I/O optimization.

• Huaiming Song, Yanlong Yin, Xian-He Sun, Rajeev Thakur, Sam Lang. "Server-Side I/O Coordination for Parallel File Systems." Accepted to appear in the Proc. of the ACM/IEEE SuperComputing Conference (SC'11), Nov. 2011.

Download (Latest version: 0.9.1)

pfs-ioc-0.9.1.tar.gz

• MPICH2-1.4.1p1.
• PVFS2-2.8.2.

Faculty

• Xian-He Sun
• Huaiming Song

Software developer

• Huaiming Song
• Jibing Li
• Yanlong Yin

The authors are thankful to Jibing Li from Illinois Institute of Technology and Robert Ross from Argonne National Laboratory for their constructive and thoughtful suggestions toward this study. This research was supported in part by National Science Foundation under NSF grant CCF-0621435 and CCF-0937877, and in part by the Office of Advanced Scientific Computing Research, Office of Science, U.S. Department of Energy, under Contract DE-AC02- 06CH11357.