6533b82afe1ef96bd128cb56

RESEARCH PRODUCT

MERCURY: A Transparent Guided I/O Framework for High Performance I/O Stacks

James MorseFederico PaduaTim SubGiuseppe CongiuMatthias GrawinkelAndré Brinkmann

subject

File systemPOSIXComputer scienceScalabilityNon-blocking I/OOperating systemNetwork File SystemAsynchronous I/OLinux kernelLustre (file system)computer.software_genrecomputer

description

The performance gap between processors and I/O represents a serious scalability limitation for applications running on computing clusters. Parallel file systems often provide mechanisms that allow programmers to disclose their I/O pattern knowledge to the lower layers of the I/O stack through a hints API. This information can be used by the file system to boost the application performance. Unfortunately, programmers rarely make use of these features, missing the opportunity to exploit the full potential of the storage system. In this paper we propose MERCURY, a transparent guided I/O framework able to optimize file I/O patterns in scientific applications, allowing users to control the I/O behavior of applications without modifications. This is done by exploiting the hints API provided by the back-end file system to guide data prefetching. MERCURY effeciently converts numerous small read requests into a few larger requests. Furthermore, it increases the I/O bandwidth, reduces the number of I/O requests, and ultimately the application running time. Moreover, we also propose a Linux kernel modification that allows network file systems, specifically Lustre, to work with our guided I/O framework through the posix_fadvise interface.

yearjournalcountryeditionlanguage
2017-01-012017 25th Euromicro International Conference on Parallel, Distributed and Network-based Processing (PDP)
https://doi.org/10.1109/pdp.2017.83