Vidya: Performing code-block I/O characterization for data access optimization
Proceedings of the 25th International Conference on High Performance Computing · IEEE
Abstract
Understanding, characterizing and tuning scientific applications’ I/O behavior is an increasingly complicated process in HPC systems. Existing tools use either offline profiling or online analysis to get insights into the applications’ I/O patterns. However, there is lack of a clear formula to characterize applications’ I/O. Moreover, these tools are application specific and do not account for multi-tenant systems. This paper presents Vidya, an I/O profiling framework which can predict application’s I/O intensity using a new formula called Code-Block I/O Characterization (CIOC). Using CIOC, developers and system architects can tune an application’s I/O behavior and better match the underlying storage system to maximize performance. Evaluation results show that Vidya can predict an application’s I/O intensity with a variance of 0.05%. Vidya can profile applications with a high accuracy of 98% while reducing profiling time by 9x. We further show how Vidya can optimize an application’s I/O time by 3.7x.
Connected work
Citation
@inproceedings{devarajan2018vidya,
author = {Devarajan, Hariharan and Kougkas, Anthony and Challa, Prajwal and Sun, Xian-He},
booktitle = {Proceedings of the 25th International Conference on High Performance Computing},
title = {Vidya: Performing code-block I/O characterization for data access optimization},
year = {2018},
month = dec,
publisher = {IEEE},
volume = {},
number = {},
pages = {255--264},
keywords = {I/O Profiling, Data Management in HPC, I/O Performance Optimization, Task-Based I/O},
doi = {10.1109/HiPC.2018.00036},
url = {https://ieeexplore.ieee.org/abstract/document/8638067},
}