Hreplica: a dynamic data replication engine with adaptive compression for multi-tiered storage
Proceedings of the International Conference on Big Data · IEEE
Abstract
As the diversity of big data applications increases, their requirements diverge and often conflict with one other. Managing this diversity in any supercomputer or data center is a major challenge for system designers. Data replication is a popular approach to meet several of these requirements, such as low latency, read availability, durability, etc. This approach can be enhanced using new modern heterogeneous hardware and software techniques such as data compression. However, both these enhancements work in isolation to the detriment of both. In this work, we present HReplica: a dynamic data replication engine which harmoniously leverages data compression and hierarchical storage to increase the effectiveness of data replication. We have developed a novel dynamic selection algorithm that facilitates the optimal matching of replication schemes, compression libraries, and tiered storage. Our evaluation shows that HReplica can improve scientific and cloud application performance by 5.2x when compared to other state-of-the-art replication schemes.
Connected work
Citation
@inproceedings{devarajan2020hreplica,
author = {Devarajan, Hariharan and Kougkas, Anthony and Sun, Xian-He},
booktitle = {Proceedings of the International Conference on Big Data},
title = {Hreplica: a dynamic data replication engine with adaptive compression for multi-tiered storage},
year = {2020},
month = dec,
publisher = {IEEE},
volume = {},
number = {},
pages = {256--265},
keywords = {Data Replication, Data Compression Techniques, Multi-Tiered Storage Hierarchy, I/O Acceleration},
doi = {10.1109/BigData50022.2020.9378167},
url = {https://ieeexplore.ieee.org/abstract/document/9378167},
}