akougkas.io

Enosis: Bridging the Semantic Gap between File-based and Object-based Data Models

Anthony Kougkas, Hariharan Devarajan, Xian-He Sun

The ACM SIGHPC 8th International Workshop on Data-Intensive Computing in the Clouds (DataCloud 2017), in conjunction with SC’17, Denver, CO, USAACM

Abstract

File and block storage are well-defined concepts in computing and have been used as common components of computer systems for decades. Big data has led to new types of storage. The predominant data model in cloud storage is the object-based storage and it is highly successful. Object stores follow a simpler API with get() and put() operations to interact with the data. A wide variety of data analysis software have been developed around objects using their APIs. However, object storage and traditional file storage are designed for different purpose and for different applications. Organizations maintain file-based storage clusters and a high volume of existing data are stored in files. Moreover, many new applications need to access data from both types of storage. In this paper, we first explore the key differences between object-based and the more traditional file-based storage systems. We have designed and implemented several file-to-object mapping algorithms to bridge the semantic gap between these data models. Our evaluation shows that by achieving an efficient such mapping, our library can grant 2x-27x higher performance against a naive one-to-one mapping and with minimal overheads. Our study exposes various strengths and weaknesses of each mapping strategy and frames the extended potential of a unified data access system.

Finding

Enosis’s file-to-object mapping strategies grant 2x-27x higher performance than a naive one-to-one mapping with minimal overheads.

System

  1. completed

    IRIS

    A unified I/O layer over POSIX files and object stores, from my doctoral work.

Browse all papers on IRIS
  1. 2022
    conference

    LuxIO: Intelligent Resource Provisioning and Auto-Configuration for Storage Services

    Keith Bateman, Neeraj Rajesh, Jaime Cernuda Garcia, Luke Logan, Jie Ye, … Anthony Kougkas et al. Proceedings of the 29th International Conference on High Performance Computing, Data, and Analytics

  2. 2022
    conference

    LabStor: A modular and extensible platform for developing high-performance, customized I/O stacks in userspace

    Luke Logan, Jaime Cernuda Garcia, Jay Lofstead, Xian–He Sun, Anthony Kougkas. Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis

  3. 2022
    conference

    Stimulus: Accelerate Data Management for Scientific AI applications in HPC

    Hariharan Devarajan, Anthony Kougkas, Huihuo Zheng, Venkatram Vishwanath, Xian-He Sun. Proceedings of the 22nd IEEE International Symposium on Cluster, Cloud and Internet Computing

  4. 2021
    conference

    HFlow: A Dynamic and Elastic Multi-Layered I/O Forwarder

    Jaime Cernuda, Hariharan Devarajan, Luke Logan, Keith Bateman, Neeraj Rajesh, … Anthony Kougkas et al. Proceedings of the International Conference on Cluster Computing

Citation

@inproceedings{kougkas2017enosis,
  author = {Kougkas, A. and Devarajan, H. and Sun, X.-H.},
  title = {{Enosis: Bridging the Semantic Gap between File-based and Object-based Data Models}},
  booktitle = {The ACM SIGHPC 8th International Workshop on Data-Intensive Computing in the Clouds (DataCloud 2017), in conjunction with SC'17, Denver, CO, USA},
  year = {2017},
  month = nov,
  publisher = {ACM},
  url = {http://cs.iit.edu/%7Escs/assets/files/Enosis.pdf},
}