LABIOS

a label-based i/o system for hpc and big data storage

labios (label-based i/o system) is a groundbreaking distributed storage framework that introduces a novel data labeling paradigm to address the challenges of modern high-performance computing (hpc) and big data storage systems. as scientific applications become increasingly data-intensive and storage requirements grow more complex, labios provides a unified solution that enables efficient data access, storage elasticity, and computational storage capabilities.

what makes labios special? 💡

labios revolutionizes storage access through its innovative label-based paradigm. just as a shipping label contains all the necessary information for package delivery, labios data labels encapsulate both the data and instructions for its handling. this approach enables:

  • asynchronous i/o operations
  • dynamic storage resource allocation
  • intelligent data placement and movement
  • computational storage capabilities
  • seamless integration of different storage backends

behind the innovation

labios emerged from the recognition that modern applications require a more flexible and efficient approach to data storage and access. funded by the national science foundation, the project aims to create a unified storage system that can support diverse and often conflicting i/o workloads under a single scalable platform.

key innovations

  • label-based abstraction: novel data representation that bundles operations with data
  • decoupled architecture: separation of data, metadata, and instruction paths
  • storage elasticity: dynamic resource provisioning based on workload characteristics
  • active storage: support for in-situ and in-transit data analytics
  • high performance: significant speedups for real scientific applications
  • transparency: no modification needed to existing applications

real-world impact 🌍

labios is making significant contributions across various scientific domains:

  • scientific computing: supporting applications like cm1 with efficient i/o operations
  • data analytics: enabling seamless integration of analysis with computation
  • deep learning: optimizing data movement for training and inference workflows
  • workflow management: improving data sharing between workflow stages

technical architecture

labios consists of several key components working together:

  • label manager: creates and manages data labels
  • distributed queue: handles label scheduling and distribution
  • worker pool: executes labels independently
  • storage bridge: connects to external storage systems
  • warehouse: manages intermediate data storage

looking forward

labios continues to evolve with exciting developments in:

  • enhanced support for computational storage
  • advanced label scheduling algorithms
  • deeper integration with workflow systems
  • extended support for diverse storage backends

join the labios community 🤝

labios is an open-source project welcoming contributions from both academic and industrial researchers:

  • repository: github - labios project
  • documentation: comprehensive guides and technical details
  • research papers: latest findings and technical innovations

key publications 📚

  1. labios: a distributed label-based i/o system (hpdc 2019)
    • foundational paper introducing the labios framework
    • doi: https://doi.org/10.1145/3307681.3325405
  2. bridging storage semantics using data labels and asynchronous i/o (acm trans. storage 2020)
    • explores semantic integration and asynchronous operations
    • doi: https://doi.org/10.1145/3415579
  3. quantifying the overheads of the modern linux i/o stack (sc 2020)
    • analysis of i/o stack performance implications
    • in proceedings of sc’20
  4. labstor: a modular and extensible platform for high-performance i/o stacks (sc 2022)
    • introduces the labstor platform for customized i/o stacks
    • in proceedings of sc’22
  5. an evaluation of daos for simulation and deep learning hpc workloads (sigops 2024)
    • investigates storage system performance for modern workloads
    • doi: https://doi.org/10.1145/3689051.3689058

acknowledgements 🙏

the development of labios has been made possible through the support of the national science foundation (nsf). we’re grateful to our collaborators at the illinois institute of technology, sandia national laboratories, lawrence livermore national laboratory, and various research institutions whose expertise has been instrumental in advancing this project.

Interested in learning more about LABIOS or discussing potential collaborations? Feel free to reach out!

References