The following resources provide information on using, experimenting with and scaling with the Lustre® file system.
Online Tutorials
- Using the Lustre File System.
Requires (free) registration. CyberInfrastructure Tutor, CI-Tutor is hosted by the National Center for Supercomputing Applications (NCSA) at the University of Illinois at Urbana-Champaign and supported by the National Science Foundation’s Office of Cyberinfrastructure through the Extreme Science and Engineering Discovery Environment (XSEDE) project. (Last updated 12/2011)
Documentation
Information on Lustre documentation is available at the Documentation page.
Mail Lists
Join the conversation. Information on Lustre mailing lists is available here.
Reports
- I/O Characterization of Large-Scale HPC Centers — Benchmarking Working Group, OpenSFS (PDF)
The OpenSFS Benchmarking Working Group (BWG) was created with the intent of defining an I/O benchmark suite to satisfy the requirements of the scalable parallel file system users and facilities. The first step toward this end was identified as characterization of I/O workloads, from small scale to very large scale parallel file systems, deployed at various high-performance and parallel computing (HPC) facilities and institutions. The characterization will then drive the design of the I/O benchmarks that emulate these workloads.
As part of the characterization, the BWG released a survey at the Supercomputing Conference in 2012, to solicit participation and collect data on file systems and workloads in HPC centers. This paper summarizes the data collected and our analysis.
Development
Information on contributing to the Lustre file system is at the Development page.
Media
insideHPC: “Whamcloud’s Chroma Puts a Shine on Lustre Storage”
insideHPC: “Video: Architecting High Availability Lustre Storage Solution – ClusterStor 6000”
Publications
Due to the availability of the source code, Lustre is frequently used as a vehicle for file systems research. Here are a few examples:
Henschel, R., Simms, S., Hancock, D., Michael, S., Johnson, T., Heald, N., … & Stewart, C. A. (2012, November). Demonstrating lustre over a 100Gbps wide area network of 3,500 km. In Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis (p. 6). IEEE Computer Society Press.
Xie, B., Chase, J., Dillow, D., Drokin, O., Klasky, S., Oral, S., & Podhorszki, N. (2012, November). Characterizing output bottlenecks in a supercomputer. In High Performance Computing, Networking, Storage and Analysis (SC), 2012 International Conference for (pp. 1-11). IEEE.
Video
The Rich Report has many videos about Lustre including interviews and presentations from past Lustre User Group (LUG) meetings.
In this video from the DDN User Meeting at SC13, Brent Gorda from the Intel High Performance Data Division presents: “Moving Lustre Forward: Status & Roadmap.”
White Papers
Schwan, Philip. “Lustre: Building a file system for 1000-node clusters.” In Proceedings of the 2003 Linux Symposium, vol. 2003. 2003.
Wang, Feiyi, Sarp Oral, Galen Shipman, Oleg Drokin, Tom Wang, and Isaac Huang. “Understanding Lustre Filesystem Internals.” (2009).
