veyepar: XFS: Recent and Future Adventures in Filesystem Scalability

Author(s):	Dave Chinner
Location	C001
Date	jan Wed 18	Days Raw Files
Start	11:30	First Raw Start	error-in-template
Duration	0:50:00	Offset	None
End	12:20	Last Raw End
Chapters
Total cuts_time	None min.

http://lca2012.linux.org.au/schedule/85/view_talk
raw-playlist raw-mp4-playlist encoded-files-playlist mp4 svg png

assets release.pdf XFS_Recent_and_Future_Adventures_in_Filesystem_Scalability.json
logs
Admin: episode episode list cut list raw files day marks day marks day image_files

State:

Locked:

clear this to unlock

Locked by:

user/process that locked.

Start:

initially scheduled time from master, adjusted to match reality

Duration:

length in hh:mm:ss

Name:

Video Title (shows in video search results)

Emails:

email(s) of the presenter(s)

Released:

has someone authorised pubication

Normalise:

Channelcopy:

m=mono, 01=copy left to right, 10=right to left, 00=ignore.

Thumbnail:

filename.png

Description:

markdown

Filesystems are being asked to scale to larger configurations every week. They need to store more files, larger amounts of data and be able to index that data more efficiently than ever. XFS has had a number of pain points for managing large numbers of files and complex metadata structures that limit it's ability to scale out to the capabilities of it's underlying structures. 
                                                                                 
The first part of this presentation describes the underlying reasons for these limitations and how they have been solved over the past 18 months. Through the use of pretty graphs, shiny pictures and puppies, it will be demonstrated that the work of the past 18 months has resulted in XFS having the most scalable and highest performing metadata subsystem of the current crop of Linux filesystems.  
                                                                                 
The second part of this presentation will address the challenges we are facing over the next 18 months. These challenges are focused around reliability and robustness and having confidence in a filesystem indexing petabytes of storage.  
                                                                                 
The problems being solved will be described, along with the solutions being designed, prototyped and implemented that will allow robust deployments of petabyte scale XFS filesystems. Topics being discussed involve improving error detection and handling, use of metadata CRC codes, reverse mappings for efficient lost object recovery and more. More details of some of these topics can be found here:                                                               
                                                                                 
http://xfs.org/index.php/Reliable_Detection_and_Repair_of_Metadata_Corruption    
                                                                                 
In the course of the presentation, it will become obvious why ext4 will not catch up to or replace XFS and why btrfs complements rather than competes with XFS. In other words, the presentation will show that XFS is still the filesystem of choice for large scale storage deployments on Linux and will continue to be so for the indefinite future.

The presentation is moderately technical, but most users and administrators of XFS should have no trouble understanding the content. There is, however, sufficient technical depth in the presentation that developers should still find it interesting.

Comment:

production notes