Conquest-2 File System

Conquest-2 File System
NSF grants CCR-0098363, CNS-0410908

PIs:
An-I Andy Wang , Florida State University
Peter Reiher , UCLA, The Laboratory for Advanced Systems Research

Current team members:
Charles Weddle, Cory Fox, Jin Qian, Dragan Lojpur, Florida State University
Erik Kline, UCLA, The Laboratory for Advanced Systems Research
Geoffrey H. Kuenning, Harvey Mudd College

Past contributors:
Mathew Oldham, Noriel Lu, Sean Toh, Carl Owenby, Daniel Beech, James Gonzalez,
Kelley Jones, Micah Villmow, Yiming Wu, Florida State University
RuGang Xu, Gerald Popek, UCLA

Conquest-2 is a three-year NSF-funded collaborative research grant with UCLA. In conjunction with Peter Reiher of UCLA, we are developing a file system that can simultaneously provide high file systems performance and energy efficiency. The key for reconciling these generally conflicting goals is based on the observation that RAM is becoming an inexpensive resource that can deliver all file system services with the single exception of large storage capacity. In essence, instead of storing most files on disks, Conquest-2 stores them only in RAM. Also, it stores all metadata in RAM. Since most files can then be accessed directly from RAM, there is no need to keep the disk spinning. Only large files need to be kept on disk, and such files will be accessed infrequently. While RAM running in its normal mode consumes a great deal of power, RAM can be put into an extremely power- efficient mode and returned to its normal mode much more rapidly than disk can.
Our research suggests that one can achieve better power savings than LRU caching by instead using RAM with battery backup as the primary persistent storage for most files, and providing two separate data paths to memory and disk storage. By specializing the data paths, the memory data path contains no-disk-related complexity. The disk data path consists only of optimizations for the specialized disk usage pattern.
Conquest-2 is based on the original Conquest disk/RAM hybrid file system (NSF grant CCR-0098363). Under the original Conquest, all metadata (e.g., file attributes) and all files smalled than a certain threshold are stored in RAM, while the disk stores only the data content of large files. The performance improvement is achieved by matching the user behavior with physical characteristics, and having two specialized data paths.

Conquest-2 will extend the original Conquest to handle multiple disks. Minimizing the energy use for multiple disks involves creating a bimodal distribution of active and idle disks, so we can put idle disks to sleep to conserve power. The idea is to design and implement an energy-efficient RAID that can "gear shift" disks on and off to adapt for changing loads. This energy-efficient RAID also needs to address existing load balancing and failure recovery concerns by exploting user access patterns, applying erasure encoding, and striping data in a nonuniform way.

Publications

Buzbee13
Ben Buzbee, Weisu Wang, and An-I Andy Wang. Power-Saving Approaches and Tradeoffs for Storage SystemsTechnical Report TR-130627, Department of Computer Science, Florida State University, June 2013.

Weddle07b

Charles Weddle, Mathew Oldham, Jin Qian, An-I Andy Wang, Peter Reiher, Geoff Kuenning. PARAID: A Gear-Shifting Power-Aware RAID. ACM Transactions on Storage, 3(3), October 2007 (invited).

Weddle07a

Charles Weddle, Mathew Oldham, Jin Qian, An-I Andy Wang, Peter Reiher, Geoff Kuenning. PARAID: A Gear-Shifting Power-Aware RAID. Proceedings of the 5th USENIX Conference on File and Storage Technologies (FAST), 2007.

Wang06

An-I Andy Wang, Geoff Kuenning, Peter Reiher, and Gerald Popek. The Conquest File System: Better Performance Through a Disk/Persistent-RAM Hybrid Design. ACM Transactions on Storage, 2(3), pp. 309-348, 2006.

Weddle06

Charles Weddle, Mathew Oldham, Jin Qian, An-I Andy Wang, Peter Reiher, Geoff Kuenning. PARAID: The Gear-Shifting Power-Aware RAID. Technical Report TR-060323, Department of Computer Science, Florida State University, 2006.

Weddle05

Charles Weddle. PARAID: The Gear-Shifting Power-Aware RAID, Masters Thesis, Florida State University, June 2005.

Oldham05

Mathew Oldham. A Power and Performance Measurement Framework for Server-Class Storage, Honors Thesis, Florida State University, April 2005.

Wang03b

An-I Andy Wang, Geoffrey H. Kuenning, Peter Reiher, Gerald J. Popek. The Effects of Memory-Rich Environments on File System Microbenchmarks. Proceedings of the 2003 International Symposium on Performance Evaluation of Computer and Telecommunication Systems (SPECTS), Montreal, July 2003.<http://www.cs.fsu.edu/~awang/papers/spects2003a.html>

Wang03a

An-I Andy Wang. The Conquest File System: A Disk/Persistent-RAM Hybrid Design for Better Performance and Simpler Data Paths. Ph.D. Dissertation. Computer Science Department, University of California, Los Angeles, March 2003.<http://www.cs.fsu.edu/~awang/papers/dissertation2003a.html>

Wang02b
An-I Andy Wang, Geoffrey H. Kuenning, Peter Reiher, Gerald J. Popek. Conquest: Better Performance Through a Disk/Persistent-RAM Hybrid File System. Proceedings of the 2002 USENIX Annual Technical Conference, Monterey, June 2002. <http://www.cs.fsu.edu/~awang/papers/usenix2002a.html>

Wang02a

An-I Andy Wang, Geoffrey H. Kuenning, Peter Reiher, Gerald J. Popek. Work-in-Progress Report: Conquest: Better Performance Through a Disk/Persistent-RAM Hybrid File System. On-Line Proceedings of the First USENIX Conference on File and Storage Technologies (FAST), Monterey, January 2002. <http://www.cs.fsu.edu/~awang/papers/fast2002a.html>

Wang01a

An-I Andy Wang, Geoffrey H. Kuenning, Peter Reiher, Gerald J. Popek. Position Summary: The Conquest File System--Life after Disks. Proceedings of the 8th IEEE Workshop on Hot Topics in Operating Systems (HotOS VIII), Schloss Elmau, Germany, May 2001. <http://www.cs.fsu.edu/~awang/papers/hotos2001.html>

Presentations

An-I Andy Wang. A Survey of Power-Saving Techniques for Storage Systems. Presented at the Computer Science Research Seminar, Chalmers University of Technology, Gothenburg, Sweden, May 2012.

An-I Andy Wang. PARAID: A Gear-Shifting Power-Aware RAID. Presented at the Computer Science Research Seminar, University of Wisconsin, Madison, April 2008.

An-I Andy Wang. PARAID: A Gear-Shifting Power-Aware RAID. Presented at the Computer Science Research Seminar, University of California, Santa Cruz, March 2008.

An-I Andy Wang. PARAID: A Gear-Shifting Power-Aware RAID. Presented at the Computer Science Colloquium, Harvey Mudd College, Claremont, November 2007.

An-I Andy Wang. Conquest-2: Improving Energy Efficiency and Performance Through a Disk/RAM Hybrid File System. Presented at the Computer Science Colloquium, UCLA, Los Angeles, May 2006.

An-I Andy Wang. Conquest-2: Improving Energy Efficiency and Performance Through a Disk/RAM Hybrid File System. Presented at the Computer Science Colloquium, Florida State University, Tallahassee, December 2005.

An-I Andy Wang. Conquest-2: Improving Energy Efficiency and Performance Through a Disk/RAM Hybrid File System. Presented at the Computer Science Colloquium, University of Delaware, Newark, November 2005.

An-I Andy Wang. Conquest-2: Improving Energy Efficiency and Performance Through a Disk/RAM Hybrid File System. Presented at the Computer Science Colloquium, University of California, Riverside, May 2005.

Charles Weddle. The Power-Aware RAID: PARAID. Work-in-progress-report. Presented at the 2005 USENIX Annual Technical Conference, April 2005.

Charles Weddle. The Power-Aware RAID: PARAID. Poster. Presented at the 2005 USENIX Annual Technical Conference, April 2005.

RuGang Xu. Conquest: Combining Battery-Backed RAM and Threshold-Based Storage Scheme to Conserve Power. Presented at the 19th Symposium on Operating Systems Principles (SOSP), New York, October 2003.

An-I Andy Wang. Conquest: Preparing for Life After Disks. Presented at the Computer Science Colloquium, Florida State University, Tallahassee, October 2003.

Geoffrey H. Kuenning. The Disk is Dead! Long Live the Disk!. Presented at the Computer Science Colloquium, Harvey Mudd College, Claremont, April 2003.

An-I Andy Wang. Conquest: Preparing for Life After Disks. Presented at the University of California, Los Angeles, Advanced Operating Systems Lecture, Los Angeles, October 2002.

An-I Andy Wang. Conquest: Better Performance Through a Disk/Persistent-RAM Hybrid File System. Presented at the 2002 USENIX Annual Technical Conference, Monterey, June 2002.

An-I Andy Wang. The Conquest File System: Life After Disks. Poster presented at the UCLA Computer Science Department Annual Research Review, UCLA, Los Angeles, April 2002.

An-I Andy Wang. Conquest: Better Performance Through a Disk/Persistent-RAM Hybrid File System. Presented at the First USENIX Conference on File and Storage Technologies (FAST), Monterey, January 2002.

An-I Andy Wang. Conquest: RAM as Storage; Disks as Tapes. Presented at the University of California, Los Angeles, Advanced Operating Systems Lecture, Los Angeles, November 2001.

An-I Andy Wang. The Conquest File System: Life after Disks. Poster presented at the UCLA Computer Science Department Annual Research Review, UCLA, Los Angeles, April 2001.

An-I Andy Wang. Integration of Memory and File System Services via Persistent RAM. Presented at the Computer Science Colloquium, Harvey Mudd College, Claremont, October 1999.

Press Coverage

Conquest FS: The Disk is Dead, Slashdot, April 21 2003.

Eugenia Loli-Queru. Learning About the Conquest File System, OS News, April 21 2003.

Shawn. Not Your Father's File System, hartys.net::the blog, April 21 2003.

Baltazar H. DAFS, Conquest May Pave Way for Future File Systems, eWeek, Ziff Davis Media, June 17 2002.

Conquest File System. LisoLeg: Linux Source Learning Group, 2000.

This material is based upon work supported by the National Science Foundation under Grant No. CCR-0098363 and CNS-0410896.
Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.

awang@cs.fsu.edu

Last modified: November 19, 2001