Lecture 19: Distributed File Systems

The topics are from Chapter 9 (Distributed File Systems) in Advanced Concepts in OS

Topics for Today

• Distributed File Systems: architecture, mechanisms and design issues.

Introduction

• A resource management component in a distributed operating system.
• A common file system that can be shared by all computers in the system.
• Usually use some dedicate file servers

• Network transparency: users should not be aware of the location of files.
• High availability: system failures and regular shared activities (backups) should not result in unavailability of files.

Architecture

• file servers and file clients interconnected by a communication network.
• two most important components: name server and cache manager.
• name server: map names to stored objects (files, directories)
• cache manager: perform file caching. Can present on both servers and clients.
  • Cache on the client deals with network latency
  • Cache on the server deals with disk latency

• Check client cache, if present, return data
• Check local disk, if present, load into local cache, return data
• Send request to file server
• ...
• server checks cache, if present, load into client cache, return data
• disk read
• load into server cache
• load into client cache
• return data

Mechanisms:

• mounting: binding together different filename spaces to form a single hierarchically structured name space.
  • The mount table maintains the mapping from mount points to storage devices.
  • Can be used in distributed file systems to do name resolution. Where to mountain the mount information?
    • at the clients: (NFS) different clients may see different files
    • at the server: all clients may see the same filename space. Good when files move around the servers.
• Caching: a common technique to exploit locality and reduce delays.
• Hints: caching without cache coherence. Cache coherence operations too expensive in the distributed systems (too many communications). Hints work well for applications that can recover from invalid data (e.g. address mapping)
• Bulk Data Transfer: communication cost = S + C*B, S: startup cost, mostly software, C: per byte cost, B: number of bytes to send.
• Encryption: typical method to enforce security.

Design Issues

• Naming and name resolution
  • A name is associated with an object.
  • Name resolution: mapping a name to an object
  • Name space: a collection of names. Three different methods to name files:
    • concatenate the host name to the name of the file on that host
      • simple unique file name
      • not network transparency
      • Moving a file from one machine to another may require modifications to the application.
      • name resolution is simple -- don't need a name server
    • mount remote directories onto local directory
      • Once mounted, accessing the file becomes location transparent
      • name resolution is simple.
    • have a single global directory: all files in the system belong to a single name space
    • not scalable, how to maintain a systemwide unique filenames
  • Name server
    • name resolution: just maintaining a table that maps names to objects
    • centralized server or distributed server (applications query local name server, local name servers query remote name server -- a distributed database).
• Caches on Disk or Main Memory
  • Advantage for caching on main memory
    • Can be used in diskless workstation
    • accessing a cache in memory is much faster
    • same techniques for server cache and client cache.
  • Disadvantage
    • compete with virtual memory
    • virtual memory system more complex: data cannot be in both vm and in cache.
    • large files cannot be cached completely: need to be able to chop the file, server software more complex.
  • Advantage for caching on disk
    • large file can be cached. virtual memory is simple
• Writing policy
  • Write through -- reliable, but not cache for write
  • delayed write: delay the writes to the server
    • access locality: some date can be deleted right away -- 20-30% data are erased within 30 seconds.
    • may lose data when client crashes.
  • write on close
    • not much difference from delay write for short opens.
    • for long open: less write but more susceptible to losing data
• Cache consistency
  • server-initiated: the server inform cache managers whenever the data in the client caches become stale
  • client-initiated: the client make sure the cache is clean before returning the data to the application
  • Both server-initiated and client-initiated cache consistency schemes are expensive.
  • not allow caching when concurrent-write sharing occurs.
  • sequential-write sharing may also cause problems: a client opens a files that has recently modified and closed by another client
    • outdate blocks in the cache: timestamp the cache copy. Contacting the server can find the inconsistency.
    • update data in another client's cache not in the server (delay write): flushing writes when other clients open.
• Availability
  • replication is the main mechanism
  • how to keep replicas consistent
  • how to detect inconsistencies among replicas
  • consistency problem may decrease the availability
  • replica management: voting mechanism to read and write to replica
• Scalability
  • How to meet the demand of a growing system?
  • The biggest hurdle: consistency issue.
    • Caching: reduce network load and server load
      • server-initiated cache invalidation to maintain cache consistency: complexity increases as the system size increases.
    • exploiting the mode for file use: most shared files are read only.
• Semantics
  • What a user wants? strict consistency.
  • Users can usually tolerate a certain degree of errors in file handling -- no need to enforce strict consistency.