Google File System

Google File System (GFS) is a proprietary distributed file system based on Linux and developed by Google for their applications' use. It does not appear to be publicly available; as it builds on and does not actually incorporate any ext3 or Linux's GPL'ed code, there is no legal requirement to distribute it. "Despite having published details on technologies like the Google File System, Google has not released the software as open source and shows little interest in selling it. The only way it is available to another enterprise is in embedded form--if you buy a high-end version of the Google Search Appliance, one that is delivered as a rack of servers, you get Google's technology for managing that cluster as part of the package." "How Google Works"

Design

GFS is optimized for Google's core data storage needs, web searching, which can generate enourmous amounts of data that needs to be retained "All this analysis requires a lot of storage. Even back at Stanford, the Web document repository alone was up to 148 gigabytes, reduced to 54 gigabytes through file compression, and the total storage required, including the indexes and link database, was about 109 gigabytes. That may not sound like much today, when you can buy a Dell laptop with a 120-gigabyte hard drive, but in the late 1990s commodity PC hard drives maxed out at about 10 gigabytes." "How Google Works".; Google File System grew out of an earlier Google effort, "BigFiles", developed by Larry Page and Sergei Brin in the early days of Google, whilst it was still located in Stanford "To cope with these demands, Page and Brin developed a virtual file system that treated the hard drives on multiple computers as one big pool of storage. They called it BigFiles. Rather than save a file to a particular computer, they would save it to BigFiles, which in turn would locate an available chunk of disk space on one of the computers in the server cluster and give the file to that computer to store, while keeping track of which files were stored on which computer. This was the start of what essentially became a distributed computing software infrastructure that runs on top of Linux." "How Google Works". The data is stored persistently, in very large, even multiple gigabyte-sized files which are only extremely rarely deleted, overwritten, or shrunk; files are usually appended to or read. It is also designed and optimized to run on Google's computing clusters, the nodes of which are comprised of cheap, "commodity" computers, which means precautions must be taken against the high failure rate of individual nodes and the subsequent data loss. Other design decisions select for high data throughputs, even when that makes latency worse.

The nodes are divided into two types: Master nodes and Chunkservers. Chunkservers store the data files, with each individual file broken up into fixed size chunks (hence the name) of about 64 megabytes "The files managed by the system typically range from 100 megabytes to several gigabytes. So, to manage disk space efficiently, the GFS organizes data into 64-megabyte "chunks," which are roughly analogous to the "blocks" on a conventional file system--the smallest unit of data the system is designed to support. For comparison, a typical Linux block size is 4,096 bytes. It's the difference between making each block big enough to store a few pages of text, versus several fat shelves full of books." "How Google Works", similar to clusters or sectors in regular file systems. Each chunk is assigned a unique 64-bit label, and logical mappings of files to constituent chunks are maintained. Each chunk is replicated a fixed number of times throughout the network, the default being three, but even more for high demand files like executables.

The Master server doesn't usually store the actual chunks, but rather all the metadata associated with the chunks, such as the tables mapping the 64-bit labels to chunk locations and the files they make up, the locations of the copies of the chunks, what processes are reading or writing to a particular chunk, or taking a "snapshot" of the chunk pursuant to replicating it (usually at the instigation of the Master server, when, due to node failures, the number of copies of a chunk has fallen beneath the set number). All this metadata is kept current by the Master server periodically receiving updates from each chunk server ("Heart-beat messages").

Permissions for operations are handled by a system of time-limited, expiring "leases", where the Master server grants permission to a process for a finite period of time during which no other process will be granted permission by the Master server to access the chunk. The modified chunkserver, which is always the primary chunk holder, then propagates the changes to the chunkservers with the backup copies. The changes are not saved until all chunkservers acknowledge, thus guaranteeing the completion and atomicity of the operation.

Programs access the chunks by first querying the Master server for the locations of the desired chunks; if the chunks are not being operated on (if there are no outstanding leases), the Master replies with the locations, and the program then contacts and receives the data from the chunkserver directly (similar to Kazaa and its Supernodes).

Criticism

There can only be one Master server- the code does not allow multiple Masters. This appears to be a flaw limiting the system's scalability and reliability, since its maximum size and up-time is limited by the Master server's capability and up-time, and since it catalogs all the metadata, and since also almost all actions and requests flow through it; but Google's engineers argue that it is not, as GFS scales very well "In technical presentations, Google talks about running more than 50 GFS clusters, with thousands of servers per cluster, managing petabytes of data." "How Google Works". Metadata is very compact, mere kilobytes to the megabyte, and the Master server is typically one of the most capable nodes on the network; for reliability, there is typically a "shadow" Master server, mirroring the main Master server which steps in if the Master server fails. Also, it is rarely a bottleneck, since clients only request the metadata, and typically cache it; subsequent interactions proceed directly with the chunkservers. Similarly, using a single Master server drastically cuts down on the software complexity that would be requisite to ensure data integrity, atomicity of operations, load balancing, and security, (to name only a few issues) if there were multiple Master servers.

References

"The Google File System" (PDF), Sanjay Ghemawat, Howard Gobioff, and Shun-Tak Leung; pub. 19th ACM Symposium on Operating Systems Principles, Lake George, NY, October, 2003.

External links

Computer file systems | Google | Parallel computing | Network file systems

Google File System | Google File System

Gourt Home::Gourt :: Google File System

Design

Criticism

See also

References

External links