Differences From A Local Filesystem
Although the designers of GFS/GFS2 aimed to emulate a local filesystem closely, there are a number of differences to be aware of. Some of these are due to the existing filesystem interfaces not allowing the passing of information relating to the cluster. Some stem from the difficulty of implementing those features efficiently in a clustered manner. For example:
- The flock system call on GFS/GFS2 is not interruptible by signals.
- The fcntl F_GETLK system call returns a PID of any blocking lock. Since this is a cluster filesystem, that PID might refer to a process on any of the nodes which have the filesystem mounted. Since the purpose of this interface is to allow a signal to be sent to the blocking process, this is no longer possible.
- Leases are not supported with the lock_dlm (cluster) lock module, but they are supported when used as a local filesystem
- dnotify will work on a "same node" basis, but its use with GFS/GFS2 is not recommended
- inotify will also work on a "same node" basis, and is also not recommended (but it may become supported in the future)
- splice is supported on GFS2 only
The other main difference, and one that is shared by all similar cluster filesystems, is that the cache control mechanism, known as glocks (pronounced Gee-locks) for GFS/GFS2, has an effect across the whole cluster. Each inode on the filesystem has two glocks associated with it. One (called the iopen glock) keeps track of which processes have the inode open. The other (the inode glock) controls the cache relating to that inode. A glock has four states, UN (unlocked), SH (shared - a read lock), DF (deferred - a read lock incompatible with SH) and EX (exclusive). Each of the four modes maps directly to a DLM lock mode.
When in EX mode, an inode is allowed to cache data and metadata (which might be "dirty", i.e. waiting for write back to the filesystem). In SH mode, the inode can cache data and metadata, but it must not be dirty. In DF mode, the inode is allowed to cache metadata only, and again it must not be dirty. The DF mode is used only for direct I/O. In UN mode, the inode must not cache any metadata.
In order that operations which change an inode's data or metadata do not interfere with each other, an EX lock is used. This means that certain operations, such as create/unlink of files from the same directory and writes to the same file should be, in general, restricted to one node in the cluster. Of course, doing these operations from multiple nodes will work as expected, but due to the requirement to flush caches frequently, it will not be very efficient.
The single most frequently asked question about GFS/GFS2 performance is why the performance can be poor with email servers. The solution is to break up the mail spool into separate directories and to try to keep (so far as is possible) each node reading and writing to a private set of directories.
Read more about this topic: GFS2
Famous quotes containing the words differences and/or local:
“The mother must teach her son how to respect and follow the rules. She must teach him how to compete successfully with the other boys. And she must teach him how to find a woman to take care of him and finish the job she began of training him how to live in a family. But no matter how good a job a woman does in teaching a boy how to be a man, he knows that she is not the real thing, and so he tends to exaggerate the differences between men and women that she embodies.”
—Frank Pittman (20th century)
“To see ourselves as others see us can be eye-opening. To see others as sharing a nature with ourselves is the merest decency. But it is from the far more difficult achievement of seeing ourselves amongst others, as a local example of the forms human life has locally taken, a case among cases, a world among worlds, that the largeness of mind, without which objectivity is self- congratulation and tolerance a sham, comes.”
—Clifford Geertz (b. 1926)