Defragmentation

In the context of administering computer systems, defragmentation (or defragging) is a process that eliminates fragmentation in file systems. It does this by physically reorganizing the contents of the disk in order to store the pieces of each file close together and in order (contiguously). It also attempts to create large regions of free space using compaction, to impede the return of fragmentation.

Aims of defragmentation

Reading and writing data on a heavily fragmented hard drive is slowed down as the time for the heads to move between fragments on the disk surface can be substantial. The disk operates at speeds millions of times slower than the CPU; thus the desire to process more efficiently encourages defragmentation. Operating system manufacturers often recommend periodic defragmentation in order to keep hard drive access as fast as possible.

Fragmented data also spreads over more of the disk than it needs to. Thus one may defragment in order to compact data storage before splitting a single partition into two or more partitions (for example, with GNU Parted, or PartitionMagic).

Causes and cures

Fragmentation occurs when the operating system cannot or will not allocate enough contiguous space to store a complete file as a unit, but instead puts parts of it in gaps between other files (usually those gaps exist because they formerly held a file that the operating system has subsequently deleted or because the operating system allocated excess space for the file in the first place). As advances in technology bring larger disk drives, the performance loss due to fragmentation squares with each doubling of the size of the drive. Larger files and greater numbers of files also contribute to fragmentation and consequent performance loss. Defragmentation restores a drive to its original speed. It also moves infrequently used files further from the directory area.

A defragmentation program must move files around within the free space available in order to undo fragmentation. This is a memory intensive operation and cannot be performed on a file system with no free space. The reorganization involved in defragmentation does not change logical location of the files (defined as their location within the directory structure).

Another common strategy to optimize defragmentation and to reduce the impact of fragmentation is to partition the hard disk(s) in a way that separates portions of the file system that experience much more reads than writes from the more volatile zones where files are created and deleted frequently. In Microsoft Windows, the contents of directories such as "\Program Files" or "\Windows" are modified far less frequently than they are read. The directories that contain the users' profiles are modified constantly (especially with the Temp directory and Internet Explorer cache creating thousands of files that are deleted in a few days). If files from user profiles were held on a dedicated partiton (as is commonly done on UNIX systems), the defragmenter runs better since it does not need to deal with all the static files from other directories. For partitions with relatively little write activity, defragmentation performance greatly improves after the first defragmentation, since the defragmenter will need to defrag only a small number of new files in the future. Relocating Windows user data onto a dedicated partition is not a trivial task even for experienced users.

Defragmentation issues

The presence of immovable system files (or of files that the defragmenter will not move in order to simplify its task), especially a swap file, can impede defragmentation. ntfsresize can safely move these files in order to resize an NTFS partition.

Certain file systems exhibit a greater susceptibility to fragmentation than others, for example, a FAT file system becomes fragmented much more quickly than NTFS. Many file systems on Unix-like platforms do not require defragmentation at all. These systems attempt to keep fragmentation below a certain point so defragmenting is not necessary. This fragmentation resistance works well as long as the file system has a fairly large amount of space free.

On systems without fragmentation resistance, fragmentation builds upon itself when left unhandled, so periodic defragmentation is necessary to keep disk performance at peak and avoid the excess overhead of less frequent defragmentation.

Utilities

Defragmentation programs often come bundled with an operating system (although Windows NT 4 notably did not include one).

Perhaps the best-known defragmentation utility used to be the MS-DOS and Windows program DEFRAG. Modern versions of Windows (Windows 2000 and later) ship with a basic edition of Diskeeper, which is a tremendous improvement over the old Defrag despite its limitations, found on the All Programs -> Accessories -> System Tools folder of the Start Menu or Computer Management MMC snap-in.

Other commercial defragmentation utilities: PerfectDisk, O&O Defrag and mst Defrag.

Filesystems

FAT comes with a defragmentation utility called Defrag.
ext2 uses an offline defragmenter called e2defrag.
ext3 has no defragmentation tool, but can be temporarily downgraded to ext2.
ReiserFS has no existing defragmentation tool.
JFS provides the defragfs utility. (IBM operating systems only)
HFS Plus in 1998 introduced a number of optimisations to the allocation algorithms in an attempt to defragment files automatically without requiring an external defragmenter.
XFS provides an online defragmentation utility called "xfs_fsr" (although it is largely unnecessary).

References

Jensen, Craig (1994). Fragmentation: The Condition, the Cause, the Cure. Executive Software International. ISBN 0-9640049-0-9.