UNIX Filesystems: Evolution, Design, and Implementation

Sparse Files

Due to their somewhat rare usage, sparse files are often not well understood and a cause of confusion. For example, the VxFS filesystem up to version 3.5 allowed a maximum filesystem size of 1TB but a maximum file size of 2TB. How can a single file be larger than the filesystem in which it resides?

A sparse file is simply a file that contains one or more holes. This statement itself is probably the reason for the confusion. A hole is a gap within the file for which there are no allocated data blocks. For example, a file could contain a 1KB data block followed by a 1KB hole followed by another 1KB data block. The size of the file would be 3KB but there are only two blocks allocated. When reading over a hole, zeroes will be returned.

The following example shows how this works in practice. First of all, a 20MB filesystem is created and mounted:

# mkfs -F vxfs /dev/vx/rdsk/rootdg/vol2 20m
version 4 layout
40960 sectors, 20480 blocks of size 1024, log size 1024 blocks
unlimited inodes, largefiles not supported
20480 data blocks, 19384 free data blocks
1 allocation units of 32768 blocks, 32768 data blocks
last allocation unit has 20480 data blocks
# mount -F vxfs /dev/vx/dsk/rootdg/vol2 /mnt2

and the following program, which is used to create a new file, seeks to an offset of 64MB and then writes a single byte:

#include <sys/types.h> #include <fcntl.h> #include <unistd.h> #define IOSZ (1024 * 1024 *64) main() { int fd; fd = open(“/mnt2/newfile”, O_CREAT | O_WRONLY, 0666); ...