Phenquestions
Btrfs is an extent-based Copy-on-Write (CoW) filesystem. Large files are stored in multiple data extents; and when these large files are modified, the extents to be modified are copied to new, empty extents in another location of the storage device and are modified in the new location. Then, the extents of the large files are re-linked to point to the updated extents. The old extents are never removed instantly.
This is how the Copy-on-Write (CoW) feature of the Btrfs filesystem causes fragmentation. Fragmentation means that the data extents of large files are scattered around the entire storage device. They are not instantaneous. So, the performance (read/write speed) of the filesystem may be reduced.
To solve this problem, it is necessary to defragment the Btrfs filesystem every once in a while. This article shows you how to defragment the Btrfs filesystem.
Abbreviations
The abbreviations(short forms) used in this article are as follows:
- CoW - Copy-on-Write
- Btrfs - B-tree Filesystem
- HDD - Hard Disk Drive
- SSD - Solid-State Drive
- GB - Gigabyte
- VM - Virtual Machine
Prerequisites
To try out the examples included in this article:
- You must have the Btrfs filesystem installed on your computer.
- You must have a spare HDD/SSD (of any size) or at least 1 free HDD/SSD partition (of any size).
I have a 20 GB HDD sdb on my Ubuntu machine. I will create a Btrfs filesystem on the HDD sdb.
$ sudo lsblk -e7Note: Your HDD/SSD will likely have a different name than mine, and so will the partitions. So, be sure to replace them with yours from now on.
You can create a Btrfs filesystem on your HDD/SSD (without partitioning) if you have a spare HDD/SSD. You can also create a partition on your HDD/SSD and create a Btrfs filesystem there.
For assistance with installing the Btrfs filesystem in Ubuntu, check out the article Install and Use Btrfs on Ubuntu 20.04 LTS.
For assistance with installing the Btrfs filesystem in Fedora, check out the article Install and Use Btrfs on Fedora 33.
Creating a Btrfs Filesystem
You can create a Btrfs filesystem on your HDD/SSD (unpartitioned) or on your HDD/SSD partition.
To create a Btrfs filesystem on the sdb HDD (entire HDD, no partitions) and give it the filesystem label data, run the following command:
$ sudo mkfs.btrfs -L data /dev/sdb
A Btrfs filesystem should now be created on the sdb HDD.
Create the directory /data to mount the Btrfs filesystem you have just created with the following command:
$ sudo mkdir -v /data
To mount the Btrfs filesystem created on the sdb HDD on the /data directory, run the following command:
$ sudo mount /dev/sdb /data
The Btrfs filesystem should now be mounted, as you can see in the screenshot below:
$ df -h /data
Defragmenting a Btrfs Filesystem Manually
As you can see in the screenshot below, I have copied two files on the Btrfs filesystem mounted on the /data directory to demonstrate the process of Btrfs filesystem defragmentation:
$ ls -lh /data
You can defragment a single file or an entire subvolume/directory recursively.
To defragment the single file /data/ubuntu-20.04.1-live-server-amd64.iso, we will run the following command:
$ sudo btrfs filesystem defragment -vf /data/ubuntu-20.04.1-live-server-amd64.iso
The file /data/ubuntu-20.04.1-live-server-amd64.iso should be defragmented.
To defragment every file or directory of the /data directory recursively, run the following command:
$ sudo btrfs filesystem defragment -rvf /data
As you can see, all the files of the /data directory are defragmented.
In the same way, if you had the subvolume /data/osimages, then you could defragment all the files of the /data/osimages subvolume recursively with the following command:
$ sudo btrfs filesystem defragment -rvf /data/osimages
Compressing a Btrfs Filesystem While Defragmenting
The Btrfs filesystem allows you to compress files while you defragment them.
To defragment all the files in the /data directory and compress them with the ZLIB compression algorithm at the same time, run the defragment command with the -czlib option, as follows:
$ sudo btrfs filesystem defragment -rvf -czlib /data
To defragment all the files in the /data directory and compress them with the ZSTD compression algorithm at the same time, run the defragment command with the -czstd option, as follows:
$ sudo btrfs filesystem defragment -rvf -czstd /data
To defragment all the files in the /data directory and compress them with the LZO compression algorithm at the same time, run the defragment command with the -clzo option, as follows:
$ sudo btrfs filesystem defragment -rvf -clzo /data
The files in the /data directory should be defragmented and compressed at the same time.
In the same way, you can defragment and compress the files of a Btrfs subvolume, as well.
Defragmenting a Btrfs Filesystem Automatically
You can enable automatic defragmentation on your Btrfs filesystem at mount time. This feature of the Btrfs filesystem will defragment all the files of your Btrfs filesystem automatically.
To mount the Btrfs filesystem created on the sdb HDD in the /data directory with automatic defragmentation enabled at boot time, you must add an entry for the Btrfs filesystem in the /etc/fstab file.
First, find the UUID of the Btrfs filesystem created on the sdb HDD, as follows:
$ sudo blkid /dev/sdb
As you can see, the UUID of the Btrfs filesystem created on the sdb HDD is 60afc092-e0fa-4b65-81fd-5dfd7dd884de.
It will be different for you. So, be sure to replace it with yours from now on.
Open the /etc/fstab file with the nano text editor, as follows:
$ sudo nano /etc/fstab
Add the following line to the end of the /etc/fstab file:
UUID=60afc092-e0fa-4b65-81fd-5dfd7dd884de /data btrfs autodefrag 0 0
Once you are done, press
For the changes to take effect, reboot your computer with the following command:
$ sudo reboot
As you can see, the Btrfs filesystem created on the sdb HDD is mounted on the /data directory with auto defragmentation enabled.
Problems with Defragmenting a Btrfs Filesystem
Though it may seem that defragmentation improves filesystem performance, there are some problems with defragmenting a Btrfs filesystem.
As Btrfs is a Copy-on-Write (CoW) filesystem, to understand the problems with Btrfs filesystem defragmentation, you must understand the Copy-on-Write feature of the Btrfs filesystem.
Suppose, you have a large file (file1) that uses 100 extents (you can think of extents as file blocks) of a Btrfs filesystem. If you create another copy of that large file (file2) in the same Btrfs filesystem, you will see that no additional disk space is used. That is because the files are identical, and the 100 extents of each file are the same. So, the Btrfs filesystem uses the same extents for both files.
Figure 1: file1 and file2 are identical and sharing the same Btrfs filesystem extents to save disk space
Now, say, you have modified one of the copies of the large file (file2). The modification needs to change 10 of the 100 extents. The Btrfs filesystem will copy the required 10 extents in another unused location (say, e101-e110) of the filesystem and change them there. Once the changes are written to the disk, the Btrfs filesystem will re-link the extents so that the changes are reflected in the large file. The process is illustrated in the figure below:
Figure 2: 10 extents are changed in file2. So, the extents are re-linked in the Btrfs filesystem.
From figures 1 and 2, you can understand how Copy-on-Write (CoW) works and how the Btrfs filesystem uses Copy-on-Write (CoW) to save disk space.
Now that you know how the Copy-on-Write (CoW) feature of the Btrfs filesystem works, you will understand the problems with defragmenting a Btrfs filesystem.
- Defragmenting files move Btrfs data extents and attempt to align them, one after the other. So, the Copy-on-Write links between the copies of the file breaks. This will increase redundant data extents, as well as the disk usage of a Btrfs filesystem that was previously saved by sharing data extents between identical (or nearly identical) copies of the file.
- If a Btrfs subvolume has multiple snapshots, defragmenting the subvolume will break the Copy-on-Write links between the subvolume and the snapshots. This will increase disk usage of a Btrfs filesystem.
- If you are using the Btrfs filesystem for large databases or virtual machine images (for storing VM data/disks), defragmenting the filesystem will also negatively impact the performance of the filesystem.
Conclusion
In this article, you learned how to defragment a single file and the files in a directory/subvolume recursively of a Btrfs filesystem. You also learned how to enable automatic defragmentation on a Btrfs filesystem at mount time. Finally, the article discussed some of the problems with defragmenting a Btrfs filesystem.
