You’ve just spent five figures on enterprise NVMe drives, wired up a 100GbE backplane, and meticulously configured your ZFS pools. On paper, your IOPS should be screaming. But the moment you migrate your production PostgreSQL cluster or spin up a dozen KVM instances, the latency spikes. You see your “disk busy” metrics redlining while your actual throughput looks like a trickle. It’s a gut-wrenching feeling—the realization that your high-end hardware is being choked by the very filesystem meant to protect it.

I’ve sat in that chair. I’ve spent the late nights staring at `iostat` and `zpool iostat -v`, wondering why a simple database update is triggering a cascade of disk activity that doesn’t make sense. The culprit is almost always a mismatch between the application’s internal “page size” and the ZFS `recordsize`. When these two numbers don’t align, ZFS stops being a high-performance filesystem and starts acting like a massive anchor. We’re going to fix that today by moving past the “safe” defaults and tuning your datasets for the specific way your data breathes.

Why does the ZFS default 128KB recordsize kill database performance?

ZFS is a copy-on-write (CoW) filesystem. This is its greatest strength and, if misconfigured, its greatest weakness. By default, ZFS uses a `recordsize` of 128KB. This is a brilliant choice for sequential workloads—streaming 4K video, storing large backups, or serving static assets. In these scenarios, writing large chunks of data at once reduces metadata overhead and keeps the disks happy.

However, databases like PostgreSQL and MySQL don’t work in 128KB chunks. PostgreSQL, for instance, operates on 8KB pages. When your database needs to update a single row in an 8KB page, it issues an 8KB write request to the OS. If that database is sitting on a ZFS dataset with the default 128KB recordsize, ZFS cannot simply “overwrite” those 8KB.

Instead, ZFS is forced into a “Read-Modify-Write” (RMW) cycle. It must read the entire 128KB record from the disk into memory, modify the specific 8KB that changed, and then write the entire 128KB record back to a new location on the disk. You just turned an 8KB write into a 128KB read and a 128KB write. That is a 16x write amplification factor. In a high-concurrency environment, this doesn’t just slow things down; it creates an I/O storm that can bring even the fastest NVMe arrays to their knees.

How do I tune ZFS recordsize for PostgreSQL performance?

PostgreSQL is remarkably consistent. It uses an 8KB internal page size for almost everything. If you are running Postgres on ZFS, your primary data dataset must be set to `recordsize=8k`.

When you align these values, the magic of ZFS starts to work for you rather than against you. An 8KB write from Postgres becomes a single 8KB write for ZFS. No RMW cycle, no unnecessary read latency, and no wasted IOPS.

But there is a nuance here that many admins miss: the Write Ahead Log (WAL). Postgres writes to the WAL sequentially. While you could put the WAL on an 8k recordsize dataset, it often benefits from a slightly larger recordsize (like 64k or 128k) to allow for better compression and sequential throughput. However, if you want to keep it simple, the 8k alignment for the main data directory is the single biggest win you can achieve.

One word of caution: changing the `recordsize` property on an existing dataset only affects new data. To truly fix an existing database, you must set the property and then rewrite the data—either by performing a `pg_dump` and restore or by using `rsync` to move the data to a new dataset.

What is the best ZFS recordsize for MySQL and InnoDB?

MySQL’s InnoDB engine defaults to a 16KB page size. Consequently, the logic we applied to Postgres applies here, but shifted: you want `recordsize=16k` for your MySQL data datasets.

There is a specific interaction with the InnoDB “Doublewrite Buffer” that you need to be aware of. InnoDB normally writes data twice to prevent partial page writes (torn pages). Because ZFS is a CoW filesystem, it is inherently immune to torn pages—a write either completes entirely to a new block or it doesn’t happen at all. This means you can safely disable the InnoDB doublewrite buffer (`innodb_doublewrite = 0`) when running on ZFS.

By combining `recordsize=16k` with the removal of the doublewrite buffer, you effectively halve the amount of I/O the database needs to perform. This is where ZFS starts to outperform traditional filesystems like EXT4 or XFS in database workloads, despite the CoW overhead.

Should I use a small volblocksize for Proxmox and KVM virtualization?

Virtualization introduces another layer of complexity because we are often dealing with `zvols` (block devices) rather than datasets (filesystems). For `zvols`, the equivalent property is `volblocksize`.

The default `volblocksize` is often 8k. While this sounds good for IOPS, it can be a disaster for storage efficiency, especially on RAID-Z configurations. This is known as the “Padding Tax.” On a RAID-Z2 pool with a small `volblocksize`, the parity overhead can balloon, sometimes consuming 2x or 3x the expected space because ZFS must pad each block to align with the vdev’s stripe width.

For most Proxmox or KVM workloads, I recommend a `volblocksize` of 16k or 64k. If the VM is running a database, 16k is a solid compromise. If it’s a general-purpose file server or application server, 64k or even 128k provides much better compression ratios and avoids the massive parity padding penalties of RAID-Z. If you are using mirrors (RAID-10 style), you have much more freedom to use smaller block sizes without the space penalty.

How do compression and logbias affect my ZFS storage latency?

Tuning recordsize is the foundation, but `compression` and `logbias` are the finishing touches that define the “feel” of the system.

First, always enable compression. For almost every workload, `compression=lz4` is the gold standard. It is so fast that it often reduces latency because writing compressed data to disk takes less time than writing uncompressed data, and the CPU overhead is negligible. For cold data, `zstd` is excellent, but for active databases, stick with LZ4.

Second, let’s talk about `logbias`. This property tells ZFS how to handle synchronous writes (the kind databases love).
– `logbias=latency` (the default) uses your SLOG (Separate Log) device if you have one. This is what you want for databases to ensure low-latency commits.
– `logbias=throughput` tells ZFS to write synchronous data directly into the main pool, bypassing the SLOG. This can increase total throughput for massive ingestions but will spike your latency.

If you’ve invested in a high-speed Optane or enterprise NVMe SLOG, keep `logbias` at `latency`.

The Actionable Playbook: Implementing ZFS Tuning Today

If you are ready to stop the I/O bleed, follow this checklist. Remember, you cannot change the recordsize of existing data without rewriting it.

1. Audit Your Current Alignment:
Run `zfs get recordsize,volblocksize [pool/dataset]` to see where you stand. If you see 128k on a Postgres dataset, you’ve found your bottleneck.
2. Create Optimized Datasets:
Don’t just change the parent pool. Create specific datasets for specific roles:
zfs create -o recordsize=8k -o xattr=sa -o atime=off poolname/postgres_data
zfs create -o recordsize=16k -o xattr=sa -o atime=off poolname/mysql_data
3. Tune the OS and Application:
For Linux, ensure your `ashift` was set correctly at pool creation (usually `ashift=12` for 4k sector drives). For MySQL, edit your `my.cnf` to set `innodb_doublewrite = 0`.
4. Migrate the Data:
Stop the database service. Use `rsync -avh` or a database-native backup/restore tool to move the files into the new, tuned dataset. This ensures every block written to the new dataset follows the new recordsize rules.
5. Verify with iostat:
Once the database is back up, run `iostat -x 1` or `zpool iostat -v 1`. You should see a dramatic decrease in the %util and a much tighter correlation between application writes and disk writes.
6. Set Up a SLOG for Sync Writes:
If your database latency is still higher than desired, add a dedicated log device (SLOG). Even a small 20GB partition on an enterprise-grade NVMe drive will absorb the synchronous write pressure, allowing the main pool to focus on asynchronous throughput.

Frequently Asked Questions

Can I change the recordsize on a live production database?
You can change the property while the database is running, but it will only apply to new blocks written after the change. This leads to “fragmented” record sizes, which can make performance unpredictable. It is always better to set the recordsize on a fresh dataset and migrate the data into it.

Does a small recordsize hurt compression?
Yes. Compression works better on larger blocks because there is more data to find patterns in. By moving from 128k to 8k, you will likely see your compression ratio drop (e.g., from 2.0x to 1.2x). However, for a database, the performance gain of eliminating Read-Modify-Write cycles is far more valuable than the disk space saved by better compression.

Is recordsize the same as ashift?
No. `ashift` defines the physical sector size of your disks (e.g., `ashift=12` for 4096-byte sectors) and is set at pool creation. `recordsize` is a logical property of the dataset. You can have an 8k recordsize on a pool with `ashift=12`, but you should never have a recordsize smaller than your physical sector size.