ZFS

How to use ZFS:

Background

There are three levels to understand

• zpools are a JBOD of one or more vdevs
• vdevs are groups of drives, likely in raidz (or raidz2, raidz3) or mirror.
• datasets are filesystems stored on a zpool, similar to partitions
• zvol is a virtual block device on a zpool without a filesystem

Usage

# Create a zpool with a mirror vdev.
zpool create -f -o ashift=12 -o compression=lz4 $zpool_name mirror \
  ata-diskA \
  ata-diskB

# Create a dataset.
zfs create -o encryption=aes-256-gcm -o keyformat=passphrase $zpool_name/$dataset_name

Notes

• You should always use the id under /dev/disk/by-id/
  • E.g. /dev/disk/by-id/ata-diskA

Alerts

First setup postfix to send emails.
Then setup ZED notifications

Automatic Scrubs

By default, ZFS on Ubuntu will automatically scrub every month

Automatic Snapshots

See sanoid

zfs list -t snapshot

Caching

ZFS has two read caches:

• ARC - this is enabled by default and uses half of your memory. This memory will be released if you approach out of memory.
• L2ARC - you can enable additional caching by adding an L2ARC drive for ARC to overflow to. You will need approx. 20 MB of ram per GB of L2ARC assuming 4k record sizes.

ARC

arc_summary or arcstat will show you the memory used by ARC. This does not appear in htop.

If you want to reduce arc memory usage, you can set limits by creating /etc/modprobe.d/zfs.conf:

/etc/modprobe.d/zfs.conf

# Set Max ARC size => 4GB == 4294967296 Bytes
options zfs zfs_arc_max=4294967296
# Set Min ARC size => 1GB == 1073741824
options zfs zfs_arc_min=1073741824

Pros and Cons

VS Snapraid + btrfs + mergerfs

Pros

• ZFS has realtime parity.
• ZFS can work while degraded.
• ZFS snapshots with send and receive.
• ZFS has encryption on per-dataset.
• ZFS handles everything altogether including parity on permissions

Cons

• The main con is that ZFS is less expandable.
  • You can only expand by replacing every drive or adding entire vdevs.
• If many drives die, i.e. >2 for raidz2, you lose all your data.

Resources

• Arstechnica ZFS 101
• Archwiki: ZFS