One of my filesystems just ran out of space. When I tried to grow it it complained:

$ sudo lvextend --size +400G /dev/frej/frej /dev/mapper/frej --resizefs
  Size of logical volume frej/frej changed from <15.95 TiB (4180236 extents) to <16.34 TiB (4282636 extents).
  Logical volume frej/frej successfully resized.
resize2fs 1.44.5 (15-Dec-2018)
resize2fs: New size too large to be expressed in 32 bits

fsadm: Resize ext4 failed.
  /sbin/fsadm failed: 1

The magic limit turns out to be exactly 16TiB, or $16\; *\; 2⁴⁰B$.

• The block size is $4096B\; =\; 2¹²B$
• There are $2³²$ blocks.
• $2¹²B\; *\; 2³²\; =\; 2⁴⁴B\; =\; 16\; *\; 2⁴⁰B\; =\; 16\; TiB$

The solution is naturally to use 64 bits instead. 64bit mode is described in ext4(5):

Enables the file system to be larger than 2^32 blocks. This feature is set automatically, as needed, but it can be useful to specify this feature explicitly if the file system might need to be resized larger than 2^32 blocks, even if it was smaller than that threshold when it was originally created. Note that some older kernels and older versions of e2fsprogs will not support file systems with this ext4 feature enabled.

This filesystem was created long ago. Way before I could have volumes larger than 16TiB. Over time I upgraded the hardware and resized the volume. Today I reached 16TiB and all of a sudden 32 bits was no longer enough.

Changing the volume to 64bit mode is quite easy. Simply pass the -b flag to resize2fs(8):

$ sudo resize2fs -b /dev/frej/frej
resize2fs 1.44.5 (15-Dec-2018)
Converting the filesystem to 64-bit.
resize2fs: No space left on device while trying to resize /dev/frej/frej
Please run 'e2fsck -fy /dev/frej/frej' to fix the filesystem
after the aborted resize operation.

It failed.

It took me quite a while to figure out why there was no space left. I first tried to allocate slightly more space on the underlying logical volume. That failed too.

Turns out resize2fs require some free space in the filesystem itself. Since I ran out of space just before growing the filesystem none was available. Together with this filesystem not having any reserved blocks (tune2fs -m 0) meant the conversion did not work. I removed about 5 GB and tried again:

$ sudo resize2fs -b -p /dev/frej/frej
resize2fs 1.44.5 (15-Dec-2018)
Please run 'e2fsck -f /dev/frej/frej' first.

$ sudo e2fsck -f /dev/frej/frej
e2fsck 1.44.5 (15-Dec-2018)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
frej: 165788/535072768 files (34.3% non-contiguous), 4279267286/4280561664 blocks
$ sudo resize2fs -b -p /dev/frej/frej
resize2fs 1.44.5 (15-Dec-2018)
Converting the filesystem to 64-bit.
Begin pass 2 (max = 25488)
Relocating blocks             XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Begin pass 3 (max = 130633)
Scanning inode table          XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Begin pass 5 (max = 4)
Moving inode table            XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
The filesystem on /dev/frej/frej is now 4280561664 (4k) blocks long.

And now the filesystem has the 64bit feature enabled and we can grow it past 16TiB.