TestUsage broken on ZFS

[samuelkarp]

When I run TestUsage in the fs package with a temp dir on a ZFS mount, it fails like this:

$ TMPDIR=/testpool/tmp go test ./fs -v -run TestUsage
=== RUN   TestUsage
=== RUN   TestUsage/SingleSmallFile
=== PAUSE TestUsage/SingleSmallFile
=== RUN   TestUsage/MultipleSmallFile
=== PAUSE TestUsage/MultipleSmallFile
=== RUN   TestUsage/BiggerFiles
=== PAUSE TestUsage/BiggerFiles
=== RUN   TestUsage/SparseFiles
=== PAUSE TestUsage/SparseFiles
=== RUN   TestUsage/Hardlinks
=== PAUSE TestUsage/Hardlinks
=== RUN   TestUsage/HardlinkSparefile
=== PAUSE TestUsage/HardlinkSparefile
=== CONT  TestUsage/SingleSmallFile
    du_test.go:129: Wrong usage size 1536, expected 132096
=== CONT  TestUsage/HardlinkSparefile
=== CONT  TestUsage/SparseFiles
    du_test.go:129: Wrong usage size 2560, expected 525312
=== CONT  TestUsage/MultipleSmallFile
=== CONT  TestUsage/Hardlinks
=== CONT  TestUsage/BiggerFiles
=== CONT  TestUsage/HardlinkSparefile
    du_test.go:129: Wrong usage size 1536, expected 263168
=== CONT  TestUsage/MultipleSmallFile
    du_test.go:129: Wrong usage size 2048, expected 263168
=== CONT  TestUsage/Hardlinks
    du_test.go:129: Wrong usage size 1536, expected 132096
=== CONT  TestUsage/BiggerFiles
    du_test.go:129: Wrong usage size 2560, expected 394240
--- FAIL: TestUsage (0.00s)
    --- FAIL: TestUsage/SingleSmallFile (0.00s)
    --- FAIL: TestUsage/SparseFiles (0.00s)
    --- FAIL: TestUsage/Hardlinks (0.00s)
    --- FAIL: TestUsage/BiggerFiles (0.00s)
    --- FAIL: TestUsage/HardlinkSparefile (0.00s)
    --- FAIL: TestUsage/MultipleSmallFile (0.00s)
FAIL
FAIL	github.com/containerd/continuity/fs	0.006s
FAIL

If I add a sleep before DiskUsage is called, I get slightly different but still-failing results:

$ TMPDIR=/testpool/tmp go test ./fs -v -run TestUsage
=== RUN   TestUsage
=== RUN   TestUsage/SingleSmallFile
=== PAUSE TestUsage/SingleSmallFile
=== RUN   TestUsage/MultipleSmallFile
=== PAUSE TestUsage/MultipleSmallFile
=== RUN   TestUsage/BiggerFiles
=== PAUSE TestUsage/BiggerFiles
=== RUN   TestUsage/SparseFiles
=== PAUSE TestUsage/SparseFiles
=== RUN   TestUsage/Hardlinks
=== PAUSE TestUsage/Hardlinks
=== RUN   TestUsage/HardlinkSparefile
=== PAUSE TestUsage/HardlinkSparefile
=== CONT  TestUsage/SingleSmallFile
=== CONT  TestUsage/HardlinkSparefile
=== CONT  TestUsage/Hardlinks
=== CONT  TestUsage/SparseFiles
=== CONT  TestUsage/SingleSmallFile
    du_test.go:132: Wrong usage size 2048, expected 132096
=== CONT  TestUsage/HardlinkSparefile
    du_test.go:132: Wrong usage size 266752, expected 263168
=== CONT  TestUsage/BiggerFiles
=== CONT  TestUsage/MultipleSmallFile
=== CONT  TestUsage/Hardlinks
    du_test.go:132: Wrong usage size 62976, expected 132096
=== CONT  TestUsage/SparseFiles
    du_test.go:132: Wrong usage size 399360, expected 525312
=== CONT  TestUsage/BiggerFiles
    du_test.go:132: Wrong usage size 55296, expected 394240
=== CONT  TestUsage/MultipleSmallFile
    du_test.go:132: Wrong usage size 3072, expected 263168
--- FAIL: TestUsage (0.00s)
    --- FAIL: TestUsage/SingleSmallFile (5.00s)
    --- FAIL: TestUsage/HardlinkSparefile (5.00s)
    --- FAIL: TestUsage/Hardlinks (5.00s)
    --- FAIL: TestUsage/SparseFiles (5.00s)
    --- FAIL: TestUsage/BiggerFiles (5.00s)
    --- FAIL: TestUsage/MultipleSmallFile (5.00s)
FAIL
FAIL	github.com/containerd/continuity/fs	10.005s
FAIL

It looks like the DiskUsage function is working correctly, but the expected size calculation doesn't hold for how ZFS manages its blocks. This occurs both with OpenZFS on Ubuntu 16.04 and on FreeBSD 12.

[mateuszkwiatkowski]

I ran these tests on few more filesystems on FreeBSD CURRENT (d8819d88af52). Expand to see setup instructions for each FS:

❌ UFS on zvol

zfs create -V 16G zroot/test-ufs
newfs /dev/zvol/zroot/test-ufs
mkdir /tmp/test-ufs-zvol
mount /dev/zvol/zroot/test-ufs /tmp/test-ufs-zvol
chown kwiat:kwiat /tmp/test-ufs-zvol

❌ UFS on memory disk

mdconfig -s 16g -u md0
newfs /dev/md0
mkdir /tmp/test-ufs-md
mount /dev/md0 /tmp/test-ufs-md
chown kwiat:kwiat /tmp/test-ufs-md

✅ tmpfs

mkdir /tmp/tmpfs
mount -t tmpfs tmpfs /tmp/test-tmpfs
chown kwiat:kwiat /tmp/test-tmpfs

❌ ext2 on zvol

zfs create -V 16G zroot/test-ext2
newfs /dev/zvol/zroot/test-ext2
mkdir /tmp/test-ext2-zvol
mount /dev/zvol/zroot/test-ext2 /tmp/test-ext2-zvol
chown kwiat:kwiat /tmp/test-ext2-zvol

❌ ext2 on memory disk

mdconfig -s 16g -u md1
mkfs.ext2 /dev/md1
mkdir /tmp/test-ext2-md
mount /dev/md1 /tmp/test-ext2-md
chown kwiat:kwiat /tmp/test-ext2-md

✅ ext4 on zvol

zfs create -V 16G zroot/test-ext4
newfs /dev/zvol/zroot/test-ext4
mkdir /tmp/test-ext4-zvol
mount /dev/zvol/zroot/test-ext4 /tmp/test-ext4-zvol
chown kwiat:kwiat /tmp/test-ext4-zvol

✅ ext4 on memory disk

mdconfig -s 16g -u md2
mkfs.ext4 /dev/md2
mkdir /tmp/test-ext4-md
mount /dev/md2 /tmp/test-ext4-md
chown kwiat:kwiat /tmp/test-ext4-md

[samuelkarp]

In the two years since I opened this (and that I apparently forgot it was open), the results I'm getting (now on FreeBSD 13.1-RELEASE) are slightly different:

--- FAIL: TestUsage (0.00s)
    --- PASS: TestUsage/SingleSmallFile (0.00s)
    --- FAIL: TestUsage/HardlinkSparefile (0.00s)
    --- FAIL: TestUsage/Hardlinks (0.01s)
    --- FAIL: TestUsage/SparseFiles (0.01s)
    --- FAIL: TestUsage/BiggerFiles (0.01s)
    --- PASS: TestUsage/MultipleSmallFile (0.00s)