SquashFS and Compressed Filesystems¶
Format: .sqsh, .squashfs, .snap · Type: compressed read-only filesystem · First released: 2002, merged into the mainline kernel in 2009 (2.6.29)
When you want a compressed archive you can mount rather than extract, SquashFS is often better than a tarball: random access to any file, the kernel decompresses on demand, and nothing needs unpacking. Live ISOs, Snap packages, AppImages, and many embedded and router firmware images use it.
Creating images¶
mksquashfs src_dir/ image.sqsh -comp zstd -Xcompression-level 19
mksquashfs src_dir/ image.sqsh -comp xz # smaller, slower to read
mksquashfs src_dir/ image.sqsh -comp lz4 -Xhc # fastest to read
mksquashfs src_dir/ image.sqsh -comp zstd -e .git node_modules # exclude
mksquashfs src_dir/ image.sqsh -b 1M # larger blocks, better ratio
mksquashfs src_dir/ image.sqsh -all-root # files owned by root
Supported compressors (depending on build): gzip, lzo, lz4, xz, zstd, and lzma.
Mounting and extracting¶
sudo mount -o loop,ro image.sqsh /mnt/image
squashfuse image.sqsh ~/mnt/image # unprivileged, via FUSE
unsquashfs -l image.sqsh # list
unsquashfs -s image.sqsh # superblock info (compressor, block size)
unsquashfs -d out_dir image.sqsh # extract everything
unsquashfs -d out_dir image.sqsh etc/config.yaml # extract one path
SquashFS vs. tarball¶
| SquashFS | Tarball | |
|---|---|---|
| Random access | Yes, mount and read | No, sequential |
| Modify in place | No (rebuild, or use an overlay) | Append only when uncompressed |
| Needs root/kernel support | To mount (or squashfuse) | No |
| Streamable through a pipe | No | Yes |
| Preserves Unix metadata | Yes | Yes |
An overlay filesystem gives you a writable view: SquashFS as the lower layer, a tmpfs or directory as the upper layer. Live ISOs work this way.
Transparent filesystem compression¶
Here files stay ordinary files and the filesystem compresses blocks behind the scenes.
Btrfs¶
mount -o compress=zstd:3 /dev/sdX /mnt # zstd level 3
mount -o compress-force=zstd /dev/sdX /mnt # skip the "is it compressible" heuristic
btrfs property set /mnt/logs compression zstd # per-directory/file
btrfs filesystem defragment -r -czstd /mnt/old # recompress existing data
compsize /mnt # real compression ratio
Btrfs supports zlib, lzo and zstd.
ZFS¶
zfs set compression=lz4 pool/dataset # cheap, long the recommended default
zfs set compression=zstd pool/dataset # better ratio
zfs set compression=zstd-3 pool/dataset # explicit level
zfs get compressratio pool/dataset
Changing the property affects newly written data only.
Why du and ls disagree¶
On compressed filesystems, ls -l shows the logical size and du shows on-disk usage:
ls -lh bigfile.log
du -h bigfile.log
du -h --apparent-size bigfile.log # matches ls