xz¶

Format: .xz (and legacy .lzma) · Algorithm: LZMA2 · First released: 2009 (xz-utils, Lasse Collin); LZMA itself by Igor Pavlov, late 1990s

xz brought the LZMA algorithm from 7-Zip to Unix in a proper container format with integrity checks. It gives excellent ratios, is slow to compress at high levels, and decompresses at moderate speed. Kernel source tarballs and many distro package formats used .xz for years; several distros have since moved packages to zstd.

Basics¶

xz file                  # -> file.xz, removes file
xz -k file               # keep original
xz -d file.xz            # decompress (same as unxz)
unxz file.xz
xz -c file > out.xz      # to stdout
xz -6 file               # default preset
xz -9 file               # strong
xz -9e file              # "extreme": slower still, slightly smaller
xz -0 file               # fastest
xz -T0 file              # use all cores
xz -l file.xz            # list info (sizes, ratio, blocks)
xz -lvv file.xz          # very verbose: memory needed to decompress, etc.
xz -t file.xz            # test integrity

Working with .xz files without extracting¶

xzcat file.xz
xzless file.xz
xzmore file.xz
xzgrep 'pattern' file.xz
xzdiff a.xz b.xz

With tar¶

tar cJvf archive.tar.xz src_dir/
tar xJvf archive.tar.xz           # or just tar xf
tar tJvf archive.tar.xz
tar -I 'xz -T0 -9' -cf archive.tar.xz src_dir/

Note the capital J for xz (lowercase j is bzip2).

Presets and memory¶

xz's presets mostly control the dictionary size, which costs memory on both ends:

Preset	Dictionary	Compress memory	Decompress memory
`-0`	256 KiB	~3 MiB	~1 MiB
`-6` (default)	8 MiB	~94 MiB	~9 MiB
`-9`	64 MiB	~674 MiB	~65 MiB

(Single-threaded figures; multithreaded compression multiplies the compression memory by roughly the thread count.)

On small VMs or memory-limited containers, high presets can be OOM-killed. Cap it:

xz -T0 --memlimit-compress=50% -9 file
xz -d --memlimit-decompress=256MiB file.xz

Raising the preset beyond the input size gives no benefit: a 64 MiB dictionary is pointless for a 5 MiB file.

Multithreading¶

-T0 uses one thread per core (-T4 for a fixed count). Multithreaded xz splits input into independent blocks:

The ratio drops slightly.
The output can be decompressed in parallel by newer xz and pixz.
Single-threaded output is one block and can only be decompressed serially.

Recent xz versions enable threaded mode more readily by default; set -T explicitly in scripts so the behaviour doesn't change with the version.

Legacy .lzma¶

Before xz there was LZMA Utils and the .lzma ("LZMA-alone") format, which has no integrity check and no multi-block support. xz still reads and writes it:

xz -d file.lzma          # or unlzma, lzcat
xz --format=lzma file    # create .lzma (same as `lzma file`)

Don't create new .lzma files.

Filters for executables¶

xz supports BCJ filters that preprocess machine code so it compresses better:

xz --x86 --lzma2=preset=9e program.bin

Newer xz versions apply suitable filters automatically when building some archives; for most data, presets alone are fine.

pixz¶

pixz is a parallel, indexed xz. A tarball made with it lets you extract a single file without decompressing everything before it:

tar -I pixz -cf archive.tpxz dir/
pixz -l archive.tpxz                       # list
pixz -x path/in/archive < archive.tpxz | tar x

The output is still valid xz, readable by plain xz.

The 2024 backdoor¶

xz-utils 5.6.0 and 5.6.1 shipped with a deliberately planted backdoor (CVE-2024-3094), introduced through a multi-year social-engineering campaign against the project. The malicious code was hidden in test files and release tarballs (not the Git tree) and targeted sshd on distros that link it to libsystemd, which pulls in liblzma. It was discovered in March 2024 by Andres Freund before it reached most stable distributions.

Check old images, appliances, and pinned containers:

xz --version

Anything reporting 5.6.0 or 5.6.1 should be rebuilt on a fixed release.

xz¶