Storage modes¶
Configured in /etc/systemd/journald.conf (or better, a drop-in at /etc/systemd/journald.conf.d/*.conf):
Storage= |
Behaviour |
|---|---|
volatile |
RAM only (/run/log/journal), lost on reboot |
persistent |
Disk (/var/log/journal), created if missing |
auto (default) |
Persistent if /var/log/journal exists, else volatile |
none |
Drop everything (forwarding still works) |
Enabling persistent journal storage¶
sudo mkdir -p /etc/systemd/journald.conf.d
sudo tee /etc/systemd/journald.conf.d/10-persistent.conf >/dev/null <<'EOF'
[Journal]
Storage=persistent
SystemMaxUse=2G
SystemKeepFree=4G
MaxRetentionSec=3month
Compress=yes
EOF
sudo systemctl restart systemd-journald
journalctl --list-boots # should now accumulate across reboots
Alternatively, just sudo mkdir -p /var/log/journal && sudo systemd-tmpfiles --create --prefix /var/log/journal and keep Storage=auto.
Useful journald.conf settings¶
| Setting | Purpose |
|---|---|
SystemMaxUse= |
Cap total disk used by persistent journal |
SystemKeepFree= |
Always leave at least this much free on the filesystem |
SystemMaxFileSize= |
Size at which individual journal files are rotated |
RuntimeMaxUse= |
Same as above, for the volatile journal in /run |
MaxRetentionSec= |
Delete entries older than this |
MaxFileSec= |
Rotate journal files after this long |
RateLimitIntervalSec= / RateLimitBurst= |
Per-service rate limiting (defaults 30s / 10000) |
ForwardToSyslog= |
Forward to a syslog daemon via socket |
ForwardToWall= |
Broadcast emergency messages to logged-in users |
MaxLevelStore= |
Don't store messages below this severity |
Compress= / Seal= |
Compression / Forward Secure Sealing |
Check the effective configuration with:
systemd-analyze cat-config systemd/journald.conf
journalctl essentials¶
# Following and paging
journalctl -f # follow everything
journalctl -e # jump to the end
journalctl -r # newest first
journalctl -n 100 # last 100 lines
journalctl --no-pager # for scripts / piping
# By unit, process, binary, identifier
journalctl -u nginx.service
journalctl -u nginx -u php-fpm # multiple units interleaved
journalctl _PID=1234
journalctl _COMM=sshd
journalctl /usr/sbin/sshd # by executable path
journalctl -t sudo # by SYSLOG_IDENTIFIER (the "tag")
journalctl _UID=1000
journalctl --user -u myapp.service # user units
# By time
journalctl --since "2026-10-09 08:00" --until "2026-10-09 09:30"
journalctl --since "1 hour ago"
journalctl --since today
journalctl --since yesterday --until today
# By boot
journalctl --list-boots
journalctl -b # current boot
journalctl -b -1 # previous boot (great after a crash)
journalctl -b -1 -p err # errors from the previous boot
# By priority (single level = that level and more severe)
journalctl -p err # err, crit, alert, emerg
journalctl -p warning..err # a range
journalctl -p 3 -b # numeric works too
# By facility
journalctl --facility=auth,authpriv
journalctl --facility=help # list facility names
# Kernel only
journalctl -k
# Grep inside messages (PCRE2; case-insensitive if pattern is all lowercase)
journalctl -u sshd -g 'failed password|invalid user'
# Explain messages with catalog text where available
journalctl -xe
Combining matches: different fields are ANDed; the same field repeated is ORed; a literal + means OR between groups:
journalctl _SYSTEMD_UNIT=sshd.service _PID=4321 # AND
journalctl _SYSTEMD_UNIT=sshd.service + _COMM=sudo # OR
Discovering fields and values¶
journalctl -N # list all field names in the journal
journalctl -F _SYSTEMD_UNIT # list unique values of a field
journalctl -F _COMM | sort
journalctl -u nginx -o verbose -n 1 # show every field on one entry
Commonly used trusted fields (prefixed _, set by journald, can't be spoofed by the client): _PID, _UID, _GID, _COMM, _EXE, _CMDLINE, _SYSTEMD_UNIT, _HOSTNAME, _TRANSPORT (kernel, syslog, journal, stdout, audit), _BOOT_ID.
Output formats¶
journalctl -o short-iso # ISO timestamps
journalctl -o short-precise # microsecond timestamps
journalctl -o short-monotonic # time since boot
journalctl -o cat # message text only
journalctl -o json # one JSON object per line
journalctl -o json-pretty
journalctl -o export # binary-safe export format
journalctl -o json --output-fields=MESSAGE,_SYSTEMD_UNIT,PRIORITY
JSON output pairs well with jq:
# Top 10 noisiest units in the last hour
journalctl --since "1 hour ago" -o json --output-fields=_SYSTEMD_UNIT \
| jq -r '._SYSTEMD_UNIT // "none"' | sort | uniq -c | sort -rn | head
# Failed SSH logins with source IPs
journalctl -u ssh -u sshd --since today -g 'Failed password' -o cat \
| grep -oE 'from [0-9.]+' | sort | uniq -c | sort -rn
Disk usage and maintenance¶
journalctl --disk-usage
sudo journalctl --rotate # close current files, start new ones
sudo journalctl --vacuum-size=500M # shrink archived files to 500M total
sudo journalctl --vacuum-time=30d # remove archived entries older than 30 days
sudo journalctl --vacuum-files=10 # keep at most 10 archived files
sudo journalctl --verify # check file integrity
sudo journalctl --flush # move /run journal to /var
Vacuuming only affects archived files, so run --rotate first if you need the active file to be considered.
Reading another system's journal¶
Useful from rescue media, or when inspecting a disk image or container:
journalctl -D /mnt/sysimage/var/log/journal -b -1 -p err
journalctl --file /path/to/system@….journal
journalctl -M mycontainer # systemd-nspawn / machinectl guest
Access control¶
Members of systemd-journal (and on most distros adm and wheel) can read the full system journal. Ordinary users only see their own user journal.
sudo usermod -aG systemd-journal andrew
Native journal-to-journal remote logging¶
systemd ships its own remote transport (systemd-journal-remote / systemd-journal-upload, HTTP(S) on port 19532). It preserves all structured fields but is less widely supported than syslog. For most environments, rsyslog or a modern shipper (see Shipping Logs to a Log Platform) is the more practical choice.