HTTrack downloads an entire website's structure to a local directory, rewriting links so it can be browsed offline.
sudo dnf install httrack # or apt-get install httrack
httrack "https://example.com/docs/" -O ~/mirrors/example-docs \
"+*.example.com/docs/*" -v
| Option | Meaning |
|---|---|
-O dir |
output directory |
+pattern / -pattern |
include / exclude URL patterns (wildcard filters) |
-v |
verbose progress |
-%P |
resume/update an existing mirror |
-r<N> |
limit recursion depth |
--depth=N |
same, long form |
Mirroring is a common use for documentation archival, but it can generate heavy load and get you blocked or, worse, breach a site's terms of service. Respect robots.txt, throttle with -%c<N> (connections) and rate limits, and mirror only what you need (narrow the URL filters). wget --mirror (see wget) is a lighter-weight alternative for simpler mirroring jobs.