Parallel Recursive FTPS Mirroring (Directory Download) from Command Line

Published by Philipp Schuster on

Code snippet: Command line tool lftp for a parallel recursive secure FTP download of a remote directory.


In this blog post, I show you the best option I found that you can use in 2024 to perform a parallel recursive download (backup) of an FTP directory via FTPS (explicit FTP via TLS) from the command line. lftp is the best command line tool (for Linux, macOS and other UNIX-like systems), but needs to be properly configured. For GUI-focused users, I recommend FileZilla.


FTP is a quite outdated protocol these days. Apart from accessing web storage on various simple web hosting providers, I’m not aware of any real and justified use of this technology in 2024. “Modern” alternatives such as SFTP are usually better suited, but you need SSH access to a server for that.

The Problem

Not every (old-fashioned) web hosting provider gives you an SSH/SFTP access (at least in the lower paying tiers). I’m also “trapped” in such a situation. I’m having various (10+ year) old web projects that still live on such a web hosting provider, and I just didn’t migrate them yet onto something I have more control over.

Downloading the website files onto a locally owned storage, especially when having PHP-centric software, such as WordPress or Matomo, is crucial as backup to be independent of the provider. As finding information on how to download files recursively and parallel via FTPS (explicit FTP over TLS) was not as easy or as compact to find as hoped, here’s my solution. In the end, it is quite simple.

The Solution: lftp

lftp is an open-source CLI tool available in most Linux distributions and package managers. All it takes is the following bash command:

lftp -u $USER,$PASS -e "\
	set ftp:ssl-force true \
	set ftp:ssl-protect-data true \
	" $HOST <<EOF

mirror --continue --parallel=10 --verbose $REMOTE_DIR $LOCAL_DIR


It performs a

  • TLS-secured FTP transmission (no credentials are transferred in plain text)
  • recursive mirroring of the specified remote directory
  • 10 parallel connections (FTP is slow: so parallel connections mitigate that to a certain degree)

Convenient Wrapper Script

I’ve created a convenient wrapper script acting as high-level CLI tool around lftp. You can find it as part of my NixOS configs and consume it like this (Nix required):

  • $ nix shell github:phip1611/nixos-configs#ftp-backup.
  • $ ftp-backup --help

Philipp Schuster

Hi, I'm Philipp and interested in Computer Science. I especially like low level development, making ugly things nice, and de-mystify "low level magic".


Leave a Reply

Your email address will not be published. Required fields are marked *