Parallel
2024-10-24
Pulling files from remote server with rsync
To transfer just the files:
ssh user@remote -- find /path/to/parent/directory -type f | parallel -v -j16 rsync -Havessh -aAXP user@remote:{} /local/path
To transfer the entire directory:
echo "/path/to/parent/directory" | parallel -v -j16 rsync -Havessh -aAXP user@remote:{} /local/path
Pushing files to remote server with rsync
To transfer just the files:
find /path/to/local/directory -type f | parallel -v -j16 -X rsync -aAXP /path/to/local/directory/{} user@remote:/path/to/dest/dir
Running the same command on multiple remote hosts
parallel --tag --nonall -S remote0,remote1,remote2 uptime