Shell script to copy all files recursively and upload them to remote FTP server

by Vivek Gite on April 9, 2008 · 17 comments · [ vivek@nixcraft.com ]

#!/bin/bash
# Shell script to copy all files recursively and upload them to
# remote FTP server (copy local all directories/tree to remote ftp server)
#
# If you want to use this script in cron then make sure you have
# file pointed by $AUTHFILE (see below) and add lines to it:
# host ftp.mycorp.com
# user myftpuser
# pass mypassword
#
# This is a free shell script under GNU GPL version 2.0 or above
# Copyright (C) 2005 nixCraft
# Feedback/comment/suggestions : http://cyberciti.biz/fb/
# -------------------------------------------------------------------------
# This script is part of nixCraft shell script collection (NSSC)
# Visit http://bash.cyberciti.biz/ for more information.
# -------------------------------------------------------------------------
 
FTP="/usr/bin/ncftpput"
CMD=""
AUTHFILE="/root/.myupload"
 
if [ -f $AUTHFILE ] ; then
  # use the file for auth
  CMD="$FTP -m -R -f $AUTHFILE $myf $remotedir $localdir"
else
  echo "*** To terminate at any point hit [ CTRL + C ] ***"
  read -p "Enter ftpserver name : " myf
  read -p "Enter ftp username : " myu
  read -s -p "Enter ftp password : " myp
  echo ""
  read -p "Enter ftp remote directory [/] : " remotedir
  read -p "Enter local directory to upload path [.] : " localdir
  [ "$remotedir" == "" ] && remotedir="/" || :
  [ "$localdir" == "" ] && localdir="." || :
  CMD="$FTP -m -R -u $myu -p $myp $myf $remotedir $localdir"
fi
 
$CMD
4000+ howtos and counting! If you enjoyed this article, join 45000+ others and get free email updates!

O2Link April 20, 2008

Thank you!
it was so usefull for my work.

Reply

elejido April 22, 2009

If the SAME file exists in the remote server, the script overwrite it? append? resume?

Reply

Vivek Gite April 23, 2009

It will overwrite the file.

Reply

me May 19, 2009

I love you!

Reply

me May 19, 2009

If using the AUTHFILE option, don’t forget to set the following variables in this script

myf=”my-ftp-server.com”
remotedir=”/my/remote/dir”
localdir=”/my/local/dir”

And yes, you will need ncftp installed, if it is not already
For OS X with mac ports,
sudo port install ncftp

Reply

Dabidi July 8, 2009

How to skip upload existing files on remote server?

Reply

Rotx September 10, 2009

I am taking server name from a file and everything working fine buy my problem is if a server in the list is not connecting it is not going to next available server. How to get the ftp status in the script if ftp is not connecting how to skip to next available server .

Reply

BJones December 13, 2009

Can we add this script to cron and have it auto backup? I am looking to back up entire dir every week, like at 3am, when there is the least amount of traffic, but I don’t want to have to be around for this to happen.

Reply

Fred THL December 21, 2009

Why not considering rsync over ssh ?

Reply

jamuna December 24, 2009

HI please explain it

[ "$remotedir" == "" ] && remotedir=”/” || :
[ "$localdir" == "" ] && localdir=”.” || :

Reply

Brendon February 13, 2010

How can I make this GET the remote files. Obviously I used it to send them all there – now how can I send my 100GB back to the server?

Reply

Pelle May 3, 2010

Hello,
I hade to delete “$myf” in the line below:
CMD=”$FTP -m -R -f $AUTHFILE $myf $remotedir $localdir”

Then it all went well.

Thank you for an good and simple script!
/Pelle

Reply

Gaurab August 5, 2010

Can all the files zipped in a single zip file before copying to other remoter server.

Reply

Pavan November 14, 2011

hi , I have shell script to sftp the files from one server to other server. When executed manually it works fine but when scheduled the script using crontab it fails.
Below is the piece of code. Please help me in solving this issue. Thanks in advance.

 #!/bin/bash
 USER=username
 PASSWORD=password
 HOST=hostname
 dt=`date +%Y%m%d`
 fldt=`date --date "$dt 1 days ago" +%Y%m%d`
 ldir=/home/localdir
sudo -u xyz sftp $USER@$HOST <<EOF
$PASSOWRD
 cd rmtdir
 mget filename_$fldt*.dat.txt $ldir
 quit
 EOF
 if [ $? -ne 0 ]; then
 echo "sftp failed" | mailx -s "`date` :Sftp failed" pavan@ymail.com
 else
 echo "sftp success"| mailx -s "`date` :Sftp success" pavan@ymail.com
 fi
 exit 0

Reply

Goksel December 24, 2011

/usr/bin/sudo -u xyz /usr/bin/sftp

Reply

andrew April 23, 2012

Is there a way to skip sir, it’s very important as I’m doing massive amount of files, and our destination server’s connection isn’t reliable. Cheers

Reply

Astral May 2, 2012

Fantastic! Thanks !

Reply

Leave a Comment

You can use these HTML tags and attributes for UNIX commands or shell scripts: <strong> <em> <ol> <li> <u> <ul> <blockquote> <pre> <a href="" title="">
What is 7 + 10 ?
Please leave these two fields as-is:
Are you a human being? Solve the simple math so we know that you are a human and not a script.



Tagged as: , , , , , , , , , , , , , ,