转载How to use ftp in a shell script

How to use ftp in a shell script

Bruce EdigerBruce Ediger's home page


Party on!
Continue using Mozilla's browser.

You might also consider using Linux. It's faster and has less malware.

Sometimes I want to FTP a file from one machine to another. Usually, I can do the transfer interactively, but every so often, I would like to have a shell script do the file transfer. This task has eluded me in the past, but I finally figured it out. I've not seen this particular trick documented in the past, so I submit it for your approval.

The Problem

The problem I always encountered in scripting ftp transfers involved getting a password to the ftp server. Typical ftp client programs under Unix, Linux, Solaris and NetBSD all read the ftp password from /dev/tty.

Example (non-working) script

#!/bin/sh
HOST='ftp.users.qwest.net'
USER='yourid'
PASSWD='yourpw'
FILE='file.txt' ftp $HOST <<END_SCRIPT
user $USER
$PASSWD
put $FILE
quit
END_SCRIPT
exit 0

The above script will just hang if run in the foreground (in an xterm), or if run in the background (from a cron job), it will fail to perform the work of transferring file.txt.

/dev/tty names a strange, magic device. Each process (more strictly each process group) has a different /dev/tty, and you can not naively make ftp clients read the password from some non-magic, yet convenient source, like a "here document". When run in an xterm, the script above appears to hang because it reads the password from /dev/tty. The xterm constitutes the script's /dev/tty, so the script waits for keyboard input.

Example Working Script

#!/bin/sh
HOST='ftp.users.qwest.net'
USER='yourid'
PASSWD='yourpw'
FILE='file.txt' ftp -n $HOST <<END_SCRIPT
quote USER $USER
quote PASS $PASSWD
put $FILE
quit
END_SCRIPT
exit 0

The Tricks

Getting the password to the ftp server without having the ftp client program read the password from /dev/tty requires two tricks:

  1. Using the -n option on the ftp client program to prevent the ftp client from trying to log in immediately. That way, the ftp client does not ask for a user ID and password. No use of /dev/tty.
  2. Use the ftp client program command quote to send user ID and password to the ftp server.

You must the token that ends the "here document" (END_SCRIPT in the example above) at the beginning of a line. Even if the ftp command line and the login and transfer script are indented, END_SCRIPTshould appear with the 'E' as the first character of the line.

Further Refinements

The above sh script will spew lots of ftp client output to standard output. Even if everything works perfectly, the user running the above script will see lots of incomprehensible text scrolling by quite rapidly. One refinement would send output to different places:

ftp -n $HOST > /tmp/ftp.worked 2> /tmp/ftp.failed <<END_SCRIPT

One could further refine error handling by acting on the ftp client program's exit status:

ftp -n $HOST > /tmp/ftp.worked 2> /tmp/ftp.failed <<END_SCRIPT
blah blah
END_SCRIPT EXITSTATUS=$? if [ $EXITSTATUS != "0" ]
then
# handle the error...
fi

Except that the above doesn't always work - most FTP clients always exit with a status of 0. This leads to ugly "false negatives": the file transfer fails, but the script doesn't detect the problem.

One way to verify that a file transfer took place - transfer it back:

#!/bin/sh

ftp -n << END_SCRIPT
open $1
user $2 $3
put $4
get $4 retrieval.$$
bye
END_SCRIPT if [ -f retrieval.$$ ]
then
echo "FTP of $4 to $1 worked"
rm -f retrieval.$$
else
echo "FTP of $4 did not work"
fi

Regular FTPs there and back of large files can consume a lot of time.

Control of ftp by a shell script

One obvious improvement would have the ftp client program controlled by the shell script. I don't think that would comprise an impossible task, but I also don't think that it would have much value. Scripting ftp transfer using expect might cause you less pain.


Alternative #1

I saw a second way of doing this in a usenet article:

#!/bin/sh
USER=userid
PASSWD=userpw
ftp -n f2dev <<SCRIPT
user $USER $PASSWD
binary
get some.file
quit
SCRIPT

It still uses the "-n" trick, but it sends user ID and password in the same "user" command.


Alternative #2

Use a .netrc file

Linux, Unix and BSD users have the alternative of using a .netrc file. The ftp man page documents the format of .netrc. To accomplish the task of using ftp in a shell script you would have to fill out a .netrc file something like this:

machine something.else.com
login myid
password mypassword

ftp demands that .netrc not have group or world read or write permissions:

$ ls -l .netrc
-rw------- 1 bediger users 51 Dec 16 13:30 .netrc

Using a .netrc file has a few problems that may or may not prevent you from using it.

  • A shell scripkt that does FTP using .netrc is no longer self-contained. You have to keep track of two files, which means that bugs can be less than obvious.
  • ftp reads it's user ID's .netrc. If you develop your script under a given user ID, then put it in production under a second user ID, you have to coordinate .netrc file contents between those two user IDs.

Alternative #3

Apparently, the Ckermit program from Columbia University understands FTP. You could use Ckermit to script FTP transfers. This looks to have advantages and disadvantages. On the "pro" side, it appears that Ckermit can exit on various errors, like unknown user IDs, or bad passwords. On the "con" side, you have to have Ckermit. I don't recall that it had a too onerous install, but it doesn't come with many Linux distros these days, and it probably doesn't come with any vendor Unix.


$Id$

How to use ftp in a shell script的更多相关文章

  1. shell script练习

    执行脚本的几种方式: 1. sh a.sh 或者  bash a.sh  调用的是 /bin/bash 进程执行的,所以脚本不需要执行权限. 2. 直接使用绝对路径执行, /home/script/a ...

  2. Linux 脚本运维总结之Shell script

    1. 本地变量和环境变量 变量类型 定义形式 声明位置 显示命令 作用域 本地变量 VARNAME=value 命令行或shell脚本 set (显示所有变量) 本用户,本进程 环境变量 export ...

  3. shell 编程 && bash 简介(shell 变量、shell操作环境、数据流重导向、管线命令、shell script)

    如何学习一门编程语言 数据类型 运算符 关键字 1 认识BASH 这个shell linux是操作系统核心,用户通过shell与核心进行沟通,达到我们想要的目的.硬件.核心.用户之间的关系: 原理:所 ...

  4. shell及脚本4——shell script

    一.格式 1.1 开头 必须以 "# !/bin/bash"  开头,告诉系统这是一个bash shell脚本.注意#与!中间有空格. 二.语法 2.1 数值运算 可以用decla ...

  5. shell script

    一.shell script的编写与执行 1.shell script 的编写中还需要用到下面的注意事项: a.命令的执行是从上到下,从左到右地分析与执行 b.命令.参数间的多个空白都会被忽略掉 c. ...

  6. (copy) Shell Script to Check Linux System Health

    source: http://linoxide.com/linux-shell-script/shell-script-check-linux-system-health/ This article ...

  7. 这些年我们一起搞过的持续集成~Jenkins+Perl and Shell script

    这些年我们一起搞过的持续集成~Jenkins+Perl and Shell script ##转载注明出处:http://www.cnblogs.com/wade-xu/p/4378224.html ...

  8. CentOS Linux下一个tomcat起停,查看日志的shell script

    CentOS 的tomcat安装目录:/usr/local/tomcat vi MyTomcatUitl.sh          创建文件chmod u+x MyTomcatUtil.sh   赋执行 ...

  9. Shell script for logging cpu and memory usage of a Linux process

    Shell script for logging cpu and memory usage of a Linux process http://www.unix.com/shell-programmi ...

随机推荐

  1. 亚马逊AWS的route53的收费,月费最低 0.9 USD

    亚马逊AWS的route53的收费Amazon Route 53 定价 https://aws.amazon.com/cn/route53/pricing/ 一文中,对于一些术语的解释第一项收费--域 ...

  2. Windows Embedded POSready2009

    Windows Embedded POSready2009 ,这个看上去和 XP 差不多,可能是别人说的 XPE 系统 下载 POSready2009_CD.iso, 安装 KEY :         ...

  3. eclipse - 下载网址

    这里面有着非常齐全的eclipse相关资源,而且都是放在网盘里面的,下载也方便 http://www.androiddevtools.cn/

  4. SQL中实现千分位的语句

    传递一个sql的小知识,现成的语句,在工作流的表单域中很实用. 数字或字符串转成千分位 ) 字符串转换成数值 )

  5. Codefroces 784 愚人节题目(部分)

    A. Numbers Joke time limit per test 2 seconds memory limit per test 64 megabytes input standard inpu ...

  6. CISP/CISA 每日一题 16

    CISA 每日一题(答) 作业调度软件的优点: 1.作业信息仅需建立一次,减少错误发生概率: 2.可定义作业间的依赖关系,当某一项作业失败时,依赖于该作业的后续作业就不会被执行: 3.所有成功或失败的 ...

  7. map按value查找相应元素

    find_if算法用来在map中查找value符合条件的pair元素,返回指向该符合条件元素的迭代器,如果找到,那么返回最后一个元素的后一个元素end(); 1.首先要定义头文件 #include & ...

  8. PHPki

    PHPki PHPki是一个基于开放源码Web的应用程序,用来管理遵守HIPAA的多代理"公钥基础结构".它可以用于创建X.509数字证书,并主要为支持S/MIME的电子邮件客户端 ...

  9. Elasticsearch和MongoDB

    Elasticsearch和MongoDB分片及高可用对比 本文旨在对比Elasticsearch和MongoDB高可用和分片的实现机制. Elasticsearch ES天生就是分布式的,那她又是如 ...

  10. kafka同步生产者和异步生产者深入剖析

    什么是kafka同步生产者,什么是kafka异步生产者? 比如这里某个topic有3个分区. kafka同步生产者:这个生产者写一条消息的时候,它就立马发送到某个分区去.  kafka异步生产者:这个 ...