How to use pipes to connect programs

From Linux Bash Shell Scripting Tutorial Wiki
Jump to navigation Jump to search

← PipesHomeInput redirection in pipes →

Here is how to use pipes to connect programs with examples to illustrate the concepts.

Syntax

  • Use the vertical bar (|) between two commands.
  • Programs typically receive input from the keyboard (STDIN) and output to the screen (STDOUT). Pipes allow for redirection of this flow.
  • The syntax is: command1 | command2 | command3 ..
  • The output of the first command1 becomes the input for the second (command2), and so on down the chain.
  • It's crucial to refer to the documentation because not all programs can handle piped input.

In this example, send ls command output to grep command i.e. find out if data.txt file exits or not in the current working directory):

ls | grep "data.txt"
ls -al | grep "data.txt"
  • There is no need to put spaces between command names and vertical bars, it is optional:
ls -al|grep "data.txt"
  • However, I recommend putting white spacing between the command names and vertical bars to improve the readability.
  • You can redirect pipe output to a file (output redirection with > symbol):
ps aux | grep httpd > /tmp/ps.output.log

Examples

Common shell pipe examples:

Pause ls command output

Send output of the ls command as input to the more command. So that output is printed one screen full page at a time:

ls -l | more

Show a sorted list of logged on users

Output of who command is given as input to sort command. So that it will print a sorted list of users:

who | sort
who | sort > sorted_list.txt

Count logged in users

Output of who command is given as input to wc command, so that it will number of user who logged on to the system:

who | wc -l

Find out if user vivek logged in or not

who | grep -i vivek

Count total files in current directory

ls -l | wc  -l

Execute a shutdown command at a given time

echo "shutdown -h now" | at 12am tomorrow

Format output of mount command

Display mount command output in a nice format

mount | column -t

Backup (tar over ssh)

Use tar command over secure ssh session to backup local /home file system:

tar zcvf - /home | ssh user@server "cat > /backup/home_fs.workstation_sep_21_09.tar.gz"

Case conversion

v="Unix Philosophy"; echo "$v" | tr '[:lower:]' '[:upper:]' 
echo 'tHIs IS A TeSt' | tr '[:upper:]' '[:lower:]'

Birthday Email Reminder

echo "/usr/bin/mail -s 'Birthday gift for Julia' vivek@gite.in < /dev/null" | at 17:45

Counting words

cat file.txt | grep 'something' | wc -w

Create An ISO CD Image

Create an ISO cdrom image from contents of /home/vivek/photos directory:

mkisofs -V Photos -r /home/vivek/photos | gzip -9 > /tmp/photos.iso.cd.gz

You can burn an ISO cdrom image using the following syntax:

gzip -dc /tmp/photos.iso.cd.gz | cdrecord -v dev=/dev/dvdrw -

It is also possible to create an ISO image and burn it directly to cd:

mkisofs -V Photos -r /home/vivek/photos | cdrecord -v dev=/dev/dvdrw -

Create a random password

tr -dc A-Za-z0-9_ < /dev/urandom  | head -c12 | xargs

Filtering a directory listing

ls -l | grep dir1
ls -l | grep "^d"
ls -l | grep ".pdf"

Sorting and pagination

sort filename | more
ps aux | grep 'mysql | sort -k2 | less

← PipesHomeInput redirection in pipes →