Pipes and Redirection
| pipe, >, >>, <, 2>&1, /dev/null — composing commands with streams
Skills taught: linux.pipe, linux.stdout-redirect, linux.stderr-redirect, linux.stdin-redirect, linux.dev-null
Standard streams
Every process has three streams:
0: stdin (standard input) — where it reads data from
1: stdout (standard output) — where normal output goes (usually screen)
2: stderr (standard error) — where error messages go (usually screen)
By default, both stdout and stderr appear in your terminal.
Redirection lets you send them to files or other commands.
Output redirection
# > redirects stdout to a file (OVERWRITES)
ls -la > filelist.txt # save output to file
echo "Hello" > greeting.txt
# >> appends to a file (does not overwrite)
echo "line 1" > log.txt
echo "line 2" >> log.txt # adds to log.txt
# 2> redirects stderr (errors) to a file
command 2> errors.log
# &> or 2>&1 redirects both stdout and stderr
command > output.log 2>&1 # both to same file
command &> output.log # shorthand (bash)
# Discard output (/dev/null is a black hole)
command > /dev/null # discard stdout
command 2> /dev/null # discard errors
command > /dev/null 2>&1 # discard everything
The pipe — composing commands
# | sends stdout of left command to stdin of right command
ls -la | grep ".json" # list only JSON files
cat access.log | grep "ERROR" # same as: grep "ERROR" access.log
ps aux | grep nginx # find nginx processes
# Chain multiple pipes
cat access.log | grep "404" | wc -l # count 404 errors
ls -lS | head -10 # 10 largest files
cat /etc/passwd | cut -d: -f1 | sort # sorted list of users
history | grep "git" | tail -20 # last 20 git commands
# sort and uniq for frequency analysis
cat access.log | awk '{print $1}' | sort | uniq -c | sort -rn | head -10
# → top 10 IPs by request count
# xargs: pass stdin as arguments to a command
find . -name "*.log" | xargs rm # delete found files
echo "file1 file2 file3" | xargs touch # create multiple files
Frequently Asked Questions
What is /dev/null?
/dev/null is a special file that discards anything written to it and returns EOF when read. It is used to silence output: command > /dev/null discards stdout, 2>/dev/null discards errors. Useful in cron jobs and scripts where you want commands to run silently unless they fail.