awk sed micro-primer
awk: a field-oriented pattern processing language
Awk breaks each line of input passed to it into fields. By default, a field is a string of consecutive characters delimited by whitespace, though there are options for changing this. Awk parses and operates on each separate field. This makes it ideal for handling structured text files, especially tables, or data organized into consistent chunks, such as rows and columns.
# print first field $1
# /bin/echo one two | /bin/awk '{print $1}'
one
# print second field $2
# /bin/echo one two | /bin/awk '{print $2}'
two
# print all fields $0
# /bin/echo one two | /bin/awk '{print $0}'
one two
- - - - - - - - - -
# /bin/echo one two > /tmp/output.txt
# /bin/echo three four >> /tmp/output.txt
# /bin/awk '{print $1}' /tmp/output.txt
one
three
# /bin/awk '{print $2}' /tmp/output.txt
two
four
# /bin/awk '{print $0}' /tmp/output.txt
one two
three four
Input Field Separator Variable
Awk reads and parses each line from input based on the whitespace character by default and sets the variables $1,$2 and etc. Awk FS variable is used to set the field separator for each record. Awk FS can be set to any single character or regular expression. You can use input field separator using one of the following two options:
awk -F 'FS' 'commands' inputfilename
awk 'BEGIN{FS="FS";}'
# /bin/cat /etc/passwd
root:x:0:0:root:/root:/bin/bash
- - -
# /bin/awk -F ':' '{print $1", "$6", "$7}' /etc/passwd
root, /root, /bin/bash
-or-
# /bin/awk 'BEGIN{FS=":";} {print $1", "$6", "$7}' /etc/passwd
root, /root, /bin/bash
sed: a non-interactive text file editor
sed stands for stream editor. It is a very essential tool for text processing. It is a marvelous utility that can play around regular expressions. A well-known usage of the sed command is for text replacement.
Replace Occurrences of a String With Another String
# sed 's/pattern/replace_string/' file -or- # cat file | sed 's/pattern/replace_string/' file
The default sed behavior is to write to STDOUT, use the -i option to save the changes along with the substitutions to the same file. Make sure you know what you’re doing before using -i, it might be difficult to revert file modifications that are applied; redirection is common for most users.
# sed -i 's/text/replace/' file # sed 's/text/replace/' file > newfile
The previous sed command sed 's/text/replace/' file will replace the first occurrence of the pattern in each line. In order to replace every or all occurrences, add the g parameter at the end sed 's/pattern/replace_string/g' file.