Menu

UNIX TUTORIALS - Unix - Regular Expressions

Unix - Regular Expressions

ADVERTISEMENTS

The sed Genral Syntax:

RangeDescription
pPrints the line
dDeletes the line
s/pattern1/pattern2/Substitutes the first occurrence of pattern1 with pattern2.

ADVERTISEMENTS

The sed Address Ranges:

RangeDescription
'4,10d'Lines starting from 4th till 10th are deleted
'10,4d'Only 10th line is deleted, because sed does not work in reverse direction.
'4,+5d'This will match line 4 in the file, delete that line, continue to delete the next five lines, and then cease its deletion and print the rest
'2,5!d'This will deleted everything except starting from 2nd till 5th line.
'1~3d'This deletes the first line, steps over the next three lines, and then deletes the fourth line. Sed continues applying this pattern until the end of the file.
'2~2d'This tells sed to delete the second line, step over the next line, delete the next line, and repeat until the end of the file is reached.
'4,10p'Lines starting from 4th till 10th are printed
'4,d'This would generate syntax error.
',10d'This would also generate syntax error.

ADVERTISEMENTS

Substitution Flags:

FlagDescription
gReplace all matches, not just the first match.
NUMBERReplace only NUMBERth match.
pIf substitution was made, print pattern space.
w FILENAMEIf substitution was made, write result to FILENAME.
I or iMatch in a case-insensitive manner.
M or mIn addition to the normal behavior of the special regular expression characters ^ and $, this flag causes ^ to match the empty string after a newline and $ to match the empty string before a newline.

Using Regular Expression:

CharacterDescription
^Matches the beginning of lines.
$Matches the end of lines.
.Matches any single character.
*Matches zero or more occurrences of the previous character
[chars]Matches any one of the characters given in chars, where chars is a sequence of characters. You can use the - character to indicate a range of characters.

Matching Characters:

ExpressionDescription
/a.c/Matches lines that contain strings such as a+c, a-c, abc, match, and a3c, whereas the pattern
/a*c/Matches the same strings along with strings such as ace, yacc, and arctic.
/[tT]he/Matches the string The and the:
/^$/Matches Blank lines
/^.*$/Matches an entire line whatever it is.
/ */Matches one or more spaces
/^$/Matches Blank lines

SetDescription
[a-z]Matches a single lowercase letter
[A-Z]Matches a single uppercase letter
[a-zA-Z]Matches a single letter
[0-9]Matches a single number
[a-zA-Z0-9]Matches a single letter or number

Character Class Keywords:

Character ClassDescription
[[:alnum:]]Alphanumeric [a-z A-Z 0-9]
[[:alpha:]]Alphabetic [a-z A-Z]
[[:blank:]]Blank characters (spaces or tabs)
[[:cntrl:]]Control characters
[[:digit:]]Numbers [0-9]
[[:graph:]]Any visible characters (excludes whitespace)
[[:lower:]]Lowercase letters [a-z]
[[:print:]]Printable characters (noncontrol characters)
[[:punct:]]Punctuation characters
[[:space:]]Whitespace
[[:upper:]]Uppercase letters [A-Z]
[[:xdigit:]]Hex digits [0-9 a-f A-F]

Invoking sed:

$ cat /etc/passwd | sed
Usage: sed [OPTION]... {script-other-script} [input-file]...

  -n, --quiet, --silent
                 suppress automatic printing of pattern space
  -e script, --expression=script
...............................

The sed Genral Syntax:

/pattern/action

Deleting All Lines with sed:

$ cat /etc/passwd | sed 'd'
$

$ sed -e 'd' /etc/passwd
$

The sed Addresses:

$ cat /etc/passwd | sed '1d' |more
daemon:x:1:1:daemon:/usr/sbin:/bin/sh
bin:x:2:2:bin:/bin:/bin/sh
sys:x:3:3:sys:/dev:/bin/sh
sync:x:4:65534:sync:/bin:/bin/sync
games:x:5:60:games:/usr/games:/bin/sh
man:x:6:12:man:/var/cache/man:/bin/sh
mail:x:8:8:mail:/var/mail:/bin/sh
news:x:9:9:news:/var/spool/news:/bin/sh
backup:x:34:34:backup:/var/backups:/bin/sh
$

The sed Address Ranges:

$ cat /etc/passwd | sed '1, 5d' |more
games:x:5:60:games:/usr/games:/bin/sh
man:x:6:12:man:/var/cache/man:/bin/sh
mail:x:8:8:mail:/var/mail:/bin/sh
news:x:9:9:news:/var/spool/news:/bin/sh
backup:x:34:34:backup:/var/backups:/bin/sh
$

$ cat /etc/passwd | sed -n '1,3p'

$ cat /etc/passwd | sed '1,3p'

The Substitution Command:

$ cat /etc/passwd | sed 's/root/amrood/'
amrood:x:0:0:root user:/root:/bin/sh
daemon:x:1:1:daemon:/usr/sbin:/bin/sh
..........................

$ cat /etc/passwd | sed 's/root/amrood/g'
amrood:x:0:0:amrood user:/amrood:/bin/sh
daemon:x:1:1:daemon:/usr/sbin:/bin/sh
bin:x:2:2:bin:/bin:/bin/sh
sys:x:3:3:sys:/dev:/bin/sh
...........................

Using an Alternative String Separator:

$ cat /etc/passwd | sed 's:/root:/amrood:g'
amrood:x:0:0:amrood user:/amrood:/bin/sh
daemon:x:1:1:daemon:/usr/sbin:/bin/sh

Replacing with Empty Space:

$ cat /etc/passwd | sed 's/root//g'
:x:0:0::/:/bin/sh
daemon:x:1:1:daemon:/usr/sbin:/bin/sh

Address Substitution:

$ cat /etc/passwd | sed '10s/sh/quiet/g'
root:x:0:0:root user:/root:/bin/sh
daemon:x:1:1:daemon:/usr/sbin:/bin/sh
bin:x:2:2:bin:/bin:/bin/sh
sys:x:3:3:sys:/dev:/bin/sh
sync:x:4:65534:sync:/bin:/bin/sync
games:x:5:60:games:/usr/games:/bin/sh
man:x:6:12:man:/var/cache/man:/bin/sh
mail:x:8:8:mail:/var/mail:/bin/sh
news:x:9:9:news:/var/spool/news:/bin/sh
backup:x:34:34:backup:/var/backups:/bin/quiet

$ cat /etc/passwd | sed '1,5s/sh/quiet/g'
root:x:0:0:root user:/root:/bin/quiet
daemon:x:1:1:daemon:/usr/sbin:/bin/quiet
bin:x:2:2:bin:/bin:/bin/quiet
sys:x:3:3:sys:/dev:/bin/quiet
sync:x:4:65534:sync:/bin:/bin/sync
games:x:5:60:games:/usr/games:/bin/sh
man:x:6:12:man:/var/cache/man:/bin/sh
mail:x:8:8:mail:/var/mail:/bin/sh
news:x:9:9:news:/var/spool/news:/bin/sh
backup:x:34:34:backup:/var/backups:/bin/sh

The Matching Command:

$ cat testing | sed -n '/root/p'
root:x:0:0:root user:/root:/bin/sh
[root@ip-72-167-112-17 amrood]# vi testing
root:x:0:0:root user:/root:/bin/sh
daemon:x:1:1:daemon:/usr/sbin:/bin/sh
bin:x:2:2:bin:/bin:/bin/sh
sys:x:3:3:sys:/dev:/bin/sh
sync:x:4:65534:sync:/bin:/bin/sync
games:x:5:60:games:/usr/games:/bin/sh
man:x:6:12:man:/var/cache/man:/bin/sh
mail:x:8:8:mail:/var/mail:/bin/sh
news:x:9:9:news:/var/spool/news:/bin/sh
backup:x:34:34:backup:/var/backups:/bin/sh

Using Regular Expression:

$ cat testing | sed '/^daemon/d'
root:x:0:0:root user:/root:/bin/sh
bin:x:2:2:bin:/bin:/bin/sh
sys:x:3:3:sys:/dev:/bin/sh
sync:x:4:65534:sync:/bin:/bin/sync
games:x:5:60:games:/usr/games:/bin/sh
man:x:6:12:man:/var/cache/man:/bin/sh
mail:x:8:8:mail:/var/mail:/bin/sh
news:x:9:9:news:/var/spool/news:/bin/sh
backup:x:34:34:backup:/var/backups:/bin/sh

$ cat testing | sed '/sh$/d'
sync:x:4:65534:sync:/bin:/bin/sync

Character Class Keywords:

$ cat /etc/syslog.conf | sed -n '/^[[:alpha:]]/p'
authpriv.*                         /var/log/secure
mail.*                             -/var/log/maillog
cron.*                             /var/log/cron
uucp,news.crit                     /var/log/spooler
local7.*                           /var/log/boot.log

Aampersand Referencing:

5555551212
5555551213
5555551214
6665551215
6665551216
7775551217

$ sed -e 's/^[[:digit:]][[:digit:]][[:digit:]]/(&)/g' phone.txt
(555)5551212
(555)5551213
(555)5551214
(666)5551215
(666)5551216
(777)5551217

Using Multiple sed Commands:

$ sed -e 'command1' -e 'command2' ... -e 'commandN' files

$ sed -e 's/^[[:digit:]]\{3\}/(&)/g'  \
                      -e 's/)[[:digit:]]\{3\}/&-/g' phone.txt
(555)555-1212
(555)555-1213
(555)555-1214
(666)555-1215
(666)555-1216
(777)555-1217

Back References:

(555)555-1212
(555)555-1213
(555)555-1214
(666)555-1215
(666)555-1216
(777)555-1217

$ cat phone.txt | sed 's/\(.*)\)\(.*-\)\(.*$\)/Area \
                       code: \1 Second: \2 Third: \3/'
Area code: (555) Second: 555- Third: 1212
Area code: (555) Second: 555- Third: 1213
Area code: (555) Second: 555- Third: 1214
Area code: (666) Second: 555- Third: 1215
Area code: (666) Second: 555- Third: 1216
Area code: (777) Second: 555- Third: 1217