Stream Editing using sed command

The sed command is a powerful tool in Linux that is used to edit and manipulate text files in a variety of ways. It is a stream editor, which means that it reads input from a file or standard input, makes the specified changes, and writes the output to a file or standard output.

The sed command is a powerful tool for performing search and replace operations on text. Here are a few examples of some of the most common and useful ways that you can use sed:

  1. Replacing specific text: sed can be used to replace specific words or patterns in the input text. For example, the following sed command would replace the first occurrence of the word “apple” with the word “orange” in the file fruits.txt:
    sed 's/apple/orange/' fruits.txt
  2. Deleting lines: sed can be used to delete specific lines or ranges of lines from the input text. For example, the following sed command would delete the first line in the file text.txt:
    sed '1d' text.txt
  3. Inserting text: sed can be used to insert text into the input text at a specific line number or position. For example, the following sed command would insert the word “banana” after the second line in the file fruits.txt:
    sed '2a banana' fruits.txt
  4. Modifying multiple files: sed can be used to perform search and replace operations on multiple files at once. For example, the following sed command would replace the word “apple” with the word “orange” in all .txt files in the current directory:
    sed -i 's/apple/orange/g' *.txt
  5. Replacing multiple patterns: sed can be used to replace multiple patterns in the input text with a single command. For example, the following sed command would replace the words “apple” and “banana” with the word “orange” in the file fruits.txt:
  6. sed 's/apple/orange/g; s/banana/orange/g' fruits.txt
  7. Sorting text: sed can be used in conjunction with the sort command to sort the input text. For example, the following sed command would sort the lines in the file fruits.txt in alphabetical order:
    sed -n 's/^\(.*\)$/\L\1/p' fruits.txt | sort
  8. Extracting data: sed can be used to extract specific data from the input text. For example, the following sed command would extract all email addresses from the file emails.txt:
    sed -n 's/.*\(\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]\{2,\}\b\).*/\1/p' emails.txt

One of the most common uses of sed is to perform search and replace operations on text files. You can use the s command to search for a pattern and replace it with a different string. For example, the following command would replace all occurrences of the word “apple” with the word “orange” in the file fruits.txt:

$ sed 's/apple/orange/g' fruits.txt

The g flag at the end of the s command tells sed to perform the replacement globally, which means that it will replace all occurrences of the pattern rather than just the first one.

You can also use sed to delete lines from a text file that match a certain pattern. For example, the following command would delete all lines that contain the word “apple” in the file fruits.txt:

$ sed '/apple/d' fruits.txt

In addition to these basic operations, sed has a number of other useful features that allow you to manipulate text in more advanced ways. For example, you can use the y command to translate characters from one set to another, the a command to append text to the end of a line, and the i command to insert text before a line.

You can also use sed to perform multiple operations in a single command by separating them with a semicolon (;). For example, the following command would delete all lines that contain the word “apple” and replace all occurrences of the word “banana” with the word “kiwi” in the file fruits.txt:
$ sed '/apple/d; s/banana/kiwi/g' fruits.txt

One of the strengths of sed is its ability to perform these operations on a large number of files in a script or shell pipeline. For example, you could use the find command to locate all text files in a directory tree and then pass them through sed to perform the same set of operations on each file:

$ find /path/to/directory -type f -name "*.txt" -exec sed '/apple/d; s/banana/kiwi/g' {} ;

The sed command allows you to specify instructions that tell it what operations to perform on the input text. These instructions can be specified using a combination of commands and flags.

Here are some common commands that are used in sed instructions:

  • s: This command is used to perform search and replace operations on the input text. It has the following syntax: s/search_pattern/replace_pattern/flags. The search_pattern and replace_pattern are separated by a forward slash (/), and the flags are optional.
  • y: This command is used to translate characters from one set to another. It has the following syntax: y/set1/set2/. The set1 and set2 are separated by a forward slash (/), and each character in set1 is translated to the corresponding character in set2.
  • a: This command is used to append text to the end of a line. It has the following syntax: atext_to_append. The character is used to escape the newline character, and the text_to_append is the text that you want to append to the end of the line.
  • i: This command is used to insert text before a line. It has the same syntax as the a command.
  • c: This command is used to change the text of a line. It has the following syntax: ctext_to_change_to. The character is used to escape the newline character, and the text_to_change_to is the text that you want to use to replace the original line.
  • d: This command is used to delete a line. It does not require any additional arguments.

Here are some common flags that are used in sed instructions:

  • g: This flag is used at the end of the s command to perform the replacement globally, which means that it will replace all occurrences of the search pattern rather than just the first one.
  • p: This flag is used at the end of the s command to print the replaced line.
  • n: This flag is used at the end of the p command to print the line number.
  • N: This flag is used at the end of the p command to print the line number and the line itself.

sed using regular expressions for search

The s instruction is one of the most commonly used commands in the sed command-line tool. It is used to perform search and replace operations on the input text.

Here is the basic syntax of the s instruction:

s/search_pattern/replacement/flags

The search_pattern is a regular expression that specifies the pattern to search for in the input text. The replacement is the text that will be used to replace the search pattern. The flags are optional and can be used to modify the behavior of the s command.

Here are a few examples of the s command in action:

  • sed 's/apple/orange/' text.txt: This command would replace the first occurrence of the word “apple” with the word “orange” in the file text.txt.
  • sed 's/apple/orange/g' text.txt: This command would replace all occurrences of the word “apple” with the word “orange” in the file text.txt.
  • sed 's/[Aa]pple/orange/' text.txt: This command would replace the first occurrence of the word “apple” or “Apple” with the word “orange” in the file text.txt.

Regular expressions can be used in the sed command to search for and match specific patterns in the input text. Regular expressions are a powerful tool for matching and manipulating text, and they are particularly useful when working with sed.

Here are a few examples of how you can use regular expressions in the sed command:

  • sed 's/a[bc]d/xyz/' text.txt: This command would replace the first occurrence of “abd” or “acd” with “xyz” in the file text.txt.
  • sed 's/[0-9]{3}/xxx/' text.txt: This command would replace the first occurrence of three consecutive digits with “xxx” in the file text.txt.
  • sed 's/btheb/a/' text.txt: This command would replace the first occurrence of the word “the” surrounded by word boundaries with the letter “a” in the file text.txt.

Extended regular expressions, also known as “extended regex” or “eregex,” are a variant of regular expressions that provide additional features and syntax. They can be used in the sed command to match and manipulate text using more advanced patterns.

To use extended regular expressions in sed, you can use the -r option or the -E option. For example, the following sed command would use extended regular expressions to replace all occurrences of the word “apple”, “Apple”, “banana” or “Banana” with the word “orange” in the file fruits.txt:

$ sed -r 's/([Aa]pple|[Bb]anana/orange/g' fruits.txt
Here are a few examples of extended regular expression syntax that you can use in sed:
  • (abc|def): This pattern matches either “abc” or “def”.
  • (a|b)+: This pattern matches one or more repetitions of “a” or “b”.
  • (a|b)?: This pattern matches zero or one occurrences of “a” or “b”.
  • (a|b){3}: This pattern matches exactly three occurrences of “a” or “b”.
  • (a|b){3,}: This pattern matches three or more occurrences of “a” or “b”.
  • (a|b){3,5}: This pattern matches between three and five occurrences of “a” or “b”.

The & character is a special placeholder that can be used in the replacement string of the s (search and replace) command in sed. It represents the entire matched search pattern.

For example, the following sed command would replace all occurrences of the word “apple” with the word “orange” in the file fruits.txt:

$ sed 's/apple/orange/g' fruits.txt

If you wanted to include the matched search pattern in the replacement string, you could use the & character as a placeholder. For example, the following sed command would replace all occurrences of the word “apple” with the word “apple orange” in the file fruits.txt:

$ sed 's/apple/& orange/g' fruits.txt

This can be particularly useful when you want to modify only part of the matched search pattern.

sed line selection

The sed command allows you to select specific lines in the input text using a range of line numbers or patterns. Here are a few ways that you can select lines using sed:

  • To select a specific line, you can use the line number preceded by a p command. For example, the following sed command would print line 3 in the file text.txt:
    $ sed -n '3p' text.txt
  • To select a range of lines, you can use the line numbers separated by a comma. For example, the following sed command would print lines 3 to 6 in the file text.txt:
    $ sed -n '3,6p' text.txt
  • To select all lines up to a certain line number, you can use the 1,line_number range. For example, the following sed command would print all lines up to and including line 6 in the file text.txt:
    $ sed -n '1,6p' text.txt
  • To select all lines after a certain line number, you can use the line_number,$ range. For example, the following sed command would print all lines after and including line 6 in the file text.txt:
    $ sed -n '6,$p' text.txt
  • To select lines that match a certain pattern, you can use the /pattern/p command. For example, the following sed command would print all lines that contain the word “apple” in the file fruits.txt:
    $ sed -n '/apple/p' fruits.txt

sed command using p flag to print

The p flag is used in the sed command to print the replaced line after a search and replace operation. It is typically used in conjunction with the s command, which is used to perform search and replace operations on text.

For example, the following sed command would search for all occurrences of the word “apple” in the file fruits.txt and replace them with the word “orange”. It would then print the replaced line:

$ sed 's/apple/orange/p' fruits.txt

The p flag can be used in combination with other flags, such as the g flag, which tells sed to perform the replacement globally. For example, the following sed command would replace all occurrences of the word “apple” with the word “orange” and print the replaced line:

$ sed 's/apple/orange/pg' fruits.txt

You can also use the p flag in combination with the n or N flags to print the line number along with the replaced line. The n flag prints the line number, while the N flag prints the line number and the line itself.

Here are a few examples of sed commands that use the p flag with the n and N flags:

  • sed 's/apple/orange/pn' fruits.txt: This command would replace all occurrences of the word “apple” with the word “orange” and print the line number of the replaced line.
  • sed 's/apple/orange/pN' fruits.txt: This command would replace all occurrences of the word “apple” with the word “orange” and print the line number and the replaced line.

The -n flag is used in the sed command to suppress the default behavior of printing every line in the input text. When the -n flag is used, sed will only print lines that are specifically instructed to be printed using the p flag or the P flag.

For example, the following sed command would print every line in the file text.txt:

$ sed 's/apple/orange/' text.txt

However, if you add the -n flag to the command, sed will only print the lines that are specifically instructed to be printed using the p or P flag:

$ sed -n 's/apple/orange/p' text.txt

In this case, sed will only print the lines that contain the word “apple” and have been replaced with the word “orange”.

The p flag is used to print the replaced line after a search and replace operation. It is typically used in conjunction with the s command, which is used to perform search and replace operations on text.

For example, the following sed command would search for all occurrences of the word “apple” in the file fruits.txt and replace them with the word “orange”. It would then print the replaced line:

$ sed 's/apple/orange/p' fruits.txt

The P flag is similar to the p flag, but it only prints the portion of the line that was matched by the search pattern.

For example, the following sed command would search for all occurrences of the word “apple” in the file fruits.txt and replace them with the word “orange”. It would then print the matched portion of the line (i.e. “apple”):

$ sed 's/apple/orange/P' fruits.txt

sed command using d flag to delete

The sed command can be used to remove specific lines or patterns from the input text. Here are a few ways that you can use sed to remove lines or patterns from the input text:

  • To remove a specific line, you can use the d command followed by the line number. For example, the following sed command would delete line 3 in the file text.txt:
    sed '3d' text.txt
  • To remove a range of lines, you can use the line numbers separated by a comma. For example, the following sed command would delete lines 3 to 6 in the file text.txt:
    sed '3,6d' text.txt
  • To remove all lines up to a certain line number, you can use the 1,line_number range. For example, the following sed command would delete all lines up to and including line 6 in the file text.txt:
    sed '1,6d' text.txt
  • To remove all lines after a certain line number, you can use the line_number,$ range. For example, the following sed command would delete all lines after and including line 6 in the file text.txt:
    sed '6,$d' text.txt
  • To remove lines that match a certain pattern, you can use the /pattern/d command. For example, the following sed command would delete all lines that contain the word “apple” in the file fruits.txt:
    sed '/apple/d' fruits.txt

The d flag is used in the sed command to delete lines that match a certain pattern. It is typically used in combination with a search pattern to specify which lines should be deleted.

For example, the following sed command would delete all lines that contain the word “apple” in the file fruits.txt:

$ sed '/apple/d' fruits.txt

The d flag can be used in combination with other flags or commands to perform more advanced operations. For example, the following sed command would delete all lines that contain the word “apple” and replace all occurrences of the word “banana” with the word “kiwi” in the file fruits.txt:

$ sed '/apple/d; s/banana/kiwi/g' fruits.txt

You can also use the d flag in combination with the n flag to delete every nth line in the input text. For example, the following sed command would delete every other line in the file text.txt:

$ sed 'n; d' text.txt

sed command useful options

The sed command has a number of options that can be used to control its behavior. Here are a few common options that you might use with sed:

  • -n: This option tells sed to suppress the default behavior of printing every line in the input text. When the -n option is used, sed will only print lines that are specifically instructed to be printed using the p or P flags.
  • -e: This option allows you to specify multiple instructions in a single sed command. For example, the following sed command would delete all lines that contain the word “apple” and replace all occurrences of the word “banana” with the word “kiwi” in the file fruits.txt:
    sed -e '/apple/d' -e 's/banana/kiwi/g' fruits.txt
  • -f: This option allows you to specify a file containing sed instructions, rather than specifying them on the command line. For example, the following sed command would execute the instructions in the file instructions.sed on the file text.txt:
    sed -f instructions.sed text.txt
  • -i: This option tells sed to edit the input file in place, rather than printing the modified output to the console. For example, the following sed command would replace all occurrences of the word “apple” with the word “orange” in the file fruits.txt and save the changes to the file:
    sed -i 's/apple/orange/g' fruits.txt

License

Developers ultimate guide: Linux Bash scripting Copyright © 2022 by Matin Maleki. All Rights Reserved.

Share This Book