bash Cookbook

Updating Specific Fields in Data Files

Problem

You need to extract certain parts (fields) of a line (record) and update them.

Solution

In the simple case, you want to extract a single field from a line, then perform some operation on it. For that, you can use cut or awk. See Isolating Specific Fields in Data for details.

For the more complicated case, you need to modify a field in a data file without extracting it. If it’s a simple search and replace, use sed.

For example, let’s switch everyone from csh to sh on this NetBSD system.

$ grep csh /etc/passwd
root:*:0:0:Charlie &:/root:/bin/csh

$ sed 's/csh$/sh/' /etc/passwd | grep '^root'
root:*:0:0:Charlie &:/root:/bin/sh

You can use awk if you need to do arithmetic on a field or modify a string only in a certain field:

$ cat data_file
Line 1 ends
Line 2 ends
Line 3 ends
Line 4 ends
Line 5 ends


$ awk '{print $1, $2+5, $3}' data_file
Line 6 ends
Line 7 ends
Line 8 ends
Line 9 ends
Line 10 ends


# If the second field contains '3', change it to '8' and mark it
$ awk '{ if ($2 == "3") print $1, $2+5, $3, "Tweaked" ; else print $0; }' data_file
Line 1 ends
Line 2 ends
Line 8 ends Tweaked
Line 4 ends
Line 5 ends

Discussion

The possibilities here are as endless as your data, but hopefully the examples above will give you enough of a start to easily modify your data.

See Also

• man awk

• man sed

• http://sed.sourceforge.net/sedfaq.html

• http://sed.sourceforge.net/sed1line.txt

• Figuring Out Date and Time Arithmetic

• Isolating Specific Fields in Data