AWK-I

Awk Introduction Tutorial – 7 Awk Print Examples

Awk Introduction and Printing Operations

Awk is a programming language which allows easy manipulation of structured data and the generation of formatted reports. Awk stands for the names of its authors “Aho, Weinberger, and Kernighan”

The Awk is mostly used for pattern scanning and processing. It searches one or more files to see if they contain lines that matches with the specified patterns and then perform associated actions.

Some of the key features of Awk are:

• Awk views a text file as records and fields.
• Like common programming language, Awk has variables, conditionals and loops
• Awk has arithmetic and string operators.
• Awk can generate formatted reports

Awk reads from a file or from its standard input, and outputs to its standard output. Awk does not get along with non-text files.

Syntax:

awk '/search pattern1/ {Actions}
     /search pattern2/ {Actions}' file

In the above awk syntax:

• search pattern is a regular expression.
• Actions – statement(s) to be performed.
• several patterns and actions are possible in Awk.
• file – Input file.
• Single quotes around program is to avoid shell not to interpret any of its special characters.

Awk Working Methodology

1. Awk reads the input files one line at a time.
2. For each line, it matches with given pattern in the given order, if matches performs the corresponding action.
3. If no pattern matches, no action will be performed.
4. In the above syntax, either search pattern or action are optional, But not both.
5. If the search pattern is not given, then Awk performs the given actions for each line of the input.
6. If the action is not given, print all that lines that matches with the given patterns which is the default action.
7. Empty braces with out any action does nothing. It wont perform default printing operation.
8. Each statement in Actions should be delimited by semicolon.

Let us create employee.txt file which has the following content, which will be used in the
examples mentioned below.

$cat employee.txt
100  Thomas  Manager    Sales       $5,000
200  Jason   Developer  Technology  $5,500
300  Sanjay  Sysadmin   Technology  $7,000
400  Nisha   Manager    Marketing   $9,500
500  Randy   DBA        Technology  $6,000

Awk Example 1. Default behavior of Awk

By default Awk prints every line from the file.

$ awk '{print;}' employee.txt
100  Thomas  Manager    Sales       $5,000
200  Jason   Developer  Technology  $5,500
300  Sanjay  Sysadmin   Technology  $7,000
400  Nisha   Manager    Marketing   $9,500
500  Randy   DBA        Technology  $6,000

In the above example pattern is not given. So the actions are applicable to all the lines.
Action print with out any argument prints the whole line by default. So it prints all the
lines of the file with out fail. Actions has to be enclosed with in the braces.

Awk Example 2. Print the lines which matches with the pattern.

$ awk '/Thomas/
> /Nisha/' employee.txt
100  Thomas  Manager    Sales       $5,000
400  Nisha   Manager    Marketing   $9,500

In the above example it prints all the line which matches with the ‘Thomas’ or ‘Nisha’. It has two patterns. Awk accepts any number of patterns, but each set (patterns and its corresponding actions) has to be separated by newline.

Awk Example 3. Print only specific field.

Awk has number of built in variables. For each record i.e line, it splits the record delimited by whitespace character by default and stores it in the $n variables. If the line has 4 words, it will be stored in $1, $2, $3 and $4. $0 represents whole line. NF is a built in variable which represents total number of fields in a record.

$ awk '{print $2,$5;}' employee.txt
Thomas $5,000
Jason $5,500
Sanjay $7,000
Nisha $9,500
Randy $6,000

$ awk '{print $2,$NF;}' employee.txt
Thomas $5,000
Jason $5,500
Sanjay $7,000
Nisha $9,500
Randy $6,000

In the above example $2 and $5 represents Name and Salary respectively. We can get the Salary using  $NF also, where $NF represents last field. In the print statement ‘,’ is a concatenator.

Awk Example 4. Initialization and Final Action

Awk has two important patterns which are specified by the keyword called BEGIN and END.

Syntax: 

BEGIN { Actions}
{ACTION} # Action for everyline in a file
END { Actions }

# is for comments in Awk

Actions specified in the BEGIN section will be executed before starts reading the lines from the input.
END actions will be performed after completing the reading and processing the lines from the input.

$ awk 'BEGIN {print "Name\tDesignation\tDepartment\tSalary";}
> {print $2,"\t",$3,"\t",$4,"\t",$NF;}
> END{print "Report Generated\n--------------";
> }' employee.txt
Name Designation Department Salary
Thomas   Manager   Sales           $5,000
Jason   Developer   Technology   $5,500
Sanjay   Sysadmin   Technology   $7,000
Nisha   Manager   Marketing   $9,500
Randy   DBA     Technology   $6,000
Report Generated
--------------

In the above example, it prints headline and last file for the reports.

Awk Example 5. Find the employees who has employee id greater than 200

$ awk '$1 >200' employee.txt
300  Sanjay  Sysadmin   Technology  $7,000
400  Nisha   Manager    Marketing   $9,500
500  Randy   DBA        Technology  $6,000

In the above example, first field ($1) is employee id. So if $1 is greater than 200, then just do the default print action to print the whole line.

Awk Example 6. Print the list of employees in Technology department

Now department name is available as a fourth field, so need to check if $4 matches with the string “Technology”, if yes print the line.

$ awk '$4 ~/Technology/' employee.txt
200  Jason   Developer  Technology  $5,500
300  Sanjay  Sysadmin   Technology  $7,000
500  Randy   DBA        Technology  $6,000

Operator ~ is for comparing with the regular expressions. If it matches the default action i.e print whole line will be  performed.

Awk Example 7. Print number of employees in Technology department

The below example, checks if the department is Technology, if it is yes, in the Action, just increment the count variable, which was initialized with zero in the BEGIN section.

$ awk 'BEGIN { count=0;}
$4 ~ /Technology/ { count++; }
END { print "Number of employees in Technology Dept =",count;}' employee.txt
Number of employees in Tehcnology Dept = 3

Then at the end of the process, just print the value of count which gives you the number of employees in Technology department.

Awk Tutorial: Understand Awk Variables with 3 Practical Examples

In this article let us review how to define and use awk variables.

• Awk variables should begin with the letter, followed by it can consist of alpha numeric characters or underscore.
• Keywords cannot be used as a awk variable
• Awk does not support variable declaration like other programming languages
• Its always better to initialize awk variables in BEGIN section, which will be executed only once in the beginning.
• There are no datatypes in Awk. Whether a awk variable is to be treated as a number or as a string depends on the context it is used in.

Now let us review few simple examples to learn how to use user-defined awk variables.

Awk Example 1: Billing for Books

In this example, the input file bookdetails.txt contains records with fields — item number, Book name, Quantity and Rate per book.

$ cat bookdetails.txt
1 Linux-programming 2 450
2 Advanced-Linux 3 300
3 Computer-Networks 4 400
4 OOAD&UML 3 450
5 Java2 5 200

Now the following Awk script, reads and processes the above bookdetails.txt file, and generates report that displays — rate of each book sold, and total amount for all the books sold.

So far we have seen Awk reads the commands from the command line, but Awk can also read the commands from the file using -f option.

Syntax:

$ awk -f script-filename inputfilename

Now our Awk script for billing calculation for books is given below.

$ cat book-calculation.awk
BEGIN {
 total=0;
}
{
 itemno=$1;
 book=$2;
 bookamount=$3*$4;
 total=total+bookamount;
 print itemno," ", book,"\t","$"bookamount;
}
END {
 print "Total Amount = $"total;
}

In the above script,

• Awk BEGIN section initializes the variable total. itemno, total, book, bookamount are userdefined awk variables.
• In the Awk Action section, Quantity*bookprice will be stored in a variable called bookamount. Each bookamount will be added with the total.
• Finally in the Awk END section, total variable will have total amount.

Now execute the book-calculation.awk script to generate the report that displays each book rate and total amount as shown below.

$ awk -f book-calculation.awk bookdetails.txt
1   Linux-programming   $900
2   Advanced-Linux   $900
3   Computer-Networks   $1600
4   OOAD&UML   $1350
5   Java2   $1000
Total Amount = $5750

Awk Example 2. Student Mark Calculation

In this example, create an input file “student-marks.txt” with the following content — Student name, Roll Number, Test1 score, Test2 score and Test3 score.

$ cat student-marks.txt
Jones 2143 78 84 77
Gondrol 2321 56 58 45
RinRao 2122 38 37 65
Edwin 2537 78 67 45
Dayan 2415 30 47 20

Now the following Awk script will calculate and generate the report to show the Average marks of each student, average of Test1, Test2 and Test3 scores.

$cat student.awk

BEGIN {
 test1=0;
 test2=0;
 test3=0;
 print "Name\tRollNo\t Average Score";

}
{
 total=$3+$4+$5;
 test1=test1+$3;
 test2=test2+$4;
 test3=test3+$5;
 print $1"\t"$2"\t",total/3;
}
END{
 print "Average of Test1="test1/NR;
 print "Average of Test2="test2/NR;
 print "Average of Test3="test3/NR;

}

In the above Awk script,

• In the Awk BEGIN section all the awk variables are initialized to zero. test1, test2, test3 and total are user-defined awk variables.
• In the Awk ACTION section, $3, $4, $5 are Test1, Test2 and Test3 scores respectively. total variable is the addition of 3 test scores for each student. The awk variable test1, test2 and test3 has the total scores of each corresponding test.
• So in the Awk END section, dividing each test total by total number of records (i.e student) will give you the average score.  NR is an Awk built-in variable which gives total number of records in input.

Awk Example 3. HTML Report for Student Details

In the above two example, we have seen awk variable which has numbers as its values. This example shows awk script to generate the html report for the students name and their roll number.

$ cat string.awk
BEGIN{
title="AWK";
print "\n\n";

}
{
name=$1;
rollno=$2;
print "";

}
END {
    print "

Student Details
"name"
"rollno"
\n";
}

Use the same student-marks.txt input file that we created in the above example.

$ awk -f string.awk student-marks.txt



Student Details
 
Jones
2143
Gondrol
2321
RinRao
2122
Edwin
2537
Dayan
2415

We can store the above output, which gives the following html table. In the above script, variable called name and rollno are string variable, because it is used in string context.

Student Details
Jones
2143
Gondrol
2321
RinRao
2122
Edwin
2537
Dayan
2415