Home | Tip of the Week | Tip of the Week Archive | Partners | Clients | History | Friends

Tip of the Week (Archived)
"But All I Wanted Was Yes or No, Did It Run or Not!"
sed and awk to the Rescue

The request was a simple one. A job ran every night, moving transaction data from an application database to a new database that allowed users to view the source documents. The project manager asked for email to be sent so he would know if the job ran successfully or not.

The job consisted of two PL/SQL procedures in a database package being run from a SQL script on a Unix server. A log file was already being produced, but it was not being sent to anyone and only contained the following output:

PL/SQL procedure successfully completed.

PL/SQL procedure successfully completed.

As the developer of this process, I knew exactly what should be in the email to be sent. A couple of hours of development, and the log file now contained the following (data changed to protect the innocent):

Create repository start time 02-MAY-2006 06:11:11                                
A01
N11
New repository count 2                                                        
Create repository end time 02-MAY-2006 06:11:12                                  

PL/SQL procedure successfully completed.

New data start time        02-MAY-2006 06:11:12                                  
Records read               19330                                                
Records written            19300                                               
Errors                     6                                               
Ignored                    24                                                   
New data end time          02-MAY-2006 06:12:25                                
--                                                                             
Update start time          02-MAY-2006 06:12:25                                 
Records read               317                                               
Records written            314                                               
Errors                     3                                                 
Update end time            02-MAY-2006 06:12:28                                  

PL/SQL procedure successfully completed.

Now the log contains a lot of information...how long the job ran, how much data was processed, errors that might need to be researched, etc....lots of information  I proudly forwarded the new log file to the IT manager, who approved and forwarded it to the project manager.

And the response was..."But all I wanted was yes or no, did it run or not!" (Philosophical question...If the job does not run at all, how does it send notification of said fact...but I digress. The sed / awk one line thing is still pretty cool.)

The IT manager and I talked about the fact that success or failure could easily be determined from this log and that it was important to know that errors had occurred. So we decided to have further discussions with the project manager about the log. Half jokingly, I suggested that I was sure I could write something really techie, using awk or sed, to parse the file and return either yes or no in an email. And the answer was, "You have my approval. Have fun."

With permission to turn my inner geek loose, I decided not only to make it happen, but to do it all in one line, with the following result.

sed -n '/'successfully'/p' job.log | wc -l | awk '{if($1 == 2) print "yes"; else print "no"}' | mailx -s "Yes or No" user@yourcompany.com

This line works as follows

  1. sed -n '/'successfully'/p' job.log - sed reads the file job.log and prints only the lines that contain the string successfully. Since there are two PL/SQL steps in this process, success means the string is found in two lines. Anything else means the process did not run successfully. The lines retrieved by sed are passed to step 2..

  2. wc -l - Count the number of lines passed by step 1. The count is passed to step 3.

  3. awk '{if($1 == 2) print "yes"; else print "no"}' - If the count from step 2 equals 2, then print the string yes. If the count is anything else, print the string no. This result is passed to step 4.

  4. mailx -s "Yes or No" user@yourcompany.com - The output from step 3, a single word yes or no, becomes the body of an email sent to user@yourcompany.com with the subject line Yes or No.

A challenge well met, and sed and awk used in the same line. It was a very good day.

Note: This tip was tested using Solaris 9, korn shell.

Was this tip useful? Did you find any errors? Do you have any suggestions? Do you care? Click here for the tip feedback page. Thank you.

2000 Turnberry Circle, Glenmoore, PA 19343
Voice: (610) 942-1979
Fax: (610) 942-1990
Email


© 2006 Alydan Consulting, Inc.