Dynamic conversion of epoch timestamps in logs

In the course of your logs or text processing, you may come across certain timestamps in epoch format.  Whilst there’s always online resources to assist with the conversion of such timestamps, it may not be the best way if you need to keep the timestamp “secret” during then, or if you have many timestamps to convert going by the thousands, millions, etc.

Whilst there’s always free tools like Splunk which is available for free to the masses (and yes, it does automatically convert epoch timestamps for you), there’s always our “humble” awk. 😀

The linux awk command has the ability to invoke other commands as part of its computation.  The date command can be used to convert epoch times to local times.  Putting both together would allow us to do just what we need here!

First some examples with the date command:

$ date -d @1280921130.313
Wed Aug  4 19:25:30 SGT 2010

Or should we want to get the dates only:

$ date -d @1280921130.313 +%D

Now, making use of awk to convert only one epoch timestamp:

$ echo -n "1280921130.313" | 
awk '{"date -d @"$1" +%D" | getline myvariable; print myvariable}'

The important part to note is that we must enclose the “external” command in quotes (we use the unquoted $1 variable to pass the epoch timestamp from awk), and that we pipe the output of that command to the getline directive in awk.  getline by itself would replace the $0 variable in awk when referencing it subsequently, whereas specifying a variable (“myvariable” in this example) would keep the $0 variable as it is, allowing you to use the variable to reference the output of the external command.

Final example showing how logs preprocessing using these commands might look like:

$ cat sample.log
1280921130.313 logentry1
1280921131.313 logentry2
1280921132.313 logentry3
1280921133.313 logentry4
$ cat sample.log | 
awk '{"date -d @"$1 | getline myvariable2; print myvariable2 "t" $0}'
Wed Aug  4 19:25:30 SGT 2010    1280921130.313 logentry1
Wed Aug  4 19:25:31 SGT 2010    1280921131.313 logentry2
Wed Aug  4 19:25:32 SGT 2010    1280921132.313 logentry3
Wed Aug  4 19:25:33 SGT 2010    1280921133.313 logentry4

Have fun!


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s