Convert Epoch command history logs into human readable format

Today we're going to discuss on how to customize user's command history logging and on converting Epoch command history logs into human readable history logs

In short

• Customize C shell and Bash shell user's command history logging to store date-time of command executed
• Store command history logs in different directory (not in user's home dir)
• Store 20,000 lines of cmd executed into log file and allow system to remember 5000 cmds run at one time
• Convert Epoch time to human readable time for all command history logs
• Set up cron job to schedule conversion and housekeeping

Please ensure you have the target directories and empty log files already created to store the command history logs. On my server I store them in /var/log. Oh, and another thing, you got to make sure that the file's ownership is set to the user, if it belongs to root user the log files won't get written unto.

Cmd history logs

#/var/log/user_history/username_history

Epoch convert logs

#/var/log/user_history/Epoch_Convert/username

Lets start with the options I used to customise the cmd history logging. I edited the user's .bashrc or .cshrc profiles and add the following lines shown below at the end of their profiles. On some cases, you get users that have both .cshrc and .bashrc profiles, so to clear the confusion on which profile to edit, just do a "finger" on the user and you will find out its true shell profile.

Bash profile

export HISTFILE=/var/log/user_history/$(whoami)_history

export HISTFILESIZE=20000

export HISTSIZE=5000

export HISTTIMEFORMAT="%F %T "

If you're lazy then you can also make changes to the system wide profile, that is the .bashrc located in /etc/skel [for SLES] Make sure you take a backup copy of the profile before you start clobbering it

C shell Profile

echo "Date is: `date`" >> /var/log/user_history/sapadm_history

set histfile=/var/log/user_history/sapadm_history

set history=5000

set savehist=20000

I have also conveniently setup a script that stores all users' command history logging and put a cronjob to it, it runs once every end of the month to copy the logs into a month-stamped directory then nullifies the old log files

#!/bin/bash

HIST_DIR="/var/log/user_history"    #where my users' cmd history logs go to

MNTH_DEST_DIR="/var/log/user_history/old_logs/$(date +'%m')"    #where my users' cmd history logs go to end of the month

mkdir -p "${MNTH_DEST_DIR}"  #create an empty month directory (i.e 03 for March)

cp -p ${HIST_DIR}/*history ${MNTH_DEST_DIR}

cd ${HIST_DIR}

for i in /var/log/user_history/*history;  do    #I nullify the old logs after copying them to monthly dirs

  :>"$i"

done

While "history" command results in cmd history log with readable time stamp on it, when it gets saved the time stamps are automatically converted into Epoch/Unix time, so when the logs are saved they are all stored in seconds from 1 January 1970, so reading them is really cumbersome. Lets have a look into my cmd history log for root user from before and after conversion

#pwd

 /var/log/user_history/root_history

# tail root_history
#1349252703
ifconfig
#1349748599
cd /usr/local/bin/
#1349748630
ll

After conversion you will get a nicely formatted date and time stamp to go with each command:

#/usr/local/bin/epoch_converter >>/var/log/user_history/Epoch_Convert/root

# tail root

Wed Sep 26 12:08:33 JST 2012

/var/log/user_history/root_history

Wed Sep 26 17:05:39 JST 2012

ifconfig

Wed Sep 26 19:00:55 JST 2012

cd /usr/local/bin/

Tested on RHEL 5.3 and 6.3

# more /usr/local/bin/epoch_converter 

cat /var/log/user_history/root  | while read line ; do  if [[ $line =~ '^#' ]]; then  date -d "@$(echo $line | cut -c2-)"; else echo $line ; fi; done

Tested on SLES 10 and 11

# more /usr/local/bin/epoch_converter

cat /var/log/user_history/root  | while read line ; do  if [[ $line =~ ^# ]]; then  date -d "@$(echo $line | cut -c2-)"; else echo $line ; fi; done

If you're smart enough you will notice that the second script doesn't have quotations, if it did it wouldn't have worked on SLES. 

Of course no one wants to do this conversion job manually so I recommend you cronjob everything to make things easy for you

#cronjob to convert Epoch time in cmd history to human readable format

00 21 29 * * /usr/local/bin/epoch_converter >>/var/log/user_history/Epoch_Convert/root

While you're at it, its a good idea to schedule a job that runs every other month or so, depending on the size of your /var; to housekeep the converted logs. I am not a fan of removing log files, so I just nullify it:

#!/bin/bash

LOGS="/var/log/user_history/Epoch_Convert/*"

LOGS2="/var/log/user_history/Epoch_Convert/old_logs/$(date +'%m')"

mkdir -p ${LOGS2}

cp -p ${LOGS} ${LOGS2}/

cd ${LOGS}

for i in /var/log/user_history/Epoch_Convert/*;  do

  :>"$i"

done

If you any questions or recommendations please comment and I'll try to best to assist