Linux System Log Rotation
From Free Knowledge Base- The DUCK Project: information for everyone
/etc/logrotate.d
--------------------------------------------------------------------------------
Setting up and verifying system logging: syslog and klog; /etc/syslog.conf;
remote logging; monitoring logs using swatch; managing logs using log rotate.
It is possible to force log rotation to test new configuration.
{
dbw comment: invalid entries in logrotate.conf - such as logs for customer
web directories since deleted, can cause logrotate to fail completely! You
_can_ force a log rotation checking your logrotate.conf file for validity.
syntax: logrotate -f /etc/logrotate.conf
}
man logrotate
--------------------------------------------------------------------------------
#note: endscript should appear only in the last entry if in logrotate.d
# and should not appear at all if in logrotate.conf
--------------------------------------------------------------------------------
How logrotate is invoked from cron.
On one of my RedHat 5.2 systems, `crond' is started from the script
/etc/rc.d/init.d/crond. The manual `man cron' says that cron
searches for /etc/crontab and the files in the /etc/cron.d/ directory.
My /etc/crontab file contains the following.
SHELL=/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=root
# run-parts
01 * * * * root run-parts /etc/cron.hourly
02 4 * * * root run-parts /etc/cron.daily
22 4 * * 0 root run-parts /etc/cron.weekly
0 0 1 * * root run-parts /etc/cron.monthly
And the /etc/cron.d directory contains nothing.
So clearly, any logrotate commands are being run out of one or more of the
/etc/cron.* files. The manual `man 5 crontab' indicates that the above
commands mean that the program `run-parts' will be run as user `root' for
each of the scripts.
The command `run-parts' turns out to be the script /usr/bin/run-parts, which
is very short, as follows.
#!/bin/bash
# run-parts - concept taken from Debian
# keep going when something fails
set +e
if [ $# -lt 1 ]; then
echo "Usage: run-parts <dir>"
exit 1
fi
if [ ! -d $1 ]; then
echo "Not a directory: $1"
exit 1
fi
for i in $1/* ; do
[ -d $i ] && continue
if [ -x $i ]; then
$i
fi
done
exit 0
In essence, this just runs all of the scripts in the specified directory.
E.g. each hour, the executable plain files in /etc/cron.hourly are run.
It turns out that the `logrotate' program is invoked from /etc/cron.daily.
This is the contents of the file /etc/cron.daily/logrotate.
#!/bin/sh
/usr/sbin/logrotate /etc/logrotate.conf
This means that the only file that `logrotate' gets its instructions from
directly is /etc/logrotate.conf, which contains the following lines.
# see "man logrotate" for details
# rotate log files weekly
weekly
# keep 8 weeks worth of backlogs
rotate 8
# send errors to root
errors root
# create new (empty) log files after rotating old ones
create
# uncomment this if you want your log files compressed
#compress
# RPM packages drop log rotation information into this directory
include /etc/logrotate.d
# no packages own lastlog or wtmp -- we'll rotate them here
/var/log/wtmp {
monthly
rotate 1
}
# system-specific logs may be configured here
This is all explained in the manual `man logrotate'.
Operation of logrotate.
The directory /etc/logrotate.d contains the following files.
-rw-r--r-- 1 root root 354 Oct 13 1998 apache
-rw-r--r-- 1 root root 108 Aug 28 1999 cron
-rw-r--r-- 1 root root 188 Oct 14 1998 linuxconf
-rw-r--r-- 1 root root 156 Oct 13 1998 mgetty
-rw-r--r-- 1 root root 327 Aug 12 1998 syslog
-rw-r--r-- 1 root root 457 Sep 10 1998 uucp
The file /etc/logrotate.d/apache is the one I'm interested in for this
exercise. This file contains the following.
/var/log/httpd/access_log {
postrotate
/usr/bin/killall -HUP httpd
endscript
}
/var/log/httpd/agent_log {
postrotate
/usr/bin/killall -HUP httpd
endscript
}
/var/log/httpd/error_log {
postrotate
/usr/bin/killall -HUP httpd
endscript
}
/var/log/httpd/referer_log {
postrotate
/usr/bin/killall -HUP httpd
endscript
}
When I installed the latest version of Apache to get the PHP3 and
PostgreSQL to work (around 20 March 2000) on my web server machine,
I installed Apache so that the log files were in /home2/apache/logs
instead of /var/log/httpd.
Therefore what I need to do now is to modify the
/etc/logrotate.d/apache file so that the files referred to are all
in the directory /home2/apache/logs instead. My new
/etc/logrotate.d/apache script is as follows, and I saved the old
one in directory /etc/logrotate.d/old1.
# The new improved logrotate script for apache on fox.
/home2/apache/logs/*-access_log {
rotate 9
monthly
errors akenning@fox.topology.org
create
ifempty
olddir /home2/apache/logs/oldlogs
postrotate
/usr/bin/killall -HUP httpd
endscript
}
/home2/apache/logs/*-combref_log {
rotate 9
monthly
errors akenning@fox.topology.org
create
ifempty
olddir /home2/apache/logs/oldlogs
postrotate
/usr/bin/killall -HUP httpd
endscript
}
/home2/apache/logs/*-error_log {
rotate 9
monthly
errors akenning@fox.topology.org
create
ifempty
olddir /home2/apache/logs/oldlogs
postrotate
/usr/bin/killall -HUP httpd
endscript
}
--------------------------------------------------------------------------------
Now on Sunday 30 September 2001, it's time to get `logrotate' going on my new
SuSE 7.1 web server `dog'. The problem is that SuSE 7.1 does not come with
logrotate software on it. So I had to go and look for it. The link at redhat
was
wrong, but by looking around a bit, I finally found it at
ftp://ftp.redhat.com/pub/redhat/linux/code/logrotate/
I downloaded this file: logrotate-3.3.tar.gz. This can be compiled very simply
with `make' and `make install'. There's a manual logrotate.8 with it too. The
binary is installed as /usr/sbin/logrotate. So now all I have to do is write a
configuration file for the logrotate program and then write a cron script for
it.
(Note that since my main initial consideration is to rotate the httpd logs for
the beginning of October 2001, I could just use the Apache tool for this
purpose,
which is /usr/local/apache/bin/rotatelogs on my system. However, I can't
understand the documentation for this. So I'm playing safe and using the more
flexible redhat `logrotate' tool instead.)
On my SuSE 7.1 machine `dog', the manual says that after reading the per-user
crontab files in /var/spool/cron/tabs on start-up, the cron process reads the
file /etc/crontab. On my machine as configured, I find the following.
root@dog# more /etc/crontab
SHELL=/bin/sh
PATH=/usr/bin:/usr/sbin:/sbin:/bin:/usr/lib/news/bin
MAILTO=root
#-* * * * * root test -x /usr/sbin/atrun && /usr/sbin/atrun
0 21 * * * root test -x /usr/sbin/faxqclean && /usr/sbin/faxqclean
5 22 * * * root test -x /usr/sbin/texpire && /usr/sbin/texpire
25 23 * * * root test -e /usr/sbin/faxcron && sh /usr/sbin/faxcron | mail
FaxMaster
#
# check scripts in cron.hourly, cron.daily, cron.weekly, and cron.monthly
#
-*/15 * * * * root test -x /usr/lib/cron/run-crons &&
/usr/lib/cron/run-crons
59 * * * * root rm -f /var/spool/cron/lastrun/cron.hourly
14 0 * * * root rm -f /var/spool/cron/lastrun/cron.daily
29 0 * * 6 root rm -f /var/spool/cron/lastrun/cron.weekly
44 0 1 * * root rm -f /var/spool/cron/lastrun/cron.monthly
There's nothing here to help me with initiating my daily script. (By the way,
the fax commands are a bit worrying. I'll get rid of those when I understand
exactly what they do. They obviously produce many meaningless message which
root receives every day!)
So my next step is to look at the files in /etc/cron.d, because the cron
manual
says that all scripts in this directory are read next. On my machine the only
file in /etc/cron.d is a script `seccheck', which produces copious useless
messages to root every day and week. (I'll see if I can get rid of that some
day too!)
The directory /etc/cron.daily contains a script `aaa_base_rotate_logs' which
contains a complex set of rotation rules, but how are the scripts in this
directory invoked? Hmmm... Maybe they're invoked from that
`/usr/lib/cron/run-crons' script. Yes!! That's where it's invoked from. Yet
another big, incomprehensible script. The core of that script is the following
Bourne-shell loop.
SPOOL=/var/spool/cron/lastrun
for CRONDIR in /etc/cron.{hourly,daily,weekly,monthly} ; do
test -d $CRONDIR || continue
BASE=${CRONDIR##*/}
test -e $SPOOL/$BASE && {
case $BASE in
cron.hourly) TIME="-cmin +60 -or -cmin 60" ;;
cron.daily) TIME="-ctime +1 -or -ctime 1" ;;
cron.weekly) TIME="-ctime +7 -or -ctime 7" ;;
cron.monthly) TIME="-ctime +`date -u +%d`" ;;
esac
eval find $SPOOL/$BASE $TIME | xargs -r rm -f
}
if test ! -e $SPOOL/$BASE ; then
touch $SPOOL/$BASE
# keep going when something fails
set +e
for SCRIPT in $CRONDIR/*[^~,] ; do
test -d $SCRIPT && continue
test -x $SCRIPT || continue
case "$SCRIPT" in
*.rpm*) continue ;;
*.swap) continue ;;
*.bak) continue ;;
*.orig) continue ;;
\#*) continue ;;
esac
/sbin/checkproc $SCRIPT && continue
nice -15 $SCRIPT
done
fi
done
Now what does this mean?
The command `BASE=${CRONDIR##*/}' means that BASE is set to the `longest
substring of $CRONDIR which matches pattern "*/"', according to my Bash
reference card. This just means that the leading path components are removed.
(This is done more simply in C-shell!) In the case of the daily cron job, if
there is a file /var/spool/cron/lastrun/cron.daily (which is true), then the
following command is run.
eval find /var/spool/cron/lastrun/cron.daily -ctime +1 -or -ctime 1 | xargs -r
rm -f
The `xargs' command (which I have never seen before) builds and executes a
command line from standard input. Wierd! The `xargs' manual says this.
If the standard input does not contain any non-blanks,
do not run the command. Normally, the command is run once
even if there is no input.
So in this case, the command `rm -f' is executed for each of the files with
modification times within 24 hours of the current time. I don't really
understand this.
It looks like the file removal commands in the file /etc/crontab are designed
to synchronise the operation of the quarter-hour operations. The commands are
only executed if the files in /var/spool/cron/lastrun have been removed. What
a convoluted way of achieving a simple objective!!
Next any file in the directory /etc/cron.daily which do not end in the
characters `~' or `,' (presumed to be edited files) are executed if they are
executable and do not have the endings .rpm, .swap, .bak or .orig as follows.
/sbin/checkproc $SCRIPT && continue
nice -15 $SCRIPT
The loop continues is the process is already running. Otherwise it is run with
nice level 15.
What this all means finally is that any script in the directory
/etc/cron.daily
will be run at 00:00 on each day. More to the point, the /etc/cron.monthly
scripts are run at 00:00 at the beginning of each month. It all looks a bit
dodgy because the `date' test is `date -u', which gives the current UTC day.
But this should work, although the motivation for the `-u' is not quite clear.
All I have to do now is write a script and put it in /etc/cron.monthly. No
problem!
--------------------------------------------------------------------------------
sample methods:
# system-specific logs may be configured here
## rotate apache logs weekly, keeping 8 rotations
/www/apache/var/logs/*log {
rotate 8
postrotate
kill -HUP `cat /www/apache/var/logs/httpd.pid`
endscript
}
## rotate rembo logs weekly, keeping 4 rotations
/opt/rembo/logs/*.log {
rotate 4
postrotate
/etc/rc.d/init.d/rembo stop
/etc/rc.d/init.d/rembo start
endscript
}
--------------------------------------------------------------------------------
the roach method (for virtual hosting):
I have many virtual hosts. I feared that apache would be start/stop
frequently,
once per virtual host, during the rotation process. Therefore, I decided to
use
the 'copytruncate' option. With this option the apache web server does not
require a restart. -roach
/usr/apache/log/*log {
weekly
rotate 52
notifempty
missingok
copytruncate
}
/usr/apache/log/dynahost.dbw.org/*log {
weekly
rotate 52
notifempty
missingok
copytruncate
}
--------------------------------------------------------------------------------
Configure the new /etc/logrotate.d/apache file
Now Apache logs files residing in the /chroot/var/log/httpd directory instead
of
/var/log/httpd and for this reason we need to modify the
/etc/logrotate.d/httpd
file to point to the new chrooted directory. Also, we've compiled Apache with
mod_ssl, so we'll add one more line to permit the logrotate program to rotate
the ssl_request_log and ssl_engine_log files. Configure your
/etc/logrotate.d/apache file to rotate your log files each week automatically.
Create the apache file, touch /etc/logrotate.d/apache and add:
/chroot/httpd/var/log/httpd/access_log {
missingok
postrotate
/usr/bin/killall -HUP /chroot/httpd/usr/sbin/httpd
endscript
}
/chroot/httpd/var/log/httpd/error_log {
missingok
postrotate
/usr/bin/killall -HUP /chroot/httpd/usr/sbin/httpd
endscript
}
/chroot/httpd/var/log/httpd/ssl_request_log {
missingok
postrotate
/usr/bin/killall -HUP /chroot/httpd/usr/sbin/httpd
endscript
}
/chroot/httpd/var/log/httpd/ssl_engine_log {
missingok
postrotate
/usr/bin/killall -HUP /chroot/httpd/usr/sbin/httpd
endscript
}
--------------------------------------------------------------------------------
Credits and Quasi-bibliography:
The majority of this text is from:
http://www.topology.org/linux/logrotate.html
Reference: Alan Kennington's
Some information was gathered from:
http://misc.epfl.ch/rembo/logrotate_conf.html
Reference: Unknown
Securing and Optimizing Linux: RedHat Edition -A Hands on Guide
http://www.tldp.org/LDP/solrhe/Securing-Optimizing-Linux-RH-Edition-v1.3/chap29sec258.html
Anything else was created by:
roach
date: Sun May 12 18:03:33 CDT 2002
--------------------------------------------------------------------------------
Logrotate: error reading top line of /var/lib/logrotate.status
If you get an error saying
/etc/cron.daily/logrotate:
error: error reading top line of /var/lib/logrotate.status
delete /var/lib/logrotate.status, and then run logrotate once with -f flag,
like
#> logrotate -f /etc/logrorate.d/syslog
This should initialize the status file and the error should not be repeating.
