/usr/share/doc/logtools/bulk-virtual-webalizer.txt is in logtools 0.13d.
This file is owned by root:root, with mode 0o644.
The actual contents of the file can be viewed below.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 | I have:
APACHE_POST_SCRIPT=/usr/local/sbin/runallweblogs
Also to have log data from multiple domains in the one log file you need a
statement like the following in httpd.conf to save the target domain name at
the start of the line:
LogFormat "%V %h %l %u %t \"%r\" %s %b \"%{Referer}i\" \"%{User-agent}i\" %T"
Also you need to load the vhost_alias module:
LoadModule vhost_alias_module /usr/lib/apache/1.3/mod_vhost_alias.so
You will need to specify which interface to listen on with a Listen directive,
and have something similar to the following for re-writing cgi-bin and
specifying the virtual document root:
RewriteEngine on
RewriteRule /cgi-bin/(.*) /cgi-bin/cgiwrap/%{HTTP_HOST}/$1 [PT]
VirtualDocumentRoot /home/hosting/%-1/%-2/%-3/%-4+
VirtualScriptAlias /home/hosting/%-1/%-2/%-3/%-4+
UseCanonicalName Off
/usr/local/sbin/runallweblogs has the following contents:
---
#!/bin/sh
/etc/init.d/apache reload
ROOTS="/home/chroot/asp /home/chroot/php4"
for n in $ROOTS ; do
## First we run the apache logrotate script inside each chroot environment
chroot $n /etc/cron.daily/apache
## Then we add its log file to the list.
OTH_INPUT="$OTH_INPUT $n/var/log/apache/access.log.1"
done
export OTH_INPUT
/usr/local/sbin/runweblogs
---
Edit the /etc/logrotate.d/apache file to run /usr/local/sbin/runallweblogs
instead of "/etc/init.d/apache reload".
Alternatively if you don't have a chroot environment then you can have
logrotate call /usr/local/sbin/runweblogs, in which case you need the
apache reload in there.
---
The ROOTS variable has the directory names of the chroot environments I use for
strange web setups.
/usr/local/sbin/runweblogs is separate so that I can easily interrupt things
and re-start them manually half way through if necessary, there's nothing
preventing you from combining them into one script.
/usr/local/sbin/runweblogs has the following contents:
---
#!/bin/bash -e
INPUT=/var/log/apache/access.log.1
MANGLE=/var/log/apache/mangled.log
SPLITDIR=/var/log/apache/split
## clfmerge -d will merge the logs
clfmerge -d $INPUT $OTH_INPUT >> $MANGLE
## Run webazolver to do DNS lookups on the log file with
## target domains merged in.
webazolver -N30 < $MANGLE &
if [ ! -d $SPLITDIR ]; then
mkdir $SPLITDIR
fi
cd $SPLITDIR
## while webazolver is running we split the log file into one file per domain
clfdomainsplit -o $SPLITDIR < $MANGLE &
## wait for webazolver to finish (it takes ages)
wait
if [ ! -d /var/log/apache/webstats/all ]; then
mkdir /var/log/apache/webstats/all
fi
## Run webalizer on all virtual web servers to give an overview of what the
## entire ISP traffic is.
webalizer -o /var/log/apache/webstats/all -t "All ISP customers" < $MANGLE
rm $MANGLE
## Now do the per domain processing from a Perl script.
/usr/local/sbin/process_logs.pl
---
Here's the contents of /usr/local/sbin/process_logs.pl:
---
#!/usr/bin/perl
use strict;
# expects to find CLF logs in the current directory where each log file name
# is the name of the domain it belongs to.
my @files = `ls -1`;
foreach my $file (@files)
{
chomp $file;
my $outputdir = "/var/log/apache/webstats/$file";
mkdir($outputdir, 0775);
if(system("webalizer -o $outputdir -t $file -n $file -r $file/ < $file") ==
0)
{
unlink($file) == 1 or printf(STDERR "Can't remove file $file.\n");
}
else
{
printf(STDERR "Error running webalizer!\n");
}
}
---
|