NAME SquidAnalyzer - Squid access log report generation tool

DESCRIPTION SquidAnalyzer parse native access log format of the Squid proxy and generate general statistics about hits, bytes, users, networks, top url, top second level domain and denied URLs. Common and combined log format are also supported. SquidGuard logs can also be parsed and ACL's redirection reported into denied URLs report.

Statistic reports are oriented to user and bandwidth control, this is
not a pure cache statistics generator. SquidAnalyzer use flat files to
store data and don't need any SQL, SQL Lite or Berkeley databases.

This analyzer is incremental so it should be run in a daily cron. Take
care if you have rotate log enable to run it before rotation is done.

REQUIREMENT Nothing is required than a modern perl version 5.8 or higher. Graphics are based on the Flotr2 Javascript library so they are drawn at your browser side without extra installation required.

CHANGES from https://github.com/darold/squidanalyzer SquidAnalyzer.pm can read compressed 'stat_code.dat', so you can compress previous years .dat files (e.g.: find 2023 -iname '*.dat' | xargs xz -9). You can install SquidAnalyzer.pm into custom LIB directory, so you can keep all files together (e.g. /opt/squidanalyzer) Hungarian translation

INSTALLATION Generic install If you want the package to be installed into the Perl distribution just do the following:

    perl Makefile.PL
    make
    make install

Follow the instruction given at the end of install. With this default
install everything configurable will be installed under
/etc/squidanalyzer. The Perl library SquidAnalyzer.pm will be installed
under your site_perl directory and the squid-analyzer Perl script will
be copied under /usr/local/bin.

The default output directory for html reports will be
/var/www/squidanalyzer/.

On FreeBSD, if make install is freezing and you have the following
messages:

        FreeBSD: Registering installation in the package database
        FreeBSD: Cannot determine short module description
        FreeBSD: Cannot determine module description

please proceed as follow:

        perl Makefile.PL INSTALLDIRS=site
        make
        make install

as the issue is related to an install into the default Perl vendor
installdirs it will then use Perl site installdirs.

Note: you may not encountered this issue any more, since v6.6
SquidAnalyzer use site as default installation directory.

Custom install You can create your fully customized SquidAnalyzer installation by using the Makefile.PL Perl script. Here is a sample:

        perl Makefile.PL \
                LOGFILE=/var/log/squid3/access.log \
                BINDIR=/usr/bin \
                CONFDIR=/etc \
                HTMLDIR=/var/www/squidreport \
                BASEURL=/squidreport \
                MANDIR=/usr/share/man/man3 \
                DOCDIR=/usr/share/doc/squidanalyzer

Or you can install everything into one directory (e.g.: /opt/squidanalyzer)

        perl Makefile.PL \
                LOGFILE=/var/log/squid/access.log \
                BINDIR=/opt/squidanalyzer/bin \
                CONFDIR=/opt/squidanalyzer/etc \
                HTMLDIR=/var/www/squidreport \
                BASEURL=/squidreport \
                MANDIR=/opt/squidanalyzer/share/man/man3 \
                DOCDIR=/opt/squidanalyzer/share/doc/squidanalyzer
                LIB=/opt/squidanalyzer/lib

If you want to build a distro package, there are two other options that
you may use. The QUIET option is to tell to Makefile.PL to not show the
default post install README. The DESTDIR is to create and install all
files in a package build base directory. For example for Fedora RPM,
thing may look like that:

        # Make Perl and SquidAnalyzer distrib files
        %{__perl} Makefile.PL \
            INSTALLDIRS=vendor \
            QUIET=1 \
            LOGFILE=/var/log/squid/access.log \
            BINDIR=%{_bindir} \
            CONFDIR=%{_sysconfdir} \
            BASEDIR=%{_localstatedir}/lib/%{uname} \
            HTMLDIR=%{webdir} \
            MANDIR=%{_mandir}/man3 \
            DOCDIR=%{_docdir}/%{uname}-%{version} \
            DESTDIR=%{buildroot} < /dev/null

See spec file in packaging/RPM for full RPM build script.

Local install You can also have a custom installation. Just copy the SquidAnalyzer.pm and the squid-analyzer perl script into a directory, copy and modify the configuration file and run the script from here with the -c option.

Then copy files sorttable.js, squidanalyzer.css and
logo-squidanalyzer.png into the output directory.

Post installation 1. Modify your httpd.conf to allow access to HTML output like follow:

        Alias /squidreport /var/www/squidanalyzer
        <Directory /var/www/squidanalyzer>
            Options -Indexes FollowSymLinks MultiViews
            AllowOverride None
            Order deny,allow
            Deny from all
            Allow from 127.0.0.1
        </Directory>

2. If necessary, give additional host access to SquidAnalyzer in
httpd.conf. Restart and ensure that httpd is running.

3. Browse to http://my.host.dom/squidreport/ to ensure that things are
working properly.

4. Setup a cronjob to run squid-analyzer daily or more often:

        # SquidAnalyzer log reporting daily
        0 2 * * * /usr/local/bin/squid-analyzer > /dev/null 2>&1

or run it manually. For more information, see README file.

If your squid logfiles are rotated then cron isn't going to give the
expected result as there exists a time between when the cron is run and
the logfiles are rotated. It would be better to call squid-analyzer from
logrotate, create file /etc/logrotate.d/squid with the following
content:

        /var/log/squid/*.log {
                daily
                compress
                delaycompress
                rotate 5
                missingok
                nocreate
                sharedscripts
                postrotate
                        test ! -e /var/run/squid.pid || test ! -x /usr/sbin/squid || /usr/sbin/squid -k rotate
                        /usr/local/bin/squid-analyzer -d -l /var/log/squid/access.log.1
                endscript
        }

Be sure that the paths used in this script correspond to your system.

5. Adjust the configuration

Make sure that the LogFile path is correct in your squidanalyzer.conf
file. For instance:

        LogFile /var/log/squid/access.log

You can also use network name instead of network ip addresses by using
the network-aliases file. Also if you don't have authentication enable
and want to replace client ip addresses by some know user or computer
you can use the user-aliases file to do so.

See the file squidanalyzer.conf to customized your output statistics and
match your network and file system configuration.

Upgrade Upgrade to a new release or to last development code is just like installation. To install latest development code to use latest ehancements process as follow:

        wget https://github.com/darold/squidanalyzer/archive/master.zip
        unzip master.zip
        cd squidanalyzer-master/
        perl Makefile.PL
        make
        sudo make install

then to apply change to current reports you have to rebuild them using:

        squid-analyser --rebuild

This command will rebuild all your reports where there is still data
files I mean not removed by the retention limit. It can takes a very
long time if you have lot of historic, in this case you may want to use
option -b or --build_date to limit the rebuild period.

USAGE SquidAnalyzer can be run manually or by cron job using the squid-analyzer Perl script. Here are authorized usage:

Usage: squid-analyzer [ -c squidanalyzer.conf ] [logfile(s)]

    -c | --configfile filename : path to the SquidAnalyzer configuration file.
                                 By default: /etc/squidanalyzer/squidanalyzer.conf
    -b | --build_date date     : set the date to be rebuilt, format: yyyy-mm-dd
                                 or yyyy-mm or yyyy. Used with -r or --rebuild.
    -d | --debug               : show debug information.
    -h | --help                : show this message and exit.
    -j | --jobs number         : number of jobs to run at same time. Default
                                 is 1, run as single process.
    -o | --outputdir name      : set output directory. If it does not start
                                 with / then prefixes Output from configfile
    -p | --preserve number     : used to set the statistic obsolescence in
                                 number of month. Older stats will be removed.
    -P | --pid_dir directory   : set directory where pid file will be stored.
                                 Default /tmp/
    -r | --rebuild             : use this option to rebuild all html and graphs
                                 output from all data files.
    -R | --refresh minutes     : add a html refresh tag into index.html file
                                 with a refresh intervalle in minutes. 
    -s | --start HH:MM         : log lines before this time will not be parsed.
    -S | --stop  HH:MM         : log lines after this time will not be parsed.
    -t | --timezone +/-HH      : set number of hours from GMT of the timezone.
                                 Use t

Squidanalyzer

Install / Use

README