Yalsp - Yet Another Logfile Statistical Program.

Sorry my english is very poor.

To compile run make, I only try Linux RedHat 6.2, but probably there are no
problems on the other versions, the code is very simple and no strange library
are necessary.

This program take as imput a Squid 2.x logfile, and produce an Html file in
standard output with the statistics required.
The error are produced on standard error, but the only error supported is
memory problem.

The Squid 2.x log file must be native, so emulate_httpd_log must be off (default).
No whitespace for URI (default). In any case look at uri_whitespace options.

The Usage is:

YALSP -I input_file [-f num] [-s num] [-c num] [-h num] [-p]
                    [-C ip_client] [-H url_addr] [-v num_line]
                    [-x num] [-X num ][-O b|r] [-a] [-t num]

-I: This is the input file, must be a Squid 2.x logfile. REQUIRED.

-O: Statistics order option: B = Bytes - R = Requests.
-a: This option avoid print the general statistics table that are at the
    beginning of html file.

-v: This is for verbose mode, print the number of line processed every
    'num' of lines on standard error.

-f: Produce top 'num' first level domain hosts visited. IP address are
    classified not resolved. Default 10.
-s: Produce top 'num' second level domain hosts visited. IP address are
    grouped by IP class (A/B/C). Default 10.

-p: Enable Protocol statistics (not really interesting).

-c: This print the top 'num' client (your network) sorted by bytes, or
    requests (option depend, default bytes). The default is printing the
    top 10 clients, so the top IP address that make major traffic.
-C: Process only this IP client. With this option all lines in the log file
    that are not regarding this IP client are wiped out. So all statistic
    tables are for this IP. Address must be in dotted form (10.1.1.1).

-h: This print the top 'num' hosts visited (the world) sorted by byte usage,
    or requests (option depend, default bytes). The default is printing the
    top 10 hosts, so the top hosts address that make major traffic to your
    network.
-H: Process only this host. With this option all lines in the log file
    that are not regarding this host are wiped out. So all statistic
    tables are for this host. Address must be in normal form (www.xxx.xxx)

-x: Use this option if You want the Client/Hosts matrix table.
    In this table are reported ALL client (sorted), and foreach client, the
    top 'num' hosts that make major traffic (sorted). Warning may produce a
    big file so the default are disabled (num=0).
-X: Use this option if You want the Hosts/Client matrix table.
    In this table are reported ALL hosts (sorted), and foreach host, the
    top 'num' clients that make major traffic for visit it (sorted).
    WARNING MAY PRODUCE A GIGALACTIC FILE so the default are disabled (num=0).

-t: I CREATE THE PROGRAM FOR THIS STATISTIC.
    This one report really the major host visited by your network.
    ONE visit means that ONE client in ONE day trasfer more than 100Kbytes.
    (use --klimit option if you want change, warning this option take
    kilobytes as argument NOT bytes, so 100 means 100KB).
    So if you visit two time in a day only one is considered (Warning for
    Dhcp).
    Foreach hosts, report the number of visits, the number of clients that
    visited the host, and the days.
    Probably the number of clients is less then visits, because the same
    client can visit the host in two or tree day. The days may be less then
    the total days of logfile, because there are some days that the host is
    not visited. Sorry my English is very poor.

Advanced Options:

--nocolors     Print html table with no colors.
--klimit       Use with -t.
--day          Enable days of week statistics
--hour         Enable hours of days statistics
--fullcache    Print more details cache status statistics
--onlyip       Don't resolve IP address of squid clients

Example:

In my computer there is this situation:
I want the program run in a weekly log file.
So in crontab there are

0 0 * * 1 squid -k rotate
0 1 * * 1 /path/yalsp -I /log/access.log.0 > /path/last_week.html

In the first line squid make a rotation on Monday 00:00, so a week start
here and terminate on Sunday 23:59. In squid.conf must be logrotate option
enabled, warning for logrotate deamon, disable it for squid, make squid do a
log rotation itself.
So after a rotation the access.log was renamed access.log.0, and now (after
1 hour) a run program to make my statistics.
The program is very CPU intensive so it's better
0 1 * * 1 /bin/nice -n 10 /path/yalsp -I /log/access.log.0 > /path/last_week.html

The future:
I have a great idea...... :)