Sawmill
Download Sawmill 8.7.3
30 Days Free Trial
Home Products Downloads Purchase Support About About
Sawmill Sawmill

SAWMILL 6 VERSION HISTORY

ALL PLUG-INS

Sawmill has plug-ins to support the following log formats:

line

This is the version history for Sawmill 6. The Sawmill 7 version history is here and the Sawmill 8 version history is here.

Version 6.5.11, shipped July 24, 2004

Bugs fixed in version 6.5.11:

  • Fixed a bug which could cause some log entries to be ignored when DNS lookup was turned on.
  • Fixed a bug with Symantex Gateway Security log format.
  • Fixed a bug which could cause a crash while processing corrupt ZIP or gzip files with an FTP log source.
  • Fixed crash bug which could occur at end of log processing.
  • Fixed a bug where expired days did not become unlinked in the Calendar.
  • Fixed Sidewinder Syslog format to support single-digit days.
  • Fixed some small problems with Firewall-1 (fw logexport export) Log Format, which resulted in some web-server-style wordings in the statistics.
  • Fixed Safari IP-cookie warning message, which is no longer needed for Safari 1.2.2 and later because Apple fixed the bug. Yay, Apple! Warning message still appears when using Safari 1.2.1 and earlier, which have the IP cookie bug.
  • Fixed a bug where certain database builds/updates would result in a DOS window temporarily popping up on Windows.
  • Fixed bug where CSV header parsing would split the last field name if it contained whitespace.

New features in 6.5.11:

  • Added support for Interscan Messaging Security Suite Virus Log Format.
  • Added support for "Squid (with Referrer and Agent) Log Format (Syslog Required)" log format.
  • Added support for Netgear FVS318 log format.
  • Added support for Oracle Failed Login Attempts Log Format.
  • Added support for EmailCatcher log format.
  • Added support for BEA WebLogic log format.
  • Added support for Ethereal log format.
  • Added support for a variant of Postfix log format.
  • Added support for ClamAV log format.
  • Added support for Symantec AntiVirus Corporate Edition (VHIST Exporter) log format.
  • Improved "GFI Attachment & Content Log Format".
  • Added support for IPEnforcer log format.
  • Added support for NetGear DG834G Log Format.
  • Improved autodetection of Declude SPAM log format.
  • Modified Interscan Security Suite log format to accommodate a variant.
  • Improved support for Watchguard SOHO Log Format.
  • Fixed bug where CSV output was misformatted when URLs contained newline characters.
  • Added detection of X-Forwarded-Host; it is now used as cookie domain when present.
  • Improved Merak SMTP Log Format to autodetect in more cases and to track geographic information.
  • Added support for Symantec Gateway Security 5400 log format.
  • Improved Postfix log format to report usernames of authenticated SMTP users.
  • Added support for Watchguard Firebox Export Log Format.
  • Improved ServUFTP so it correctly reads the Error lines with partial bytes.
  • Added support for Cisco Router Log Format (no syslog).
  • Added support for GFI Spam Log Format.
  • Improved auto date format to handle three-letter months.
  • Added support for Proxy-Pro GateKeeper Log Format.
  • Added support for Active PDF Log Format.
  • Added support for MailSweeper 24 Hour Log Format.
  • Added support for Servers Alive Statistics log format.
  • Added support for Anti-Spam SMTP Proxy (ASSP) Log Format.
  • Added support for Ironmail AV Log Format (Sophos).
  • Added support for mmm/dd/yy date format.
  • Improved Postfix log format to track "from" and "to" domains.
  • Enhanced Cisco VPN Concentrator log format to report disconnection reasons.
  • Added support for IronPort Log Format.
  • Improved LinkSys Router Log Format to handle a different date format.

Version 6.5.10, shipped May 30, 2004

Bugs fixed in version 6.5.10:

  • Fixed a bug where log filters could be doubled in newly-created configurations.
  • Fixed Common Referrer log format so it handles empty referrers properly.
  • Fixed bug where URL was not redisplayed properly when using HTTP log source.
  • Added support for Snare log format.
  • Fixed a bug where entry/exit pages sort did not work properly.
  • Fixed a bug where authenticated users were not tracked correctly in WS_FTP Log Format.
  • Fixed a problem where NetCache NetApp 5.5 Log Format did not handle x-localtime fields correctly.
  • Fixed a bug where log formats which used the carryover feature would not report hours of day properly in some cases.
  • Fixed a bug where durations were formatted in CSV export, instead of appearing as simple seconds.
  • Changed internal name of localtime field to date/time to prevent conflict in W3C logs where both localtime and #Date exist.
  • Fixed DNS lookup bug which could causes crashes or odd errors.

New features in 6.5.10:

  • Added support for GMS POP Log Format.
  • Added support for Merak Mail Server Control Log Format.
  • Added export_pathname option to specify pathname of file for CSV export.
  • Added support for "Symantec Gateway Security 5400" format.
  • Added support for "Vamsoft Open Relay Filter Enterprise Edition Log Format".
  • Added support for Aladdin eSafe Sessions Log Format.
  • Added support for Fortinet Log Format (syslog required).
  • Enhanced "Windows 2000/XP Event Log Format (export list-CSV) ddmmyyyy" to handle AM/PM times.
  • Added a version of Unix Syslog format which extracts date from m-d-yyyy.log filename, working around the problem caused by lack of year information in Unix Syslog.
  • Added support for Apache/NCSA Common Agent Log Format.
  • Enhanced Common Proxy Log Format to use GeoIP to compute countries/regions/cities of source IPs.
  • Added support for Amavis Log Format Log Format.
  • Added country/region/city tracking for NetScreen Traffic Log Format.
  • Added support for Watchguard WELF Log Format.
  • Added support for FortiGate Space Separated Log Format.
  • Added support for Netgear Security Log Format (logging to syslog).
  • Added support for Netilla Log Format.
  • Added support for FortiGate Traffic Log Format.
  • Fixed a bug with expanding/collapsing subviews in non-English installations.
  • Added support for WebNibbler log format.
  • Added support for Eventlog to Syslog Format.
  • Added support for Packet Dynamics Log Format.
  • Added geographic country/region/city reporting of source IPs for Cisco PIX/IOS log format.
  • Added support for Nagios Log Format.
  • Added support for Bind 9 Query Log Format (without timestamp).
  • Improved Postfix support to report Vexira virus scanning lines.
  • Improved support for Bind 9 Query Log Format (with timestamp) to report day of week and hour of day.
  • Added support for IPCop Syslog.
  • Added support for NetCache NetApp 5.5 Log Format.
  • Added support for Trend Micro ScanMail For Exchange Log Format.
  • Added support for NetApp Filers Audit Log Format.
  • Added support for Symantec Enterprise Firewall 8 Log Format.
  • Added support for Message Sniffer Log Format.
  • Improved LogSat SpamFilterISP Log Format to report Bayesian filter lines.
  • Enhanced LISTSERV format to support single-digit dates.
  • Added support for a slight variant of NetCache NetApp Log Format.
  • Improved auto data format to allow only reasonable values for year, month, and day.

Version 6.5.9, shipped April 03, 2004

Bugs fixed in version 6.5.9:

  • Improved detection of corrupt TAI64N date/times.
  • Fixed a bug where if an Apache or Blue Coat format string contained characters which were also regular expression operators (like |), all log entries would be rejected.
  • Fixed a bug where if a timestamp field was used in W3C logs, and a #Date line was also present, date/time information was not reported properly.
  • Fixed Blue Coat W3C format to support explicitly listed URL fields in the log data (it no longer attempts to build the URL from pieces if it's there in the log data).
  • Fixed a bug where corrupt compressed log data would halt all log processing, instead of just skipping the corrupt file.
  • Fixed a bug where CSV export of a multi-table view like Single-page summary would incorrectly add a comma to the beginning of most of the header rows.
  • Fixed a bug where Sawmill would try to create a file called last_format_line.txt at the root of the disk, sometimes resulting in a permission error.
  • Fixed a bug which could cause a crash in some cases while processing complex log formats with DNS lookup turned on.
  • Fixed a bug where quotes in field values were not escaped properly when running commands.
  • Fixed a bug where using # characters in some cases (like field names) would cause errors when viewing statistics.
  • Fixed a bug where using the filter: format for expiring database data would cause the remaining data to be tripled.

New features in 6.5.9:

  • Enhanced LogSat SpamFilterISP Log Format to handle a slight variant.
  • Enhanced Lucent Brick format to extract data from 144-code lines.
  • Improved iPlanet Messenging Server 5MTALog format to support a slight variant.
  • Fixed a performance issue which could result in significant periodic slowdowns while processing datasets with very large numbers of unique values for multiple fields.
  • Improved support for Zone Alarm Log Format, to handle an alternate date format.
  • Added support for Syslog NG Log Format, including full message tracking.
  • Added support for Passlogd syslog format, including full message tracking.
  • Added support for Unix Syslog full message tracking.
  • Added support for InfiNet Log Format.
  • Added support for MailEnable Log Format.
  • Improved Novell NetMail to report on antivirus and antispam logging.
  • Added support for Sendmail NT Log Format.
  • Added support for Array 500 Combined Log Format.
  • Added support for MTS Professional Log Format.
  • Improved tcpdump log format to support resolved IPs.
  • Added support for Servers Alive log format.
  • Added support for GW Guardian Spam Log Format.
  • Added auto-detection of Domino Access log format.
  • Added support for Fountry BigIronII Log Format.
  • Added support for Microsoft ISA Server Packet Logs.
  • Added support for GW Guardian Antivirus Log Format.
  • Added support for Watchguard WSEP Text Exports Log Format (Firebox II & III & X)".
  • Enhanced d/m/yyyy date format to allow arbitrary dividers.
  • Added support for Cyberguard WELF Log Format.
  • Enhanced Bulletproof FTP log formats to track uploads
  • Improved NetScreen log format to handle any ordering of fields in log data.
  • Added support for a variant of Microsoft Exchange Internet Mail Log Format.

Version 6.5.8, shipped February 14, 2004

Bugs fixed in version 6.5.8:

  • Fixed a bug which could cause a crash in session calculations if all sessions were filtered out.
  • Fixed a problem with MailSweeper Log Format which could cause all log entries to be rejected.
  • Fixed N2H2 / Novell Border Manager Log Format to support single-digit days.
  • Fixed a bug where if the log data contained "format" comments (like W3C or WebSTAR), then during a database update, the first data seen would be processed with default format rather than honoring the preceding format line, possibly resulting in all the data being rejected or mis-categorized.
  • Fixed a bug where sorting some session views alphabetically could cause a crash.
  • Fixed a bug where some types of sort were not available in the Session Pages view.
  • Fixed a bug where if there was only one page view in the session information, the Session Pages and Session Visitors views would be empty.
  • Fixed a bug where if the Licensing page was cached, and the 30-day trial button was clicked each time the page appeared, a 30-day trial would run out in two sessions (possible less than 1 day).
  • Fixed a problem with WebSTAR analysis where if both URL and CS-URI were present in the same log, URLs would only be shown two levels deep.
  • [by request] Fixed a bug in Unix Syslog which could cause it to reject entries from certain variants.
  • [by request] Fixed a bug where if the running server URL didn't have a slash after the hostname, it would not be used to create full URL.

New features in 6.5.8:

  • Added support for Winproxy 5.1 Log Format (yyyy-mm-dd dates).
  • Added support for Astaro Log Format.
  • Added support for PortalXPert Log Format.
  • Added support for FirstClass Server Log Format.
  • Added support for iChain Log Format.
  • Added support for Socks 5 log format.
  • Added support for Windows Event Log Format (dumpel.exe export) Log Format.
  • Improved mm/dd/yyyy:hh:mm:ss format to handle all-lower-case months.
  • [by request] Added support for Trend ServerProduct CVS Admin Log Format.
  • [by request] Enhanced Netscreen IDP format to handle a slight variant.
  • [by request] Added support for Netgear Security Log Format.
  • [by request] Improved Interscan Messaging Security Suite Log Format to a handle a slight variant.

Version 6.5.7, shipped January 25, 2004

Bugs fixed in version 6.5.7:

  • [by request] Fixed a bug where the built-in web server did not respond properly to HEAD commands.
  • [by request] Fixed a bug where date/time fields were not handled properly in some formats, including Squid with epoc timestamp, resulting in all log entries being rejected.
  • [by request] Fixed a bug in the mmmmm/dd/yyyy date format which caused some dates to be ignored.
  • [by request] Fixed a bug where view URLs of the format rfcf+configname+cm+vs did not trigger an auto-update-on-view.

New features in 6.5.7:

  • [by request] Fixed Netscape Directory Server to handle a positive time zone offsets. Removed the default renaming of "method" to "operation", which could cause problems in log formats which had both.
  • [by request] Added support for Symantec Enterprise Firewall log format.
  • [by request] Improved Unix Syslog format to handle a couple small variants.
  • [by request] Improved PIX Firewall format to support a slight variant.
  • [by request] Enhanced Cisco VPN Concentrator format to handle formats with data split across multiple lines.
  • [by request] Added support for Cisco Access Control Server log format.
  • [by request] Improved Microsoft Media Server Log Format to handle a slight variant.
  • [by request] Added support for NVDcms log format.
  • [by request] Added support for Firewall-1 Log Viewer 4.1 Export Log Format.
  • [by request] Improved "alphabetical" table sort to ignore leading whitespace.

Version 6.5.6, shipped January 11, 2004

Bugs fixed in version 6.5.6:

  • [by request] Fixed support for WebSTAR log formats-- search engines and search phrases were not being properly reported.
  • [by request] Fixed a bug where submitting some forms (in particular the "Make This Data Available" button) could generate a strange error with some browsers.
  • [by request] Fixed a serious bug where the "update" and "rebuild" links (and auto-update-when-older-than) did not work in CGI mode, and clicking them could cause an arbitrary number of processes to be created, possibly overloading the system where Sawmill was installed.
  • [by request] Fixed a bug where for certain log formats (including Eudora Internet Mail Server), Sawmill would get confused about the date format, and would reject valid log entries.
  • [by request] Fixed a bug where the pie chart legend for the last row in the table was sometimes gray when it should have been colored.
  • [by request] Fixed a bug where Sawmill could use too much memory, and generate strange errors, when DNS was turned on in some cases.
  • [by request] Fixed a bug where progress information was not displayed for background database builds, like those started from the "update" and "rebuild" links in the statistics.
  • [by request] Fixed a bug where adding an extra "filter" option after view+config in the URL did not apply the filter properly.

New features in 6.5.6:

  • [by request] Improved Postfix log format to handle a variant.
  • [by request] Improved Cisco Router log format to extract more information from the messages.
  • [by request] Added support for Minirsyslogd log format.
  • [by request] Improved Cisco Router format to handle a slight variant.
  • [by request] Added support for WebSEAL Error Log Format.

Version 6.5.5, shipped December 13, 2003

Bugs fixed in version 6.5.5:

  • [by request] Fixed a bug where if session memory was exceeded in the Paths Through A Page view, no error would be generated.
  • [by request] Improved/fixed active FTP transfers so they choose the client connection port in a way that ensures it isn't still "lingering" from the last time; this eliminates a bug which could happen on some systems, where an attempt to rebuild or updated immediately after a previous build/update would fail with an FTP error.
  • [by request] Fixed a bug in ISC DHCP Log Format which could cause an error when creating a configuration.
  • [by request] Fixed a bug where expiration expired the highest-numbered day in the database (usually the last day) even when it shouldn't, in some cases.
  • [by request] Fixed a bug where turning on DNS lookup
  • [by request] Fixed a bug in the Microsoft Media Server log format plug-in which caused the number of visitors to always be 1.
  • [by request] Fixed a bug which could cause crashes when resolving IP addresses.
  • [by request] Fixed Squid (syslog required) log format to handle usernames and mime types properly.

New features in 6.5.5:

  • [by request] Added support for "Microsoft ICF Log Format" log format.
  • [by request] Added support for Zyxel Firewall WELF format.
  • [by request] Added support for BroadVisionError log format.
  • [by request] Added support for Cisco NetFlow log format.
  • [by request] Added support for ISC DHCP log format.
  • [by request] Improved "iPlanet Messaging Server 5 MTA Log Format" to handle a slight variant.
  • [by request] Renamed the "time spent per page" view to "session pages", and added a columns showing the number of sessions that visited each page; also, changes the page views column to be visible by default (it used to be off by default, available as an Option).
  • [by request] Added support for Norton Personal Firewall 2003 Connection Log Format.
  • [by request] Improved FTP progress indicator to show which file is being downloaded.
  • [by request] Fixed a bug where if a field value in the log data was a single double-quote ("), it could cause a crash while processing the log data.
  • [by request] Locked down browse_only mode a little more by eliminating the options to rebuild/update a database from the statistics, even when logged in as administrator.
  • [by request] Added support for Symantec AntiVirus Corporate Edition 8.0 Log Format.
  • [by request] Added support for NetKey Log Format.
  • [by request] Added support for Cisco Border Manager Log Format.
  • [by request] Added support for Unicomp Guinevere Virus Log Format.
  • [by request] Improved table toolbar to dim the "10 rows" button if there are less than 10 rows.
  • [by request] Added support for Firewall-1 (fw log -ftn export) Log Format.

Version 6.5.4, shipped November 08, 2003

Bugs fixed in version 6.5.4:

  • [by request] Fixed a bug where when both FTP and DNS were being used, they could step on each other's sockets, causing networking errors.
  • [by request] Fixed a bug where database expiration did not work at all in some cases, adding data to the database instead of removing it.
  • [by request] Fixed a bug which could cause a crash if the last character of a configuration file was a #.
  • [by request] Fixed IAS log format to work properly with the new allow_spaces_in_listed_field_values option.
  • [by request] Fixed view and subview editing so the "incremental" checkbox was used and saved properly.
  • [by request] Fixed a bug where if an item contained an asterisk (*), and it was used in a statistics filter, it would cause an error.
  • [by request] Fixed a bug where referrer, referrer description, search engine, and search phrase fields were not automatically added to the database when a referrer field was present in W3C log data (and other data with a field header).
  • [by request] Improved Helix Universal log format to handle negative GMT offsets and spaces in URLs.
  • [by request] Fixed a bug where if there was no maximum session duration for a configuration (which is rare) then the last session in the data would be ignored.
  • [by request] Fixed a bug where if the default configuration had been modified to turn on visitor tracking by default, and a W3C log (or other log with field headers) was being analyzed, then an error would occur ("multiple visitor id fields").
  • [by request] Fixed a bug where if field values contained pointy brackets (greater than or less than signs), quotes, or certain other characters, the generate HTML statistics could be displayed incorrectly in some browsers.
  • [by request] Fixed a bug where some older IIS configurations did not trigger the 6.4-to-6.5 converted, and therefore showed only 1 visitor in the statistics.
  • [by request] Improved instant messanging formats so they don't show screen info or worm fields, which are meaningless for those formats.
  • [by request] Fixed (again) a bug where Blue Coat IM format did not report sessions properly.
  • [by request] Fixed a bug where sessions tracked was not enabled properly unless visitor tracking was also turned on.
  • [by request] Fixed a bug where the "rebuild" link in the statistics actually updated the database, rather than rebuilding it.
  • [by request] Fixed a bug in SonicWall5 log format plugin which caused an error without reading logfile translated properly.
  • [by request] Fixed a bug where certain log filter values were English and were not translated properly.
  • [by request] Fixed a bug where the Instructions were missing from many of the views.
  • [by request] Fixed a bug where all individual IP numbers appeared as part of the hostname hierarchy, under the "IP Numbers" item, when omit_ip_numbers_from_host_hierarchy was true. Now, it correctly shows the IP Numbers item, but with no subitems-- IPs are omitted as they should be.

New features in 6.5.4:

  • [by request] Improved Interscan Proxy Log Format log format to handle a slight difference.
  • [by request] Added support for Critical Path Mail Server log format.
  • [by request] Added support for BroadVision Observation log format.
  • [by request] Added username tracking to Blue Coat RealMedia log format.
  • [by request] Added support for Novell NetMail Log Format.
  • [by request] Improved iMail to track source and destination emails by domain.
  • [by request] Added support for N2H2 Sentian Log Format.
  • [by request] Added a feature to automatically open or view the single configuration in the list, if there is only one, when Open Configuration or View Statistics is clicked in the Administrative Menu.
  • [by request] Added support for iPlanet Messaging Server 5 MTA Log Format.
  • [by request] Improved Blue Coat Custom log format so it includes information about all log fields in the database, not just those known by the log format plug-in.
  • [by request] Added support for Ascend Log Format.
  • [by request] Added support for Siteminder WebAgent log format.
  • [by request] Added support for Windows 2000 event log CVS export log format, Windows 2000 event log "Save" log format, and Windows NT event log CVS export log format.
  • [by request] Added support for x-localtime field in W3C logs.
  • [by request] Added support for FortiGate log format.
  • [by request] Improved Blue Coat IM log format in several ways, to report better who messages are sent from and to, and to eliminate some meaningless reports (including worms).
  • [by request] Added support for uw-imap log format.
  • [by request] Added support for Tellique log format.
  • [by request] Added support for XMail SMTP log format.
  • [by request] Improved the built-in HTTP/1.1 server to handle the "100 Continue" message in a way more in line with the latest HTTP/1.1 specification (it now honors the "Expect: 100-continue" request from HTTP clients). This *may* eliminate some issues where pages were not loading properly, and should in general improve compatibility with modern web browsers.
  • [by request] Added support for Tivoli Storage Manager TDP for SQL Server log format.
  • [by request] Added support for WallWatcher log format.
  • [by request] Added support for Cisco SCA log format.
  • [by request] Fixed several problem with Netscreen IDP log format, which prevented it from being analyze properly in most cases.
  • [by request] Added support for NetForensics Syslog Format.
  • [by request] Fixed a bug where background updates (like the one generated by clicking "update" in the statistics) would display output and progress information on the command line.
  • [by request] Fixed a bug where errors during background updates (like the one generate by clicking "update" in the statistics) were not reported properly.
  • [by request] Added support for Cognos Powerplay Enterprise Server log format.
  • [by request] Added support for EzProxy log format.

Version 6.5.3, shipped October 12, 2003

Bugs fixed in version 6.5.3:

  • [by request] Fixed a bug where IM sessions were not reported properly.
  • [by request] Fixed a bug where Sawmill could crash when running out of memory.
  • [by request] Fixed a problem with SonicWall format which could result in some fields being mis-reported.
  • [by request] Improved/fixed the automatic creation of database field for log formats like W3C and WebSTAR (formats which use fields headers) to work in more cases, and to handle cases where multiple fields of the same type exist.
  • [by request] Fixed a bug where when tallying hits on complete IPs, Sawmill would treat each octet in the IP as a character (using its ASCII code), and would compare them case-insensitively, with the result that IPs like 123.124.125.71 and 123.124.125.103 were treated as equal (because 71 is ASCII for G, and 103 is ASCII for g), and all hits from either IP would be grouped under one of the two IP, but would not be reported separately.
  • [by request] Fixed a bug in Squid (syslog required) log format which prevented the last few fields in the line from being extracted properly.
  • [by request] Fixed a bug in IIS SMTP W3C log format where sessions information was apparently (incorrectly) available.
  • [by request] Fixed ISA W3C log format, which was generating errors about duplicate fields.
  • [by request] Fixed a bug in Websweeper log format which caused it to generate an error when processing log data.
  • [by request] Fixed a bug where the "rekey" operation (used to parse certain complex log formats) was not working properly when the key contained uppercase letters.
  • [by request] Fixed a bug where the 404s view did not appear for W3C log data.
  • [by request] Improved the 6.4-to-6.5 configuration converter to work properly with more W3C configurations.
  • [by request] Changed automated updates and updates generated by the "update" and "rebuild" links so they run as separate processes, rather than running two tasks (the update and then the view) in a single process. This reduces the chance of memory conflicts between the two tasks, eliminating some sources of database corruption.
  • [by request] Fixed bug where the "View to send by email" menu only showed visible views.
  • [by request] Fixed a bug where the "incremental HTML generation" feature did not correctly determine when data needed to be regenerated.

New features in 6.5.3:

  • [by request] Added allow_spaces_in_listed_field_values option, which allows there to be spaces in name=value pairs in log data.
  • [by request] Improved/fixed SonicWall Alternative Log Format to handle tabs as well as spaces, and other small improvements.
  • [by request] Added support for NetGear FR328S Log Format.
  • [by request] Added support for bpft4 format.
  • [by request] Improved Quicktime Streaming Server log format support to automatically set up database fields and views.
  • [by request] Improved Snort log format to categorize the "rule" field in English.
  • [by request] Added support for Snort 2 log format.
  • [by request] Added support for MailSweeper (long) Log Format.
  • [by request] Added support for "active" mode FTP transfers (previously, Sawmill supported only passive), and made that the default, and added an option to choose which to use. This should solve an increasingly frequent problem where some firewalls or routers were blocking passive FTP transfers (but allowing active).
  • [by request] Added support for IIS SMTP Common log format.
  • [by request] Added support for SmoothWall log format.
  • [by request] Improved Postfix log format plug-in to handle Postfix 1.x logs.
  • [by request] Added support for SimpleDNS log format.
  • [by request] Enhanced the expiration feature to that any date/time filter set can be used to expire data (not just "old than X"). This makes it possible to remove sections of data that are corrupt, and re-add them.
  • [by request] Added a process_previously_seen_data option which forces all data in the log source to be added on update, even if it has been seen before. This can be used together with the enhanced expiration feature to, for instance, remove a month of corrupt data from a log-term database, and add it back in easily.

Version 6.5.2, shipped September 14, 2003

Bugs fixed in version 6.5.2:

  • [by request] Fixed a bug where the Start Over link was broken in generated HTML files.
  • [by request] Fixed a bug where W3C log field names were not properly converted when using a 6.4 configuration with 6.5.
  • [by request] Fixed a bug where clicking a hostname which had no page views, and then switching to the session information, would cause an internal error.
  • [by request] Added support for POP tracking in MailerDaemon log format; added categorization of sessions into POP, IMAP, and SMTP.
  • [by request] Fixed a bug where Sawmill would try to create the logo.gif file in the installation directory, resulting in a "Can't create file logo.gif" error in CGI mode if the CGI directory was read-only.
  • [by request] Fixed a bug where if the selected date range was entirely outside the range of data in the database, the Overview (and possibly other views) would display strange results, or show an Internal Error.
  • [by request] Fixed a bug which could cause a crash while creating a configuration for certain circumstances (including IIS logs).
  • [by request] Fixed a bug which could cause crashes while processing log data, especially if DNS lookup was turned on.
  • [by request] Fixed the error message you get when you try to use a 6.4 license with 6.5; previous versions would report an expired license with a very old year.
  • [by request] Fixed a bug where the -laid option did not work when running in web server mode from the command line.
  • [by request] Fixed a bug where compound filters selected by checking multiple boxes in the Filter Editor did not work properly.
  • [by request] Fixed a bug where clicking a "sort column header" link in a multi-subview view (like Single-page summary) did not work properly.
  • [by request] Fixed a bug where clicking the "sort column header" link in the session visitors view could cause a crash.
  • [by request] Fixed bug where alphabetical sort was not available in the Session visitors view.
  • [by request] Fixed Lucent Brick log format, which was extracting fields incorrectly.

New features in 6.5.2:

  • [by request] Enhanced Declude Virus log format support to handle a slight variant.
  • [by request] Improved the "Session visitors" view to include a "sessions" column showing how many sessions each visitor contributed.
  • [by request] Added support for MailScanner Log Format (testfase) format.
  • [by request] Enhanced IAS log format to support 5000-class error codes; added ip:source-ip, ip:source-port, ip:destination-ip, and ip:destination-port to the default fields list.
  • [by request] Added support for Intermapper Outages log format.
  • [by request] Added a new "auto" time format (for log format creation), which automatically handles most standard time formats.
  • [by request] Improved Apache Custom log format to create database fields automatically (like W3C) based on the log fields; this allows easy analysis of unknown (custom) fields in Apache Custom log files.
  • [by request] Added support for Netscreen IDP log format.
  • [by request] Fixed ISA W3C log format, which was not naming fields correctly under the new auto-W3C-setup feature.
  • [by request] Added support for Stonegate log format.

Version 6.5.1, shipped August 23, 2003

Bugs fixed in version 6.5.1:

  • [by request] Improved/fixed DNS queries to ask for only PTR responses; this improves the number of successful DNS queries.
  • [by request] Fixed a bug where if there were multiple subviews in a view, and one of the subviews did not have a matching single-subview view elsewhere, then the items in that subview would not be linked.
  • [by request] Fixed a bug where certain DNS queries (those with two answers, where the first wasn't a PTR or A) would be seen as failed lookups, even if they weren't.
  • [by request] Fixed a bug where clicking "remove" in the Filter Editor would take you back to the statistics without removing the Filter.
  • [by request] Fixed a bug where the Date header in the HTTP response would be garbled in certain cases (including certain Italian versions of Windows XP).
  • [by request] Fixed a bug on Tru64 UNIX where builds could terminate with a divide-by-zero error.
  • [by request] Fixed a bug where the "Day by day" checkbox did not properly add date/time cross-reference with IIS W3C log format.
  • [by request] Fixed a bug where screen depth/size info was not tracked properly for IIS W3C logs.
  • [by request] Fixed a bug where FTP log sources would fail, generating "errors" that were just normal FTP server responses, when the server was using multi-line responses.
  • [by request] Fixed a bug where the shortcut was wrong for the new "remove_reloads_from_sessions" option, resulting in documentation errors and other problems.

New features in 6.5.1:

  • [by request] Updated GeoIP database to version 352. This will cause a new GeoIP database download (14 Meg) the first time a database is built with this new version.
  • [by request] Added support for Privoxy Log Format.
  • [by request] Improved country/region/city reporting so it omits "unknown" regions, which are often associated with large metro regions.
  • [by request] Limited the number of items displayed in the "Edit Filter" page to 1000, to keep it from being too slow. If more than 1000 items are available, it will display a button that shows them all.
  • [by request] Added a warning message to the login page when Sawmill detects the Safari web browser is being used with an IP URL-- Safari contains a bug which prevents login information from being saved properly in this case.
  • [by request] Improved DNS caching so it saves failed lookups in the memory cache, but doesn't save failed lookups in the disk cache-- this allows Sawmill to keep trying to lookup failed IPs once per build without slowing things down by trying to do it on every line where they occur.
  • [by request] Added support for Vidius Combined Log Format.

Version 6.5.0, shipped August 13, 2003

Bugs fixed in version 6.5.0:

  • [by request] Fixed bug where clicking Back after editing a particular statistics filter, and then clicking Apply Filters, would re-edit the filter.
  • [by request] Fixed a bug where custom hierarchical fields were grouped incorrectly in some cases.
  • [by request] Fixed bug where Snort format did not work with certain syslogs.
  • [by request] Fixed bug where GNATbox syslog was not always detected properly.
  • [by request] Fixed a bug where Webtrends Extended log format conflicted with Kiwi log format over who got to track the "priority" field, resulting in an error when they both tried to.
  • [by request] Fixed bug where country/region/city information would sometimes be garbled.
  • [by request] Fixed a problem with the date getting corrupt in the accept_carryover scenario.
  • [by request] Fixed a bug where, with certain log formats (including iMail), when Sawmill accepted the "orphan" log entries that had never been accepted before, it would not run the Log Filters on them, resulting in log entries appearing in the database which violated the Log Filters rules.
  • [by request] Fixed a parsing problem in Unix Sendmail log format.
  • [by request] Fixed a bug where statistics filtering did not work properly in some cases if filters were applied simultaneously to two or more fields.
  • [by request] Fixed a bug where "week" links in the Calendar did not work properly.
  • [by request] Fixed a bug where if there was no date/time database field, all log entries would be rejected.
  • [by request] Fixed a bug in Microsoft Exchange 2000 log format to accept only if message processing ends.
  • [by request] Fixed a bug where senders were not reported in McAfee WebShield log format.
  • [by request] Fixed a bug where the Sort menu of the "Session visitors" view did not contain an option to sort by duration, and did contain a few options that did not make sense.
  • [by request] Fixed a bug where CGI mode did not return pages on Windows.
  • [by request] Fixed a bug where applying a session filter, and then removing non-session filters, could generate an error about "unknown session ID".
  • [by request] Fixed a bug where in some cases, session information would not be tracked, even when it was enabled in the configuration.
  • [by request] Fixed a bug which could cause crashes while analyze some log data with the GeoIP database.
  • [by request] Fixed a bug where items would disappear from the Task List after 10 minutes.
  • [by request] Fixed a bug where filter values containing commas could cause errors.
  • [by request] Fixed a bug where the 95th percentile calculation could crash if there was just one data point in the graph.
  • [by request] Fixed bug where attempting to sort the "Individual sessions" view chronologically would generate an "internal error".
  • [by request] Fixed a bug where the "session visitors" table in the Single-page summary was actually a time-spent-per-page table.
  • [by request] Improved and fixed Netscape format support so it sets up database and log fields intelligently (and correctly!).
  • [by request] Fixed detection of Generic CSV format. Fixed detection of date and time fields in Generic CSV format.
  • [by request] Fixed a bug where Sawmill would incorrectly detect the IP address on MacOS X. in some cases.
  • [by request] Fixed a bug where Sawmill would skip some lines (those with + in the time) in Lucent Brick log format.
  • [by request] Fixed a bug which could cause a crash while regenerating HTML files.

New features in 6.5.0:

  • [by request] NOTE: This list includes all changes since 6.4, including the changes from each 6.5 beta version. Many of the bugs listed here may be intra-beta changes; e.g. they may have been introduced in one beta version and fixed in a later one, rather than being bugs in the 6.4 branch.
  • [by request] Version 6.5 uses new licenses. You will need to get a 6.5 format license to use this (or use a 30-day trial license). Contact ferrar@flowerfire.com to upgrade your license from 6.4 or earlier.
  • [by request] Added support for Squid Common format.
  • [by request] Improved iPlanet Error format to extract hostname, type, and other message information.
  • [by request] Added support for Cisco IDS Netranger logformat.
  • [by request] Added support for "Kiwi (mm-dd-yy dates, with type and protocol)" syslog format.
  • [by request] Added support for GFI Attachment & Content logs. Added support for GFI Spam logs. Extended support for IPTraffic log formats.
  • [by request] Added support for IPTraffic LAN Statistics, and IP Traffic TCP/UDP Services log formats. Extended support for IPTraffic log formats.
  • [by request] Added support for decimal seconds on mm/dd/yyyy hh:mm:ss.sss.
  • [by request] Added support for Win2KPerformance Monitor.
  • [by request] Extended support of Microsft Exchange Log to only count messages that are sent not just queued and to count multiple recipients.
  • [by request] Added support for Free Radius Detail Log.
  • [by request] Improved default_log_date_year option so it uses the current year by default.
  • [by request] Added support for Lucent Brick log format.
  • [by request] Added a new log filter type; convert_field_map, which maps specific field values to specific other field values; i.e. you can remap 1->cat, 2->dog, 3->lemur, etc.
  • [by request] Added support for Bind 9 Log Format.
  • [by request] Added support for WebSEAL Authorization (XML) Log Format.
  • [by request] Added explicit support for WebSTAR W3C Log Format (it was already supported indirectly through the Generic W3C Web Server Log Format).
  • [by request] Extended support of CiscoVPNConcentrator logs formats Unix Syslog.
  • [by request] Added support for IAS Alternate Log Format.
  • [by request] Added support for mm/dd/yyyy variant of Interscan Proxy Log Format.
  • [by request] Improved Interscan Proxy plug-ins to report virus activity.
  • [by request] Added support for Filemaker 3 Log Format.
  • [by request] Added lock_database_when_in_use option.
  • [by request] Added support for iPrism Log.
  • [by request] Added support for NetScreen Traffic Log.
  • [by request] Added support for Squid log format (syslog required).
  • [by request] Extended support for Cisco PIX/IOS logformats to collect information from audit lines.
  • [by request] Added support for Sawmill Task Log format.
  • [by request] Added support for ISC DHCP log format.
  • [by request] Improved EIMS SMTP format to track domains/emails hierarchically.
  • [by request] Improved Cisco VPN Concentrator format to work with a wider variety of syslogs.
  • [by request] Added support for ipfw log format.
  • [by request] Added support for Borderware log format.
  • [by request] Added support for Unix Syslog With Year format.
  • [by request] Extended Kiwi dd/mm/yyyy format to handle Kiwi UTC dd/mm/yyyy format too.
  • [by request] Extended support of Communigate Pro logformat.
  • [by request] Added tracking of regions and cities, using the GeoIP database.
  • [by request] Created a SidewiderSyslog and a SidewinderFirewall plugin pair to support a SidewinderFirewall format.
  • [by request] Changed Postfix logformat to accept only on sent messages.
  • [by request] Extended support of ImageMaker Microtech Media log more variations.
  • [by request] Extended support of Merak SMTP log.
  • [by request] Added support for Lyris MailShield log format.
  • [by request] Added support for Windows Event (Tab Delimited) log format.
  • [by request] Added support for XMail Spam log format.
  • [by request] Added support for d/mmm/yyyy as dd/mmm/yyyy.
  • [by request] Extended support if IPMon format to work as a syslog_required format.
  • [by request] Extended support for a time format without leading zeroes.
  • [by request] Added for WAP Log format.
  • [by request] Added support for MicroTech Image Maker Media Log format.
  • [by request] Added support for MicroTech Image Maker Error Log format.
  • [by request] Added support for eManager Spam Filter Log Format.
  • [by request] Added support for SonicWall Alternate Log format.
  • [by request] Added support for Kerio Network Monitor Log format.
  • [by request] Improved the Postfix log format plug-in to work with arbitrary syslogs.
  • [by request] Improved Lyris MailShield log format to handle many specific types of messages (including spam blocking messages).
  • [by request] Improved Mail Daemon Log Format to handle SMTP lines.
  • [by request] Changed the "Report It" bug reporter to use HTTP instead of SMTP to send bug reports-- this works better when reporting bugs through some firewalls.
  • [by request] Fixed a bug where if a field value contained a comma or semicolon, Filters would not work properly for that value.
  • [by request] Improved hh:mm:ss time parser to accept non-colons between the numbers. This allows Sawmill to handle a Kiwi syslog variant which uses periods.
  • [by request] Extended support for Netscreen log.
  • [by request] Improved WUFTP log format to handle a slight variant.
  • [by request] Added support for Miva Common Access log format.
  • [by request] Added a new "Rename Configuration" option to the administrative menu, which changes the name of an existing configuration. Unlike the "Save As" option in the Configuration Menu, this deletes the original configuration (after successfully writing the new one), and moves the database over from the old configuration to the new.
  • [by request] added support for RadiusAccounting Log II.
  • [by request] Improved DNS lookup so it looks up *all* IP addresses, not just those in the "host" field. Previous version would only look up IP addresses for one field-- the one whose type was "host" in the Log Fields tab. Sawmill now looks at all fields when DNS is on, and resolves all IP addresses.
  • [by request] Extended support of UnixSendmail logs.
  • [by request] Added support for msg= bandwidth in WELF logs.
  • [by request] Extended support for Netscreen logs.
  • [by request] Added support for SyslogYYYYMMDDHHMMSS.
  • [by request] Added support for SmartMaxSMTP & SmartMaxPOP logformats.
  • [by request] Added support for Kiwi (mmm/dd dates, hh:hh:ss.mmm UTC times) syslog format.
  • [by request] Added support for NetScreen Syslog Log Format.
  • [by request] Added support for Sophos Mail Monitor for SMTP Log Format.
  • [by request] Added ignore_newlines_after_log_line_regexp option, which makes it possible to create log format plug-ins that automatically combine certain consecutive lines into single lines on-the-fly (useful if a KEY field on one line is needed to parse the next line).
  • [by request] Added support for WinSyslog log format.
  • [by request] Added support for Youngzsoft CCProxy log format
  • [by request] Added support for Microsoft Exchange Internet Mail Log formats.
  • [by request] Added support for IIS SMTP Log formats.
  • [by request] Added a new "Session visitors" view which shows the visitor ids from the session information, along with the number of page views and total time spent for each.
  • [by request] Improved export of "Single sessions" view so it exports the clickstream also.
  • [by request] Improved Interscan Email Viruswall format to report "Forwarded" messages.
  • [by request] Added support for Seconds since Jan 1 1970 timestamp syslog. Extended support for Squid syslog required log format. Extended support for Webtrends syslog required & syslog log formats.
  • [by request] Improved the layout of the Session Filters description (at the left of the Session Filters editor).
  • [by request] Improved table headers so clicking a header name will re-sort the table by the corresponding field.
  • [by request] Extended support of Squid Common Log Format.
  • [by request] Added support for iPrism-rt, iPrismMonitor, Webtrends syslog, Webtrends syslog required log formats.
  • [by request] Improved Cisco PIX/IOS format to extract IDS error messages better.
  • [by request] Improved instructions for "top countries/regions/cities" view, and added a section in the documentation on using the GeoIP database.
  • [by request] Improved Bind 9 Query log format to handle a slightly variant.
  • [by request] Added support for Firewall1 Next Generation Full logformat.
  • [by request] Added support for another (brand currently unknown) syslog format.
  • [by request] Added support for Firewall1 Next Generation General logformat.
  • [by request] Added support for AladdinEsafeGateway logs.
  • [by request] Extended support for CiscoVPNConcentrator logs.
  • [by request] Added support for IIS log with mm/dd/yyyy dates.
  • [by request] Improved line parser so it strips off linefeed and newline characters at the end of blank lines; this makes it possible to write parsing filters that identify blank lines (for instance, to accept on blank lines).
  • [by request] Improved iPlanet Error Log Format to handle a slight variant.
  • [by request] Extended Support for EIMSSMTP logformat.
  • [by request] Improved "Generate HTML Files" so it updates an existing HTML files set incrementally. Previous versions of Sawmill would generate all files every time. Now, Sawmill breaks the files down by date grouping (year/month/week/day), and only regenerates a group if the number of hits/pageviews/visitors/bytes have changed since the last generation. This gives the same results as the previous version, but can be much faster if the changes are small (or if there are no changes).
  • [by request] Added support for Helix Universal log format.
  • Fixed a bug where the column headers were links even when the page was an offline page.
  • [by request] Added support for "Windows Event Log (dumpevt.exe export)" format.
  • [by request] Added support for "Timestamp (mm dd hh:mm:ss)" syslog format.
  • [by request] Enhanced Cisco VPN Concentrator format to track "visitors" (unique users) and "session" information (including individual user sessions and time connected).
  • [by request] Added support for InterScanViruswall log format.
  • [by request] Enhanced W3C and CSV "generic" log format plug-in so it builds the list of database fields from the detected log fields; this allows Sawmill to analyze W3C logs which have log fields it has never seen before.
  • [by request] Added support for Websweeper log format.
  • [by request] Extended Kiwi/ISO syslog format to handle a slight variant.
  • [by request] Extended support for SnortStandalone.
  • [by request] Added support for MailSweeper log format.
  • [by request] Added support for WatchGuard log format (with arbitrary syslog).
  • [by request] Improved browser identifier to better identify IE, Netscape, Mozilla, Chimera, Phoenix, Opera, Safari, and other major browsers.
  • [by request] Added support for Separ URL Filter Log Format.
  • [by request] Added a command-line documentation blurb which is printed when Sawmill is run with just --help, -help, -?, and a few other variants.
  • [by request] Added support for Unicomp Guinevere log format.
  • [by request] Added support for LSMS Admin log format.
  • [by request] Added support for Kiwi Cattools Port Stat log format.
  • [by request] Changed web server mode binding behavior so it binds to 0.0.0.0 when no server_hostname is specified. Previous versions would bind to the first server IP, which was sometimes difficult to determine; binding to 0.0.0.0 causes Sawmill to bind to *all* interfaces, so at least http://127.0.0.1:8987/ (the loopback interface) is guaranteed to work. It still prints the first IP of the server, as well as it can compute it, but if it gets it wrong, you can use the actual IP and it will work, without having to reconfigure Sawmill to tell it what that IP is.
  • [by request] Added support for Iplanet Messenger Server 5 log format.
  • [by request] Added support for InterScanSecuritySuite log format.
  • [by request] Added support for WinProxy Alternate log format.
  • [by request] Added support for "Generic CSV" log format, which handles all types of CSV files somewhat intelligently, even if it doesn't know the fields in advance or what they represent.
  • [by request] Modified the gnatbox logformats so that they are syslog & syslog required format combinations.
  • [by request] Extended support for MicrosoftExchange2000.
  • [by request] Added support for SL4NTDDMMYYYY.
  • [by request] Improved SonicWall format to track page views and session information, by building the URL from the proto, dstname, and arg fields.
  • [by request] Extended UnixSendmail log format to support rejects .
  • [by request] Improved the Filter Bar display in the statistics to include a dark sidebar and other ways to bring attention to the fact that Filters are active.
  • [by request] Added support for Imail7 log format (syslog required).
  • [by request] Added support for VeritasNetackup log format.
  • [by request] Added support for Intel NetStructure VPN Gateway log format.
  • [by request] Added support for Exim log format.
  • [by request] Improved Bulletproof "sessions" log format to handle multiple download lines.
  • [by request] Added support for Yamaha RTX log format.
  • [by request] Added support for SpamAssassin log format.
  • [by request] Added support for Shorewall log format.
  • [by request] Added a new "skip most recent file" feature which skips the log file which has been most recently modified, when processing log data. This is useful when attempting to optimize log processing performance by skipping previously-seen files by filename-- in IIS and other situations, we don't want to process (and later skip) the most recent file because it's still being built.
  • [by request] Added support for Arcserve NT Log Format.
  • [by request] Enhanced the "allow empty log source" option to skip non-existent directories and files in the log source (files which were there when the build began, but disappeared during the build), without an error.
  • [by request] Added support for Bulletproof/G6 FTP format with dd/mm/yyyy dates and 24-hour times.
  • [by request] Added support for Helix Univeral format.
  • [by request] Improved/fixed support for Microsoft Media Server, to track all fields automatically.
  • [by request] Added support for Cisco Access Register log format.
  • [by request] Improved PIX/IOS format to handle a different line format.
  • [by request] Added support for Watchguard Firebox Export header format.
  • [by request] Improved Bulletproof FTP log format to handle a variant where RETR lines are replaced by "started download" lines. Also improved bandwidth tracking to handle KB notation.
  • [by request] Improved Watchguard format to accept multiple spaces between fields.
  • [by request] Extended NetScreen Traffic Log format to handle either spaces or arrows (->) between some fields.
  • [by request] Added support for Watchguard SOHO log format.
  • [by request] Improved NetScreen Traffic Log to handle translated addresses and ports, when present.
  • [by request] Added support for htdig log format.
  • [by request] Added support for Windows Event Log Format (ALTools export).
  • [by request] Added support for PostWorks IMAP format.
  • [by request] Added support for PostWorks POP3 format.
  • [by request] Added support for PostWorks SMTP format.
  • [by request] Improved Merak POP/IMAP format to support USER lines.
  • [by request] Added support for tcpdump log format with no command-line options.
  • [by request] Added automatic transfer of "summary" filter value to a "session" visitor id filter when it seems appropriate. Sawmill now looks through all normal statistics filters to see if there is any field which is a single filter item, where that item is a known visitor id value. If there is such a filter, and if there are no session filters active, the session filters are set to show only sessions for that visitor id. This provides a simple way to apply visitor id session filters through the view corresponding to the visitor id field ("Top visitor domains/hosts", "Top users", or whatever it happens to be for a particular log format.
  • [by request] Added an omit_database_fields option (intended for use in log format plug-ins) which allows some log fields to be explicitly omitted from the database fields, in formats where the log and database fields are automatically generated.
  • [by request] Added visitor_name option, which makes it possible for the term "visitor" to change for formats (like proxy servers) where the term "visitor" isn't appropriate. This option is used when creating log format plug-ins.
  • [by request] Improved IIS Web log format plug-in to use the new "generic W3C" features, so its log and database fields are generated automatically.
  • [by request] Added support for Netscape Extended log format.
  • [by request] Added support for Firewall1Webtrends logformat.
  • [by request] Added support for Netwall logformat.
  • [by request] Tightened autodetect expression for Generic CSV.
  • [by request] Improved handing of FTP log source where hostname is entered as hostname/path.
  • [by request] Added a new $notsupported option which removes the entry/exit page views and other session-page-related views, for formats where there isn't really a concept of a "page" in the session information.
  • [by request] Added support for Blue Coat IM log format.
  • [by request] Added new "session logout regular expression" option, intended for use when creating a log format plug-in for a format which has session information and has a "logout" operation in the session information (unlike most web logs, which have no logout operation, requiring the use of a timeout to determine session endings). This allows for more accurate tracking of sessions for these types of logs, including Cisco VPN logs.
  • [by request] Improved session information by eliminating duplicate sequential page views in a single session, so reloading a page (or having the browser reload it) will not result in multiple events in the session displays.
  • [by request] Improved progress page to show major and minor operations in progress, even when "Details" is collapsed.
  • [by request] Improved detection of W3C and other log formats with format headers, so many more field names are converted to their "human readable" equivalents, and database fields and views are set up automatically for those fields, even if the fields are not explicitly mentioned in the log format plug-in.
  • [by request] Added automatic download of the GeoIP database file when it is first needed.
  • [by request] Improved the convert_field_map Log Filter type so it can handle maps stored in text files.
  • [by request] Enhanced country/region/city detection so region is reported by its FIPS name, rather than its FIPS code.
  • [by request] Improved WUFTP log format to handle a slight variant.
  • [by request] Improved Cisco VPN Concentrator log format to handle a variant.
  • [by request] Added support for Firewall-1 NG formats with any fields (added support for parsing FW1 NG headers, and building log and database fields lists from them).
  • [by request] Added support for NetGear log format.
  • [by request] Improved iPrism (with syslog) log format plug-in to fix several typos and add support for session information, and support a slight variant.
  • [by request] Added support for Microsoft ISA CSV format.
  • [by request] Added support for Bulletproof FTP log format with d/m/yy dates and 24-hour times.
  • [by request] Added support for WinGate log format with dd/mm/yy dates.
  • [by request] Added support for iScan log format.
  • [by request] Added support for y/m/d date format.
  • [by request] Added support for IIS (yy/mm/dd dates) log format.
  • [by request] Added support for Ingate Firewall log format.
  • [by request] Added a new date format, "auto", which attempts to auto-detect the date.
  • [by request] Improved iptables format to work with any syslog.
  • [by request] Improved Radius ACT support to use the new auto-date and generic CSV features (which lets it analyze a wider range of Radius logs, and adds reporting of only those fields which are actually present in the logs).
  • [by request] Added support for Bind 9 Query logs with any syslog.
  • [by request] Added support for Foundry BigIron log format.
  • [by request] Added support for Riverstone Router log format.
  • [by request] Improved GeoIP download so if www.sawmill.net can't be resolved, it tries the IP address, and if the HTTP connection fails, it fails quietly and just doesn't include GeoIP information in the build.
  • [by request] Added an option to turn off GeoIP for a configuration, so the GeoIP database doesn't have to be downloaded.
  • [by request] Added support for a variant of Lucent Brick format.
  • [by request] Added support for Snort Standalone log format with mm/dd/yy dates. Improved Snort standalong formats to extract interface when available.
  • [by request] Added support for NTsyslog format.

Version 6.4.8, shipped June 15, 2003

Bugs fixed in version 6.4.8:

  • [by request] Improved session information so page views tracking "screen information" does not appear in the session views (e.g. paths through the site).
  • [by request] Fixed a bug where the DIRECTORY_WORD language module variable was not being translated properly.
  • [by request] Fixed a bug where the "server response" field name was not translated properly for non-English translations, resulting in an error when the Broken Links view was displayed.
  • [by request] Fixed a bug where some Solaris versions of Sawmill would display "alarm clock" errors on the command line.
  • [by request] Cleaned up statistics HTML slightly to make it more standard-compliant.
  • [by request] Fixed a bug where emailed views showed up as attachments in Outlook.
  • [by request] Fixed a bug where if the language module was out of date, and new options had been added since the previous version, Sawmill would generate an error while creating a configuration.
  • [by request] Corrected Blue Coat Squid log format plug-in so it expects fields in the places where Blue Coat puts them (some fields, including authenticated user, were not being extracted correctly).
  • [by request] Fixed a bug where the GUI component of Sawmill (Sawmill6CL.exe on Windows) did not honor a LogAnalysisInfoDirLoc file, if one was present.
  • [by request] Fixed a bug where the last session in the session information was not being properly discarded if its length was over the limit.
  • [by request] Fixed a bug where ipchains log format would fail to analyze data with a "regular expression" error.
  • [by request] Fixed a bug where double-quotes in field values would cause "command line log filters" to break.
  • [by request] Fixed a bug where if there was more than one field of type "hostname" defined in a custom log format string, Sawmill could get confused about which field was the "real" host field, causing it to structure the host field incorrectly (for instance, not showing only bottom-level items even if you had asked for it when the configuration was created).
  • [by request] Fixed a bug where HTTP log source URLs were not shown properly in the Log Source tab of the Configuration Options.
  • [by request] Fixed a bug where certain W3C log formats, including Blue Coat W3C with a "timestamp" field, would not process properly.
  • [by request] Fixed a bug which could cause crashes when viewing statistics if there was no date/time field in the database.
  • [by request] Added charset customization to emails, using the CONTENT_TYPE_CHARSET language module variable from the Stats module.
  • [by request] Fixed a bug where only one 30-day trial was allowed before the "Try Sawmill For 30 Days" button disappeared-- now two trials are allowed.
  • [by request] Fixed a bug where if visitor ids contained double-quotes, you could get an error trying to apply a session filter.

New features in 6.4.8:

  • [by request] Changed default view names so they wouldn't contain a slash (/). Internet Explorer insists on breaking lines at the slash, which caused some view buttons to span multiple lines even when there was no need for it.
  • [by request] Improved dmmmyyyy format to allow a leading space when d is one digit.
  • [by request] Added a new FILTERS_DESCRIPTION language module variable, which can be used in the subject of emails to describe the filters.

Version 6.4.8a, shipped June 15, 2003

Bugs fixed in version 6.4.8a:

New features in 6.4.8a:

  • [by request] Fixed a bug where if you had only an "unlimited" license, you would not be able to use any configuration because Sawmill would report that you were licensed for 0 configurations.

Version 6.4.7, shipped May 10, 2003

Bugs fixed in version 6.4.7:

  • [by request] Fixed a bug where some HTML tables were closed incorrectly, causing emailed statistics to render incorrectly in some mail clients.
  • [by request] Fixed a bug where the installer would not install all components properly on Windows 2003 (and sometimes failed on Windows 2000).
  • [by request] Fixed a bug where in CGI mode, the licensing page would generate a permission error if the CGI directory was not writable by Sawmill.
  • [by request] Fixed a bug where database expiration would sometimes fail, not modifying the database.

New features in 6.4.7:

  • [by request] Added a FAQ entry discussing log processing performance, and providing detailed tables of performance numbers of Athlon and SPARC.
  • [by request] Improved the configuration script's behavior (in the encrypted source distribution) in the case that there is no C++ compiler installed on the system.
  • [by request] Change the dots in the IP address in a temporary URL to underbars; this works around an issue where URLScan was being picky about URLs with too many dots.
  • [by request] Fixed a bug where in CGI mode on Windows, result pages could be truncated.

Version 6.4.6, shipped April 26, 2003

Bugs fixed in version 6.4.6:

  • [by request] Fixed a bug where if you rebuild a database where there was no log data, and the "allow_empty_log_source" option was true, it would result in a corrupt database.
  • [by request] Fixed a bug where the "database structure" page of the Create Configuration interview was not set from the Default Configuration's Database Structure tab.
  • [by request] Fixed support for %X directive in Blue Coat log format strings.

New features in 6.4.6:

  • [by request] Improved LSMTP format to handle logging with job number, event id, and egroup information.
  • [by request] Improved LSMTP format to handle a slightly variant.
  • [by request] Improved granularity of the reported bytes-per-second in the processing page, to show hundredths of a unit.
  • [by request] Improved the Task List so it shows whether processes are complete (and turns them grey id they are), and so it shows tasks during the "safe update copy" stage of database updates, and so it shows "stalled" next to tasks which may be stalled or terminated.
  • [by request] Added a new entry to the Database Info window showing which log format is in use.

Version 6.4.5, shipped April 21, 2003

Bugs fixed in version 6.4.5:

  • [by request] Fixed a bug where if the configuration file contained a space, export would fail on Mozilla.
  • [by request] Fixed bug where custom log format strings with single-quotes (') in them would cause an error.
  • [by request] Fixed a bug where the "hostname" field was not being extracted properly in W3C log formats in non-English installations, resulting in all visitor counts being 1.
  • [by request] Fixed a bug where if you logged in to a password-protected config, and then tried to go to another password-protected config without logging out, you would get the Administrator login. Now, it shows you the login for the new config.
  • [by request] Fixed a bug which could cause crashes while generating the "Paths through a page" view.
  • [by request] Fixed a bug where the "visitor id" sort did not work properly in the Individual Session(s) view.
  • [by request] Fixed a bug where updating a database that had never been build would not properly build the database (as it should).
  • [by request] Fixed a bug where if there were multiple date/time fields in Blue Coat Custom log format, Sawmill would reject all log entries.
  • [by request] Removed "Export All" link from the clickstream table, since it was redundant-- The clickstream table always shows all rows, so Export Table is the same as Export All.
  • [by request] Fixed a bug were exports of the Individual Sessions views could contain HTML in the CSV table header, if the Start Time or End Time columns were turned on.
  • [by request] Fixed a bug where generating the "Top weekdays, avg" view with Filters applied could generate an Internal Error.
  • [by request] Fixed a bug where Sawmill would not email a view that was "invisible offline".
  • [by request] Fixed a bug where if there was a double-quote in the "extra options" in the Scheduler, everything after the first double-quote would be stripped off.
  • [by request] Fixed a bug where if an error occurred during a database build, the database would be left in a state where it looked built to Sawmill (so "View Stats" was available), but it wasn't actually, so viewing would generate an error about a critical file being missing from the database.
  • [by request] Fixed a bug where error messages in bug reports had unreplaced variables like {QUOTEVAR:PARAM1}.
  • [by request] Fixed a bug where some of the values in the Sessions Overview would be random if there were no page views in the dataset.
  • [by request] Fixed bug where %X support was not working for Blue Coat Custom Log Format.
  • [by request] Fixed a bug which could cause an Internal Error (or a bunch of them) if a day_of_week_computed or hour_of_day_computed subview was used, and there was not day_of_week or hour_of_day database field (which shouldn't be necessary).

New features in 6.4.5:

  • [by request] Updated CiscoPIXIOS log format
  • [by request] Improved Options menu in multi-subview views (like Single-page summary) so it affects all subviews.
  • [by request] Added detection of Safari browser.
  • [by request] Improved Individual Sessions export so that if there is only one session (so the clickstream for that session is exported), the session list itself is omitted, and only the clickstream table is exported.
  • [by request] Added a new extra_session_columns option which adds addition columns to the Individual Sessions clickstream table. This option, together with specially- formatted Log Filters which append additional field values to the "page" field, make it possible to have any number of columns in the clickstream table, for instance to show the category of the hits, or the country, or anything else.

Version 6.4.4, shipped April 05, 2003

Bugs fixed in version 6.4.4:

  • [by request] Fixed a bug where authenticated user information was not being extracted correctly from Squid logs.
  • [by request] Fixed a bug which could cause crashes when analyzing certain types of logs (those parsed with the log_file_format_regular_expression option), especially on Solaris.
  • [by request] Fixed a bug where if pie charts were turned on for the entry/exit page views, Sawmill could crash while generating those views.
  • [by request] Fixed a bug where %Z fields in Blue Coat Custom logs were not parsed correctly.
  • [by request] Fixed a bug where if you used a double-quote (") in the Matches field of the log field editor, it would generate an error.
  • [by request] Fixed a bug where on some Solaris platforms, Sawmill would terminate periodically with an "alarm clock" message on the terminal.
  • [by request] Fixed a bug where "--==LogAnalysis-Boundary-Alternative" would appear at the bottom of some emailed views.
  • [by request] Fixed a bug which could cause a crash when duplicating a Scheduled Task.

New features in 6.4.4:

  • [by request] Added support for a "localtime" token in W3C logs.

Version 6.4.3, shipped March 23, 2003

Bugs fixed in version 6.4.3:

  • [by request] Fixed a bug where the "Sawmill" logo was tiny in the header of CGI mode administrative pages.
  • [by request] Fixed a bug where if a item in a table contained a plus (+), clicking it would generate an error message.
  • [by request] Fixed a bug where long operations could fail with "Can't rename file" errors.
  • [by request] Added support for "timestamp" fields in W3C logs.
  • [by request] Fixed a bug where if there were multiple log sources, Sawmill could get confused while updating the database, and could re-add some data that it had already added.
  • [by request] Fixed a bug where if a field value contained a quote, command-line filtering would fail.
  • [by request] Fixed a bug where the Scheduler cached the names of configuration files, so deleted configuration files would still be used by the Scheduler, and newly-created ones would not.
  • [by request] Fixed a bug where if the last character on a white-space-separated log line was a quote, Sawmill would include that quote in the value of the final field.
  • [by request] Fixed a bug where tasks disappeared from the Tasks list after 10 minutes, even if they were still running.
  • [by request] Fixed a bug where exported data had semicolons as thousand dividers.
  • [by request] Fixed a bug where Sawmill could crash when displaying a view with no subviews.
  • [by request] Fixed a bug where clicking "remove" on a particular filter in the Filter Editor would not remove that filter.
  • [by request] Fixed bug where the "Top days" view was not created by default.
  • [by request] Fixed a bug which could cause a crash on database rebuild, if there was a page log field but no page database field.

New features in 6.4.3:

  • [by request] Fixed a bug where 95th percent calculations were randomly incorrect (and often showed 0).
  • [by request] Improved Sawmill's compatibility with the Safari beta browser.
  • [by request] Fixed a bug where command-line CSV exports used "Meg", "Gig", etc. format for bandwidth, instead of integer byte counts.
  • [by request] Fixed a bug in the Cisco PIX/IOS plugin which could cause source IPs to be extracted incorrectly in some cases, from Teardown Local lines.
  • [by request] Fixed a bug where the authentication command line didn't work on Unix-- it ignored lines past the first one.
  • [by request] Fixed a rare bug which could cause crashes when sending views by email, when viewing statistics, when generating HTML files, and in other situations.
  • [by request] Added detection of Wget as a web browser type (it was already detected as a spider).
  • [by request] Added information to bug reports showing which mode (CGI or Web Server) Sawmill is running in.
  • [by request] Improved the "Hints" Option so it controls whether the paragraphs in the Overview are displayed.
  • [by request] Added CONTENT_LANGUAGE and CONTENT_TYPE_CHARSET variables to the language module, which are used in the HTTP headers to set the charset and Content-language. These are needed when generating statistics in some languages.
  • [by request] Improved WU-FTP log format to handle a slight variant.
  • [by request] Added sub_view_header and sub_view_footer options that can be used to add custom HTML above and below each subview.

Version 6.4.2, shipped February 15, 2003

Bugs fixed in version 6.4.2:

  • [by request] Fixed a bug in the Configuration Editor which could generate invalid HTML, causing problems with some browsers.
  • [by request] Fixed a bug where on some operating systems (including BSD/OS), database updates would fail with the error message "Can't rename {config}__old/ to {config}".
  • [by request] Worked around a bug which could cause a "Can't rename ...Result-old to ...Result" error on Windows, especially during long builds.
  • Fixed a bug where Sawmill could crash if there was no data after session filters were applied.
  • [by request] Fixed bug where the progress description was incorrect for Scan and Summarize progress pages.
  • [by request] Fixed a bug where the TempLogs folder (where Sawmill stores log data temporarily when processing compressed logs from an FTP or HTTP server) was not cleaned out properly when an error occurred.
  • [by request] Fixed a bug where if there were custom numerical filters on a field, Sawmill could crash in several situations, including while generating HTML files.
  • [by request] Fixed a bug where clicking "update" in the statistics, and then clicking something else, would sometimes re-update, doubling the data in the database on the second update.
  • [by request] Fixed memory management during CSV export, so it doesn't use arbitrary amounts of memory.
  • [by request] Fixed a bug where if DNS lookup was enabled, and a series of IP addresses were very slow to resolve, Sawmill could get in a loop where it stopped processing log data and consumed all available memory.
  • [by request] Fixed a bug where %A was not processed properly in Blue Coat Custom Log Format, causing some log entries to be ignore.
  • [by request] Fixed a bug where the "paths through the site" view showed variable names like {VISITORS_THEN_EXITED} instead of proper descriptions.
  • [by request] Fixed a bug in PIX/IOS parsing where bandwidth information was not tracked in some cases.
  • [by request] Fixed bug where month formatting was off in the Calender in some cases, when the first day of the week was not Sunday.
  • [by request] Fixed a bug where if you created a View or Subview filter, and then rebuilt the database with different parameters, the item numbers in the database would no longer match up with the item numbers in filter you created, causing the filter to change, or even causing a crash. Filters are now always saved using symbols, rather than numbers, so that rebuilding will not cause this problem.

New features in 6.4.2:

  • [by request] Added support for 24-hour d/m/yyyy in Windows Event Log Format.
  • [by request] Improved statistics footer so it doesn't show the "Log Out" item if the configuration does not require a password to log in.
  • [by request] Improved screen info tracking, so the /loganalysis_screen_info.gif file is converted to (screen info) in the statistics. This serves two purposes: 1) it omits it by default from the "top pages" view, and 2) it causes it to be counted as a page view, so screen information is available in configurations where only page views are tracked.
  • [by request] Added tracking of the "server IP" field in IIS W3C log format.
  • [by request] Improved the default order of the views for web server log analysis, to "Top pages/directories" follows immediately after "Top pages" and "Top search terms" follows "Top search phrases".
  • [by request] Improved language module loading so if the specified language doesn't have files of the appropriate language and version, Sawmill will scan the Languages folder looking for ANY modules of the correct language, regardless of version, and will use them if it finds them. This makes it possible to install, say, a German module once, and have that work forever even if you install newer English versions on top of it.

Version 6.4.1, shipped February 01, 2003

Bugs fixed in version 6.4.1:

  • [by request] Fixed a bug where Sawmill would crash on exit on MacOS X, in some cases.
  • [by request] Fixed a bug where CGI mode did not work at all on Windows.
  • [by request] Fixed a bug where cookies were not being sent properly in some cases, making it impossible to stay logged in.
  • Fixed a bug where very large form submissions could causes crashes.

New features in 6.4.1:

  • [by request] Improved date/time range display so if you choose a date/time range, and there are gaps in that range in the data, Sawmill displays the range in the Filter bar, rather than all the little ranges you get after the gaps are removed.

Version 6.4.0, shipped January 17, 2003

Bugs fixed in version 6.4.0:

  • [by request] Disabled numerical and alphabetical sorts for views where it didn't make sense.
  • [by request] Fixed a bug where database updates would fail if 1) they were safe updates, and 2) the configuration name contained a space.
  • [by request] Fixed a bug where safes updates would fail on Windows because it couldn't delete the "writeLock" file.
  • Fixed a bug where the date/time range controls would appear in static pages.
  • [by request] Fixed a bug where some of the log format plug-ins were corrupt, causing errors when autodetecting log formats.
  • [by request] Fixed a bug that could cause floating point exceptions during command line database builds.
  • [by request] Fixed bug where Sawmill was not communicating correctly with some FTP servers, including UNIX servers, resulting in hung FTP transfers.
  • [by request] Improved behavior when no log format is selected; Sawmill redisplays the "choose log format" screen with a red prompt now, rather than showing an error screen.
  • [by request] Fixed a bug where exported session tables sometimes contained
    .
  • [by request] Added "total session page views" and "average page views per session" to the Sessions Summary view.
  • [by request] Improved support for iMail log format, so it picks up more information, and reports more accurately.
  • [by request] Improved Squid log format to accept resolved hostnames as well as IPs.
  • [by request] Fixed a bug where updating a database using "safe update" could cause an error on Windows.
  • [by request] Fixed a bug where file types were not detected properly if there were URL parameters.
  • [by request] Fixed a bug where if the "page" field was not deep enough, session information would be incorrect. This was not usually a problem because the page field is usually very deep (9 levels).
  • [by request] Fixed incompatibility with OmniWeb which would result in empty pages when forms were submitted.
  • [by request] Fixed a bug where the "visitors" column did not use thousands separators in its numbers.
  • [by request] Fixed the "Kill Server" button, which stopped working around 6.4b2.
  • [by request] Added support for Eurodate SYSLOG
  • [by request] Fixed a bug where CSV export would fail if the configuration name contained a space.
  • Fixed a bug where if you used a Syslog and Generic log format plug-in, and clicked the "visitors" checkbox, Sawmill would generate an error.
  • [by request] Fixed a bug where Sawmill could crash while processing certain types of log formats, including iMail logs.
  • [by request] Fixed a bug where Sawmill could crash under certain circumstances, in particular when you click "Make Data Available" when running Sawmill on Windows. In theory, this could cause crashes on any platform, under various circumstances, but it was especially noticeable (and reproducible) on Windows.
  • [by request] Fixed a bug where the syslog format menu would not appear if there were multiple possible formats, and the first one was a generic format.
  • [by request] Fixed a bug where the bookmark URL would sometimes get mangled.
  • [by request] Fixed WebServerAccessLog logging so it reports the IP address correctly (as it did in 6.3.x).
  • [by request] Fixed a bug where if there were two formats detected, and one of them was a "Generic" format, Sawmill would (correctly) throw the Generic out of the matching list, but would still prompt for a choice with a list containing just one item. It now assumes you want the only one available.
  • [by request] Fixed a bug where Sawmill and IE were using different timeouts, causing clicks to result in broken pages if the click was between 10 and 35 seconds after the previous click.
  • [by request] Corrected date format for Centrinity First Class
  • [by request] Fixed problem with bandwidth support with syslog required log formats
  • [by request] Fixed a bug where pattern filters containing curly brackets did not look right in the Filters bar and the Filter Editor.
  • [by request] Fixed a bug where if pattern filters contained curly brackets, they would not bookmark or link correctly.
  • [by request] Fixed a bug where if the database was auto-updated-on-view, and safe updates were on (the default), Sawmill would fail to update, and would give an error about not being able to copy some directory to itself.
  • [by request] Fixed a bug where the "Don't autodetect log format" checkbox wasn't working properly.
  • [by request] Fixed a bug where #Fields: lines in W3C logs with tabs after the colon were not handled correctly.
  • [by request] Fixed a bug where if a format matched a single non-syslog-required plugin and also a syslog, Sawmill would use them both. Now it uses only the non-syslog-required plug-in.
  • [by request] Improved error reporting in the case that a syslog-require plugin matched but no syslog did.
  • [by request] Fixed a bug where Sawmill did not handle format= lines correctly when field names were surrounded by quotes.
  • [by request] Fixed bug where "show logo in docs header" wasn't working.
  • [by request] Fixed bug where full URLs (like bookmark URLs) were computed incorrectly in some cases in web server mode.
  • [by request] Fixed hostname problem in NetCacheNetApp log format
  • Fixed a bug in session median computation where the wrong two data points were being averaged in the even-number-of-sessions case.
  • [by request] Fixed a bug where editing preferences (and certain other operations) could cause an internal error.

New features in 6.4.0:

  • [by request] NOTE: These are *all* features added since 6.3.16; i.e. they are all the new 6.4 features. Many of these features have been listed in previous 6.4 "beta" and "stabilizing" releases.
  • [by request] Extended support of OpenwaveIntermail Server Logs to capture bandwidth information
  • [by request] Extended support of Postfix Mail Server Logs
  • [by request] Changed language module version numbers to match Sawmill version numbers. For instance, where the language module version for Sawmill 6.3.10 used to be v88, it is now v6.3.10. This makes it easier to find the language module for a particular version, and eliminates certain types of uncommon errors where a pre-release version would not be able to find recent language module variables.
  • [by request] Added support for War FTP logs
  • [by request] Added date range filtering-- the current shown date range now appears at the top of the statistics, in the Filters bar, and you can choose the starting and ending year/month/day to zoom in on a particular date range.
  • [by request] Added a "safe update" feature (on by default) which backs up the database before updating it. This all but eliminates the possibility of an update corrupting the database on an error, but it requires twice the disk space and some additional time for an update.
  • [by request] Added an "If A, then do B followed by C" filter type.
  • [by request] Optimized database updating in the case where the is no new data in the log source. Previous versions would re-consolidate the database and re-write the index, even though there was no change. This version skips that step, resulting in a much faster update in this situation.
  • [by request] Added support for Apache Combined with Server Name after Agent
  • [by request] Added support for Cisco IOS (Unix syslogd) as well as created language variable IOS_PARSING_FILTERS logs
  • [by request] Added support for Cisco CE Common logs
  • [by request] Changed sort order to list directories first in the Browse... window.
  • [by request] Added support for Cisco PIX SL4NT logs
  • [by request] Added a new "paths through a page" view, which shows all paths through a particular page by showing the immediate predecessors and successors in the clickstream.
  • [by request] Added support for a LOGANALYSISINFODIR environment variable which, when set, determines the location and name of the LogAnalysisInfo folder.
  • [by request] Added a number_thousands_separator option, which is a comma (,) by default, and which is used to separate thousands in large numbers.
  • [by request] Extended support for SonicWallKiwi logs
  • [by request] Extended support for Netegrity Siteminder logs
  • [by request] Added support for OpenwaveIntermail logs
  • [by request] Extended support in CiscoIOS log format to support AUDIT_TRAIL information
  • [by request] Added support for SonicWallKiwi (yyyy-mm-dd) log format
  • [by request] Added support for Firebox log format
  • [by request] Added support for Sambar Server log format
  • [by request] Added support for Cisco IDS IOS log format
  • [by request] Added support for Net-Acct log format
  • [by request] Added support for PostOfficeMailServer log format
  • [by request] Added support for IISFTP log format
  • [by request] Added a new option to make graph bars non-clickable (by default, they are clickable). This is useful in cases where there are so many bars that the overheard of making them clickable bogs down the network, Sawmill server, and/or browser.
  • [by request] Added support for Sidewinder log format
  • [by request] Added support for InterscanEmailViruswall log format
  • [by request] Extended support of TinyPersonalFirewall to include a new variety of log
  • [by request] Added support for TomcatAlt log format, also Greg modified Sawmill so that it can collect and carryover
  • [by request] Added support for SquidGuard log
  • [by request] Added an accept_collected_entry_regexp_carryover filter, which works like accept_collected_entry_regexp except that it carries the collected values over rather than resetting them. So a single field value can be collected early on, and later acceptances will all use that value, even if it does not appear again.
  • [by request] Added support for MailerDaemon log
  • [by request] Improved Performance for Cisco PIX KIWI ISO
  • [by request] Added support for Cisco PIX KIWI ISO.
  • [by request] Added support for Whistle Blower Performance Metrics Log.
  • [by request] Added support for Netscape Directory Server log format.
  • [by request] Added "numerical ascending" and "numerical descending" table sorts.
  • [by request] Improved W3C formats to support x-datestamp fields.
  • [by request] Added support for MailMax SE POP log format.
  • [by request] Added support for NetCache NetApp log format.
  • [by request] Added support for MailMax SE SMTP log format.
  • [by request] Added support for Atom Mesge log format.
  • [by request] Added support for RealProxy log format.
  • [by request] Added support for ColdFusion Web Server log format.
  • [by request] Added support for ColdFusion Application log format.
  • [by request] Added support for Microsoft SQL Profiler log format.
  • [by request] Added support for "m/d/yy h:mm" date/times.
  • [by request] Added support for files (like CSV files) which can have line breaks in the middle of a field, as long as the field is quoted.
  • [by request] Added support for WhistleBlower log format.
  • [by request] Added support for Windows Event log format.
  • [by request] Added support for FedEx tracking log format.
  • [by request] Added support for FastHosts log format.
  • [by request] Improved Postfix log format to handle some different types of log entries.
  • [by request] Added support for Network syslog format.
  • [by request] Added support for Merak SMTP Log Format.
  • [by request] Added support for Merak IMAP/POP3 Log Format.
  • [by request] Improved IPTables format to handle a slight variant.
  • [by request] Added support for WebShield SMTP log format.
  • [by request] Added support for Webtrends Extended log format.
  • [by request] Added support for Zyxel/Kiwi log format.
  • [by request] Improved session IDs so they contain the visitor id, for easier identification of the source of the session.
  • [by request] Added a new type of session filter, which lets you zoom in on particular session using its ID.
  • [by request] Improved the Sessions view (now called "Individual Session(s)" so the session IDs are clickable, and apply a session filter to show only that session.
  • [by request] Improved the Sessions view (now called "Individual Session(s)" to show a click-by-click listing of the session, if there is only one session selected.
  • [by request] Added support for Cisco VPN Concentrator (Comma-separated) log format.
  • [by request] Improved iMail log format to track local domains and "ldeliver" messages.
  • [by request] Added support for Hosting.com log format.
  • [by request] Added support for Cisco PIX (NT Syslog, With Hostname) log format.
  • [by request] Added support for NetScreen log format.
  • [by request] Added support for X-Stop log format.
  • [by request] Added support for Microsoft Proxy logs with m/d/yyyy dates.
  • [by request] Added support for ProFTP log format.
  • [by request] Added support for URLScan log format.
  • [by request] Improved error messages to eliminate the ugly traceback information; on a related topic, you can now click Report It to email any error message to our support email address.
  • [by request] Added an "expand all" link to the "paths through the site" toolbar, to fully expand the entire paths tree.
  • [by request] Added support for %{%Y-%m-%d}t and %{%T}t dates and times in Apache custom log format strings.
  • [by request] Added support for Quicktime Streaming Server logs
  • [by request] Added support for CacheFlowRealMedia & CacheFlowerWindowsMedia logs
  • [by request] Added support for LSMTPAccess logs
  • [by request] Changed the way Sawmill handles GUI requests. Previous versions would handle requests by creating a separate thread within the main web server process; Sawmill now creates a separate process for each new request. This has some major advantages. Most importantly, it improves stability by making it impossible for separate tasks to stomp on each others' memory. This was a particularly big problem on Solaris, where there has been a bug which causes crashes when using Sawmill in web server mode; this bug is now fixed. Other lesser bugs (including some unknown ones) may also be fixed by this change. This change also makes other features possible, including the improved progress feature and the web server administration improvements in this version.
  • [by request] Improved progress reporting in CGI mode and in single-threaded mode. Sawmill now uses the full progress page in all cases. Previous versions used a similar version in CGI/single-threaded mode with Netscape, but the feature used was not available in other browsers (like IE), so it was forced to fall back on a "stacked lines" progress indicator, where the progress information appeared as a series of tiny lines at the top of the page. That is gone-- Sawmill uses reloading progress in all cases now. It even uses reloading progress pages when it's used from the command line, and the new Web Server Administration task list lets you tap into a running command-line build to see full progress information.
  • [by request] Improved the Web Server Administration page so it shows a table of active tasks. You can click a task to see progress information for that task. This works even if the tasks are CGI-based, Scheduled, or command-line-- any task will show up in the list, and you can get full progress information for any task by clicking it. The Web Server Administration was not available in CGI mode in earlier versions, but now that it has a useful purpose, it has been enabled.
  • [by request] Added support CacheFlow RealMedia logs
  • [by request] Added command-line CSV export of the Sessions (summary) and Overview views.
  • [by request] Added support Watchguard WELF log format
  • [by request] Added expansion of path segment larger than a certain size, automatically in offline pages and manually through an "expand all" link in dynamic stats.
  • [by request] Extended support for Netegrity SiteMinder 4.X logs
  • [by request] Added support for Centrinity FirstClass logs
  • [by request] Added checkbox to view/subview filter editor so "incremental" filters can be specified from the GUI.
  • [by request] Added support for syslog-only log format plugins, and "generic" log format plugins. Previous versions of Sawmill supported separately each combination of logging device and syslog server. This could get very ugly, with Cisco PIX as the classic example-- 6.3b3 supported 15 versions of Cisco PIX, all similar but using different Syslog servers. Now, there is a "generic" Cisco PIX plugin, and a simple Kiwi Syslog plugin that is not tied to any particular device, and Sawmill lets you choose both of them, and combined them to support Cisco PIX logged through Kiwi (if only one "generic" and only one "syslog" format match, it will be chosen for you automatically, comparably to how it's done now). We will continue to add new "generic" plug-ins and "syslog" plug-ins until all previous syslog-based plugins have been obsoleted, but there will be a period of overlap. When we're done, the total number of formats supported by Sawmill will be dramatically higher than before, because every one of 20-some syslog formats will work with every of the 20-some generic formats, for 400 new effective supported formats.
  • [by request] Added support for IST logs
  • [by request] Added support for CiscoVoiceRouter (KIWI) logs
  • [by request] Added support for MonitorWare logs
  • [by request] Added support for MonitorWareAlt logs
  • [by request] Added an "allow viewers to rebuild" option. When this option is checked, there will be rebuild/update links in the info bar of the configuration, even if the user isn't administrator, and the user will be able to manually update/rebuild the database by clicking the links. No other administrative operations will be available.
  • [by request] Added ignore_quotes option. This option, which is generally of use only if you're creating a custom format, tells Sawmill to treat quotes (" or ') the same as any other character. Some formats require quotes to be treated specially for optimal processing; some require they they *not* be treated specially; this lets you choose.
  • [by request] Implemented ColdFusion Application (CSV) log format.
  • Improved error handling in CGI mode in the event that the specified "temporary directory" cannot be created, or is a file.
  • [by request] Added update/rebuild links to the info bar in the config menu, to allow for easy updating/rebuilding of database from the statistics (admin only).
  • [by request] Extended support for iMail logs
  • [by request] Added support for Cisco IOS Alt3 logs
  • [by request] Improved W3C support to handle #Fields: lines with a tab after the :. Also, added support for #Start-Date: headers in W3C logs.
  • [by request] Added rekey_collected_entry and reverse_rekey_collected_entry log filter actions. These filters are of interest only to people creating log format plug-ins for strange formats. They allow Sawmill to process log formats where the key field changes as the entry goes through stages, but where there is some value that ties the old and new keys together.
  • [by request] Added a new concatenate2 log filter type, which puts the result in field2 instead of field1.
  • [by request] Added support for Cisco PIX KIWI Alt2 logs
  • [by request] Added support for Cisco SOHO77 logs
  • [by request] Improved the "Data Not Available" error message and page. The error message has been simplified in the event that the viewer is not administrator, and no longer talks about how the data can be made available. When the view is an administrator, the message is similar to before, except that it also includes a "Make Data Available" button at the bottom. Clicking this button adds the necessary xrefs, rebuilds the database, and shows the view again (now with data available).
  • [by request] Added support for Siteminder Netegrity beyond 4.x logs
  • [by request] Added a new "Support email address" field in the Preferences (Miscellaneous tab) which specifies the address where bug reports should be sent. This should be set by administrators of multi-user installations, because the bug reports are usually configuration issues that can be corrected by the Sawmill administrator, and cannot be corrected by us (the software vendor), since we don't have access to the installation. By default, support mail will continue to come to us until this option is changed.
  • [by request] Extended support for Cisco PIX KIWI logs
  • [by request] Added support for GNATBox Kiwi yyyy-mm-dd logs
  • [by request] Improved Squid log format to accept resolved hostnames as well as IPs. Improved export so that the statistics are shown after the export, and the export is done with a .csv file extension and Content-type text/csv. Previously, exports long enough to require progress pages would be left dangling at the progress page after the export completed; now the statistics are shown at the end of the progress cycle. Also, the exported filename was previously the same as the name of the Sawmill executable (e.g. "sawmill" or a derivative), which made it difficult to figure out what format the export was in. Now the file ends with .csv, so the format is clearer (and the browser will be able to open it with an appropriate program, like Excel).
  • [by request] Extended support for Snort logs
  • [by request] Added support for Snort Alternate logs
  • [by request] Added "don't autodetect log format" checkbox to configuration creator, so if the autodetector is getting it wrong, it can be overridden.
  • [by request] Improved "log out" so it takes you back to the configuration login page, if appropriate.
  • [by request] Moved the "single-page summary" to the bottom of the views list, so first-time Sawmill uses won't be discouraged by the long time it takes to generate that view, when they're first clicking through the views.
  • [by request] Added an "extra options" field to Scheduler items, so arbitrary command line options can be sent to Sawmill as part of the scheduled command line task. This allows powerful options like Scheduling different views to be emailed to different people, with different filters, which used to require the use of an external scheduler.
  • [by request] Fixed a bug where errors were not reported properly, especially on Windows (a blank page would appear instead).
  • [by request] Added autodetection of RADIUS format strings, so RADIUS logs with arbitrary fields are now supported.
  • [by request] Added filter numbers to the Log Filters table.
  • [by request] Improved PIX/IOS plugin to track message code field.
  • [by request] Added a WebTrends Extended Log Format plug-in that can work with any syslog.
  • [by request] Implemented a generic Cisco VPN Concentrator log format, which works with any syslog server.
  • [by request] Improved the generic Unix Syslog plug-in to handle a slight variant.
  • [by request] Added support for CentrinityFirstClass (mm/dd/yyyy) log format.
  • [by request] Added a GeoIP database to Sawmill, so it can compute countries from IP addresses. This used to include a "Top countries" view in most configurations.
  • [by request] Improved the "collected log entries" feature of Sawmill so it would recycle collected log entries after a certain number of lines without activity. Previous versions would continue to keep the collected entries in memory for the entire log processing run, which is theoretically necessary in case a related log entry at the very end of the data, but in practice is not necessary, because associated entries tend to clump very close together. In cases of very large datasets, an infinity growth option was impractical, requiring several gigabytes of RAM in some examples. By default, Sawmill now recycles log entries after 10,000 lines of inactivity; this can be customized, and recycling can be disabled to get the old behavior.
  • [by request] Added support for Interscan Proxy log format.
  • [by request] Made filters clearer when active, and added a more prominent brief blurb about filters and views in the Overview, which appears when Filters are active. This is an attempt to make the usage of Filters and Views clearer to beginning users, in particular to make it clear that after clicking an item to apply a Filter, you can click a View down the left to see more information about that item.
  • [by request] Added support for Cisco Router log format.
  • [by request] Added support for Declude log format.
  • [by request] Added support for Bulletproof/G6 log format with mm/dd/yy dates.
  • [by request] Added support for Groupwise Web log format.
  • [by request] Improved FTP log source entry so if you enter *both* a URL and other fields (like username), the values entered in the fields will override those in the URL. Previous versions ignored all fields except the URL, if a URL was specified, so if you entered ftp://mysite.com/ as the URL, and entered the username/password/pathname below, they would be ignored, and anonymous FTP would be attempted.
  • [by request] Added support for Novell Border Manager logs
  • [by request] Added red-arrow icons to entry/exit page views, when "Server Root" is set.
  • [by request] Improved generic Cisco format to handle FW-3 and FW-4 lines better.
  • [by request] Improved Cisco Generic plug-in to correctly detect files that consisted entirely of %FW lines.
  • [by request] Fixed a bug where if the log format matched one or more syslog formats, but didn't match any other formats, Sawmill would show an empty menu of available formats.
  • [by request] Fixed a bug where images did not appear correctly in the progress page generated during a "send view by email" operation.
  • [by request] Fixed a bug where progress pages would reload forever during a "send view by email" operation, never actually showing the final result page, even though the email had been sent.
  • [by request] Split Netscreen format apart from its syslog-dependent section, so it will work with any supported syslog now.
  • [by request] Improved Netscreen format to handle a slightly variant of the format.
  • Split Firebox format apart from its syslog-dependent section, so it will work with any supported syslog now.
  • [by request] Extended support for generic Cisco logs
  • [by request] Added support for whatsUp syslog
  • [by request] Added an "Average bytes transferred per second" item to the Overview.
  • [by request] Added support for SymantecWebSecurity logs
  • [by request] Improved the syslog/nonsyslog split so the syslog format menu is only shown when a "require syslog" format is selected.
  • [by request] Improved format screen so the custom format string fields are only shown when the associated custom format plug-in is selected.
  • [by request] Improved log format detection so "Generic" formats like W3C Generic are only offered as a possibility when no non-Generic formats match.
  • [by request] Added support for GENERIC_IOS Debug IP Packet Detailed Logs.
  • [by request] Improved the log formats list so "General" log format (like General W3C, previously called Generic W3C) only appear if there are no other matches.
  • [by request] Extended support of ServU FTP logs.
  • [by request] Improved handling of DNS server failures; Sawmill now switches to the secondary as soon as it sees the first primary failure. If the secondary also fails at some point, it tries the primary again, etc.
  • [by request] Created generic Unix Syslog log format plugin.
  • [by request] Improved full-Sawmill-URL generation (used to generate bookmark URLs) to look at the SERVER_NAME and SCRIPT_NAME environment variables and use them intelligently in the URL.
  • [by request] Added support for Interscan Email Logformat
  • [by request] Added support for SNORT Portscan logs
  • [by request] Extended support of Cisco PIX KIWI IOS logs
  • [by request] Extended support of Cisco PIX UTC logs
  • [by request] Added "reverse alphabetical" as a statistics table sort order.
  • [by request] Improved the Session Filters editor. The filters no longer display arbitrarily-long menus-- instead, they display the first 1000 options, and you can enter options manually. The rest of the menus can be shown with a click, and the limit (1000) is customizable in the Stats Sizes tab.
  • [by request] Added support for Eims Mail SMTP log format
  • [by request] Improved NetScreen log format to track event types.
  • [by request] Improved GeoIP country lookup to work when DNS lookup is on. Previously, DNS lookup would resolve the IPs, preventing GeoIP from computing countries. Now, Sawmill preserves the IP for GeoIP's use, even if it's resolved, so countries are computed the same regardless of DNS settings.
  • [by request] Added support for Interscan Viruswall Virus log format
  • [by request] Improved handling of situation where the user enters something like logs/* as the log filename, and checks "pattern is a regular expression." * isn't a valid regular expression, but many users don't know that, and confuse regular expressions with wildcard expressions. Sawmill used to give a long-winded error message in this case, talking about wildcard expressions and regular expressions, but we frequently get "bug reports" containing this error message, so that wasn't working either. So Sawmill now tries the "regular expression" as a wildcard expression if it isn't a valid regular expression. In other words, entering logs/* and clicking "pattern is a regular expression" will now work-- it will analyze all files in logs/.
  • [by request] Added support for DecludeSPAM log format
  • [by request] Added red arrow icons to *all* URLs in tables (so you can go to that URL by clicking), not just referrer URLs.
  • [by request] Added support for FlashFXP log format
  • [by request] Added IIS/CGI installation chapter to documentation
  • [by request] Added support for WarFTPAlt log format
  • [by request] Reworked the PIX/IOS filter to be more flexible. Rather than having a huge list of pre-defined line formats, it now has a huge list of pre-defined log *segment* formats. So if new data is present that isn't expected, or if segments are put together in unexpected ways, or if some of the parts are missing, it will still grab whatever it can from each line. Since there are apparently an infinite number of PIX/IOS formats out there, this is necessary to provide something like universal compatibility.
  • [by request] Improved Cisco VPN Concentrator format to extract user bandwidth from "disconnect" lines.
  • [by request] Added date support for bpftTraflog
  • [by request] Added support for tcpdump With Interface Alternate
  • [by request] Added support for bpftTraflog
  • [by request] Improved Declude format to handle more fields.
  • [by request] Added support for qmail (syslog required) log format
  • [by request] Extended support for Bind Query Log Format With Timestamp log format
  • [by request] Added support for Checkpoint Alternate log format
  • [by request] Added a feature to save the contents of the "Create/Update Many Configurations" text box, so when you go back, it's how you last submitted it.
  • [by request] Improved Declude SPAM format to track message and action.
  • [by request] Added support for Radware Load Balancing log format
  • [by request] Added support for BulletproofG6FTDDMMYYYY log format
  • [by request] Added support for Samba Server log format
  • [by request] Added support for URLScan log format
  • [by request] Renamed URLScan to URLScanW3C log format
  • [by request] Extended support for Snort Alt log format and made it into a syslog required log data type
  • [by request] Added support for a new Sidewinder log format
  • [by request] Improve ipchains format to accept a slight variant.
  • [by request] Added support for RaidenFTP log format
  • [by request] Added support for GroupwiseWebmmddyy log format
  • [by request] Added support for Mod Gzip log format
  • [by request] Improved Squid format plug-in to allow headers at the end of each line
  • [by request] Improved TaskLog messages to include lines processed and bytes processed
  • [by request] Added support for Sysreset Mirc log format
  • [by request] Added support for N2H2 log format
  • [by request] Changed the "page view vs. non-page view" log filters, which categorize hits based on whether they are page views or not, so it uses the "file type" field to categorize, rather than the "page" (or equivalent) field. This is a better method because the "file type" field extracts the extension properly even when there are URL parameters, so for instance it will correctly categorize a hit like /images/some.gif?param which would have been categorized as a page view previously.
  • [by request] Added support for zyXEL Communications log format
  • Improved Raptor format to track visitors, allow session information, track hosts individually, and report IPs and ports separately.
  • [by request] Added a recentminutes:N filter that shows hits in the past N minutes.

Version 6.4s1, shipped November 23, 2002

Bugs fixed in version 6.4s1:

  • [by request] Fixed problem with bandwidth support with syslog required log formats
  • [by request] Fixed a bug where pattern filters containing curly brackets did not look right in the Filters bar and the Filter Editor.
  • [by request] Fixed a bug where if pattern filters contained curly brackets, they would not bookmark or link correctly.
  • [by request] Fixed a bug where if the database was auto-updated-on-view, and safe updates were on (the default), Sawmill would fail to update, and would give an error about not being able to copy some directory to itself.
  • [by request] Fixed a bug where the "Don't autodetect log format" checkbox wasn't working properly.
  • [by request] Fixed a bug where #Fields: lines in W3C logs with tabs after the colon were not handled correctly.
  • [by request] Fixed a bug where if a format matched a single non-syslog-required plugin and also a syslog, Sawmill would use them both. Now it uses only the non-syslog-required plug-in.
  • [by request] Improved error reporting in the case that a syslog-require plugin matched but no syslog did.
  • [by request] Fixed a bug where Sawmill did not handle format= lines correctly when field names were surrounded by quotes.
  • [by request] Fixed bug where "show logo in docs header" wasn't working.
  • [by request] Fixed bug where full URLs (like bookmark URLs) were computed incorrectly in some cases in web server mode.
  • [by request] Fixed hostname problem in NetCacheNetApp log format
  • Fixed a bug in session median computation where the wrong two data points were being averaged in the even-number-of-sessions case.

New features in 6.4s1:

  • [by request] Added support for qmail (syslog required) log format
  • [by request] Extended support for Bind Query Log Format With Timestamp log format
  • [by request] Added support for Checkpoint Alternate log format
  • [by request] Added a feature to save the contents of the "Create/Update Many Configurations" text box, so when you go back, it's how you last submitted it.
  • [by request] Improved Declude SPAM format to track message and action.
  • [by request] Added support for Radware Load Balancing log format
  • [by request] Added support for BulletproofG6FTDDMMYYYY log format
  • [by request] Added support for Samba Server log format
  • [by request] Added support for URLScan log format
  • [by request] Renamed URLScan to URLScanW3C log format
  • [by request] Extended support for Snort Alt log format and made it into a syslog required log data type
  • [by request] Added support for a new Sidewinder log format
  • [by request] Improve ipchains format to accept a slight variant.
  • [by request] Added support for RaidenFTP log format
  • [by request] Added support for GroupwiseWebmmddyy log format
  • [by request] Added support for Mod Gzip log format
  • [by request] Improved Squid format plug-in to allow headers at the end of each line
  • [by request] Improved TaskLog messages to include lines processed and bytes processed
  • [by request] Added support for Sysreset Mirc log format
  • [by request] Added support for N2H2 log format
  • [by request] Changed the "page view vs. non-page view" log filters, which categorize hits based on whether they are page views or not, so it uses the "file type" field to categorize, rather than the "page" (or equivalent) field. This is a better method because the "file type" field extracts the extension properly even when there are URL parameters, so for instance it will correctly categorize a hit like /images/some.gif?param which would have been categorized as a page view previously.
  • [by request] Added support for zyXEL Communications log format
  • Improved Raptor format to track visitors, allow session information, track hosts individually, and report IPs and ports separately.
  • [by request] Added a recentminutes:N filter that shows hits in the past N minutes.

Version 6.3.16, shipped January 12, 2003

Bugs fixed in version 6.3.16:

  • [by request] Fixed a bug where session totals were missing from the entry/exit pages view.
  • [by request] Fixed a bug where Sawmill would generate an error if the log source contained no files, even if allow_empty_log_source was checked.
  • [by request] Fixed a bug in FW1 format where times starting with a space were counted as corrupt.
  • [by request] Fixed a security problem where if you logged in to Sawmill, and then visited another site, that site could obtain your Sawmill login information from the browser cookies.
  • [by request] Fixed a bug where the ?browse option on the Sawmill URL would not log in properly.
  • [by request] Fixed a bug where session information was sometimes not properly removed during database expiration.
  • [by request] Fixed a bug where if session information was not properly removed (due to the above bug), it could cause a crash when another expiration was done)
  • [by request] Improved verbose output so it doesn't use carriage returns (\r) characters when it's redirected to a file.
  • [by request] Fixed a bug where the authentication_command_line option did not work under Windows.
  • [by request] Fixed a security flaw where configuration names specified in a URL using "view" or "open" options were not being screened properly, making it possible to remotely view the beginning of arbitrary files on the server.
  • [by request] Fixed a bug where if Sawmill was running in web server mode, a carefully- constructed HTTP POST operation could cause the server to terminate; i.e. this is a "denial of service" attack.
  • [by request] Fixed a bug where even if config_mode was browse_only, it was still possible to get to the Configuration Menu by logging in administratively, viewing statistics, and clicking the configuration name at the bottom.
  • [by request] Fixed a bug where expiring a database with visitor information would sometimes cause a crash.
  • [by request] Worked around a browser bug where links to Sawmill6CL.exe without any parameters would fail on Windows in some cases, with some browsers.
  • [by request] Fixed a bug where if you used Sawmill with non-English language modules, in some cases it would take you to the wrong view when you clicked a link.
  • [by request] Fixed a bug where if you used "Add Log Data To Database" and used multiple log sources, the Log Source for the configuration would be permanently modified.

New features in 6.3.16:

  • [by request] Added detection of Phoenix and Netscape 7 browsers.

Version 6.3.15, shipped November 16, 2002

Bugs fixed in version 6.3.15:

  • [by request] Fixed a bug where Sawmill could crash while processing log formats which used collected log entries and a global date (which is very uncommon, but Communigate is one such format).
  • [by request] Fixed a bug where if the final field of a log entry was in quotes, the final quote would be included in the field value.
  • [by request] Reworked MIME encoding of emails so they display properly on Eudora (and on Entourage, Outlook Expression, and Apple Mail).
  • [by request] Fixed a bug where session views were not displayed in some cases, even when session information really was available.
  • [by request] Worked around a bug in IE where saved pages would be incorrect if the displayed value of a link was a URL.
  • [by request] Fixed a bug where if the log_analysis_screen_info.gif file wasn't at the top level of the site, screen information would not be extracted correctly.
  • [by request] Fixed a bug where for certain combinations of xrefs, the date/time field would not be present in every xref group even when it was xreffed to all fields. This wasn't a bug per se, but since expiration requires that the date/time field appear in every group, and the only way of setting groups via the GUI is checking the boxes, it was necessary to add an extra rule that if all date/time boxes are checked, date/time would be added to any xref group that normally wouldn't have it, so expiration would work.
  • [by request] Fixed a bug where Microsoft Entourage did not display HTML emails sent by Sawmill correctly.
  • [by request] Fixed bug where the "user" field was not extracted properly from Rapter logs.
  • [by request] Fixed a bug where Sawmill could consume large amounts of memory while waiting for the final DNS requests to time out.
  • [by request] Fixed a bug where Sawmill could crash while processing certain uncommon types of log data with empty field values.
  • [by request] Fixed a bug where bandwidth was reported incorrectly in the "computed" weekday and hour views, for high-volume sites.

New features in 6.3.15:

  • [by request] Added direct support for WebSTAR W3C log format, and fixed a bug which could cause the old method of processing those files (using the Generic W3C plug-in) to fail.
  • [by request] Improved local IP address detection on UNIX, to handle situations where the system libraries give a local IP address that isn't a valid one for the local machine (this has been observed in many cases on MacOS 10.2). Sawmill now tries running ifconfig first, and tries to get the IP from there, and falls back on the system libraries only if that fails.

Version 6.3.14, shipped October 19, 2002

Bugs fixed in version 6.3.14:

  • [by request] Fixed a bug where on some pages (including the Configuration Options page and the Configuration Menu), if you clicked an option, then clicked Back, then clicked another option, Sawmill would act as though you had clicked the first one again.
  • [by request] Fixed a bug where Netscape proxy logs would not show page information correctly.
  • [by request] Fixed a bug where when generating "all documentation," pages would be preprocessed twice, sometimes resulting in bizarre links or other errors.
  • [by request] Fixed a bug where FTP log source connections would hang when connecting to certain FTP servers.
  • [by request] Improved support for W3C format so it can handle extra spaces after #Fields:.
  • [by request] Fixed a bug where if you used a regular expression filter on a field whose name contained a space, in a View filter, you would get an error.
  • [by request] Fixed bug in Blue Coat Custom format strings, where the date/time format for %x was wrong.
  • [by request] Improved Blue Coat Custom format strings to handle it better when there's a %U field but no %i field.
  • [by request] Fixed a bug where the page views in the "time per page" view were appearing in a columned labelled "Sessions".
  • [by request] Fixed a bug where if you attempted to export a view from the command line, but that view was hidden, then the export would fail with a "no such view" error.
  • [by request] Fixed bug where if you tried to use a "clp" URL to log in to view a configuration, when you were already logged in and viewing another configuration, you would get an error and an Admin login page.
  • [by request] Fixed a bug where the configuration name was not quoted in the "Start Over" link, so the link didn't work when running Sawmill in CGI mode under IIS on Windows.
  • [by request] Fixed a bug where Sawmill could crash while analyzing log data that contained screen dimensions information.

New features in 6.3.14:

  • [by request] Fixed a bug where progress was not shown properly for Generate HTML Files on some browsers (including Opera).
  • [by request] Changed the Calendar view to be invisible offline by default. This eliminates a common problem where a new user creates a new configuration with "day-by-day" checked, and is then shocked to find the Generate HTML takes hours because it's generating a separate set of statistics for every year, month, week, and day in the dataset. Because the Calendar is now invisible offline, only overall statistics will be generated by default. If you need day-by-day offline statistics, you can make the calendar visible offline in the Stats Views tab.
  • [by request] Improved browser detection to report AOL browsers.

Version 6.3.13, shipped September 30, 2002

Bugs fixed in version 6.3.13:

  • [by request] Fixed a bug where Sawmill would generate an error when attempting to send email.

Version 6.3.12, shipped September 29, 2002

Bugs fixed in version 6.3.12:

  • [by request] Fixed a bug where the "all docs" page had multiple tags, causing some browsers to freak out and reload it forever.
  • [by request] Fixed a bug where a command-line password would not be carried through into the statistics, resulting in another password prompt later.
  • [by request] Fixed a bug where white-labelled versions of Sawmill would show "COPYRIGHT_HOLDER" in the credit of the statistics.
  • [by request] Fixed a bug where integer visitor id values were incorrectly identified as IP addresses by Sawmill, and were stored in a compact IP address format in the database, with the result that they appeared as strange IPs in the statistics.
  • [by request] Fixed a bug where certain regular expression statistics Filters would break the statistics.
  • [by request] Fixed a bug where if a log file was empty, "update" operations could re-add previously added data.
  • [by request] Fixed a bug where if there was no date/time field, Sawmill would reject all entries.
  • [by request] Fixed a bug where you could get a weird error updating a database if it didn't have visitor information.
  • [by request] Fixed a problem where Sawmill would overwrite the configuration file every time the configuration menu was displayed. This isn't normally an issue, but it could be a problem if configuration options were modified during a long GUI-based database builds, because modifications will be lost when the build completes and displays the final config menu.
  • [by request] Fixed a bug where modifications to the log fields and database fields would be lost on rebuild, if there was a custom log format string.

Version 6.3.11, shipped September 12, 2002

Bugs fixed in version 6.3.11:

  • [by request] Improved Squid log format to accept resolved hostnames as well as IPs.
  • [by request] Fixed bug where using session filters on a visitor ID containing spaces would cause an error.
  • [by request] Fixed a bug where documentation searches could use unreasonable amounts of memory.
  • [by request] Fixed a formatting bug in "paths through the site" that resulted in "nbsp" appearing in the statistics, when viewed with some browsers.
  • [by request] Fixed a bug where the multiplier was wrong in the hits equation in the date/time graph.
  • [by request] Fixed a bug where Sawmill would prompt for the admin password to be chosen even if command-line authentication was being used, and would allow the admin password to be entered.
  • [by request] Fixed a bug where if the OPTION_SHORT_DESCRIPTION_ variable in a language module contained a newline (which could happen during translation of Sawmill to another language), generated configuration files would not work because of invalid comment syntax.
  • [by request] Fixed a possible security hole when using command-line authentication on UNIX. (which is disabled by default). Sawmill was passing the username and password to the shell, which could conceivable be used together with other security holes and shell variables/expansions to attack the server running Sawmill. This is particularly problematic if Sawmill is running as root. This version fixes this by using a fork/exec instead of popen, so it doesn't expand shell constructs.
  • [by request] Fixed a bug where weekday tables were sorted Sunday-first, even if a different first day of week had been selected.
  • [by request] Fixed a bug where entry/exit page CVS exports would have
    's in the middle of some values.
  • [by request] Fixed a bug where some files were created world-writable.
  • [by request] Fixed a bug where local log source patterns were case-sensitive on Windows, even though Windows file system isn't.
  • [by request] Fixed a bug where if the FTP URL was incorrectly formatted, Sawmill would generate an error when the configuration was edited, preventing the URL from being corrected.
  • [by request] Fixed a bug where multiple log sources were not reported correctly in the "latest operation" blurb and TaskLog.
  • [by request] Fixed bug where {} fields in Apache custom log formats were not handled correctly
  • [by request] Fixed a bug where certain types of misquoted (corrupt) log data could cause Sawmill to crash during processing.
  • [by request] Fixed a bug in non-English versions of Sawmill where the subview editing controls in the Stats Views tab did not work properly.
  • [by request] Fixed a bug were bookmarks were always HTTP even if Sawmill was running under an HTTPS server.
  • [by request] Fixed bug where virtual host field in Apache custom log formats was not detected correctly.
  • [by request] Fixed a bug where Sawmill would generate an internal error when analyzing logs with screen info, if there was no cs-uri-query field in the log data.
  • [by request] Fixed bug where pending DNS queries were discarded at end of log processing. This caused the last few log entries to be ignored. In extreme cases involving very small files, this could even cause the entire file to be ignored, resulting in a "no data" error.
  • [by request] Fixed a bug where if there was a W3C header in the log data, but the log wasn't really a W3C log, and the format said to ignore the headers, Sawmill would still process #Date lines, resulting in invalid dates.

New features in 6.3.11:

  • [by request] Added a new perl script to Extras, called dbiauth.pl, which performs command-line authentication from a database (similar to ldapauth.pl, which does it using LDAP).
  • [by request] Added a language module variable DB_DATA_MODIFICATION_DATETIME, which contains the modification date/time of the current database. This can be used in headers or footers to show when the database was last updated.

Version 6.3.10, shipped August 01, 2002

Bugs fixed in version 6.3.10:

  • [by request] Fixed detection of Netscape 6 browsers.
  • [by request] Restored the "Look up IP numbers before filtering" option, which was disabled a while ago, with the intent to reenable it soon, but never got reenabled.
  • [by request] Fixed a bug where the Session Filter Editor would only show pages/visitors which are in the current session filter set (instead of showing them all).
  • [by request] Fixed a bug where bookmarks made from password-protected configurations were not password-protected. Now they require the same password as the original configuration.
  • [by request] Fixed a bug where expiring a database could cause an internal error.
  • [by request] Fixed a bug where DNS counters were not being properly initialized, resulting in semi-random DNS counts in the progress page during database erasure.
  • [by request] Fixed a bug where the SUPPORT_EMAIL language module variable was overridden by the default, even if it was specified in a language module.
  • [by request] Fixed a bug where previous-seen files could be re-processed during Update operations.
  • [by request] Fixed a bug where the Options menu in subviews of the Stats Views tab sometimes did not show all the options it should have.
  • [by request] Fixed a bug where Sawmill would reject every log entry in files which contained a #Date line in yyyy-mm-dd format, but no separate dates in log entries.
  • [by request] Fixed a bug where spider detection (via the Spiders file) did not stop when the first match was found, but continued through the entire list. The first match is supposed to be the winner, so this did not work as advertised. Now, it stops when it finds the first one, and that one becomes the spider field value.
  • [by request] Fixed a bug where IP address comparisons were being done in reverse in Log Filters (e.g. "IP1 < IP2" was considered true when IP1 > IP2).
  • [by request] Improved W3C web server formats to handle cs-host (server domain) fields.
  • [by request] Fixed Postfix log format so it tracks bandwidth correctly.
  • [by request] Fixed a bug where generating HTML files could cause a "mismatched parentheses" error.
  • [by request] Fixed a bug where file types were not reported correctly on files with URL parameters.
  • [by request] Fixed a bug where if the "Minimum page views on a page to link" option was set, then rows with fewer than that number of page views would be blank.
  • [by request] Fixed a bug where Sawmill could generate an internal error in some uncommon cases, while accessing a database.

Version 6.3.9, shipped July 12, 2002

Bugs fixed in version 6.3.9:

  • [by request] Fixed a bug where Apache LogFormat strings were not being processed correctly on Windows in some cases.
  • [by request] Fixed a bug where adding data to the database using "Add Log Data" would modify the main Log Source.
  • [by request] Fixed a bug where the mmdddyyy log format was not supported properly.
  • [by request] Fixed a bug where the Sawmill would generate an error if you tried to generate static files and your first view was invisible.
  • [by request] Fixed a bug where the "permission" values were not formatted properly in Netscape.
  • [by request] Fixed a bug where Sawmill would try to close FTP connections twice, when multiple FTP log sources were used. This could result in very long delays during FTP processing.
  • [by request] Fixed a bug where Sawmill would report certain error messages incorrectly when the "Report It" feature was used.
  • [by request] Fixed a bug where Sawmill could get in a state where it skipped previously-seen data past the end of the file, skipping part of the next file too, even if that data hadn't been processed yet.
  • [by request] Fixed a bug where non-standard ports on FTP log sources would be lost, and would have to be re-entered every time they were used.
  • [by request] Fixed a bug where attempts to analyze certain log format would fail with the message "Unknown date format ddmmmyyyy."
  • [by request] Fixed a bug where Sawmill could generate an error if you used a + in a form field.
  • [by request] Fixed a bug where the blurb at the bottom of the Configuration Menu would contain incorrect information about the log source, if it was a single-file log source or if it was a "multiple" log source.
  • [by request] Fixed bug where sorting search terms alphabetically would generate an internal error.
  • [by request] Fixed a bug where there were no Instructions for the "screen depths" view.
  • [by request] Fixed a bug where certain types of invalid filters could be created in the GUI, but would then prevent you from correcting them in the GUI.
  • [by request] Fixed a bug where Sawmill could crash in some situations where the log data changed format in the middle.
  • Fixed a bug where if a file contained a #Date line in a format other than dd/mmm/yyyy, Sawmill would not track the date field correctly.
  • [by request] Fixed a bug where screen information was not tracked in some cases in W3C logs.
  • [by request] Fixed a bug where 404 (broken link) information was not reported in some cases, even if it was requested.

New features in 6.3.9:

  • [by request] (This was actually introduced in 6.3.7). A new distribution of Sawmill is available for download-- the "encrypted source code" release. This release is a C/C++ source code release which can be used to build Sawmill on any system with a C/C++ compiler and a "make" program, even if it's not one of the systems we normally ship Sawmill for. This distribution is bound by the same license terms of previous ones-- you may not attempt to reverse-engineer, decrypt, or view the source code. It may only be used to build Sawmill. This distribution makes Sawmill vastly more cross-platform than it was before, extending its supported platforms from about twenty to virtual every platform in the world. The build is done through a simple three-step command-line process (the extract/configure/make process familiar to GNU autoconf users).

Version 6.3.8, shipped June 07, 2002

Bugs fixed in version 6.3.8:

  • [by request] Fixed a bug where the IPNumberCache file was not being correctly written, resulting in errors on later attempts to read it.

Version 6.3.7, shipped June 05, 2002

Bugs fixed in version 6.3.7:

  • [by request] Fixed a bug where the IPNumberCache would become corrupted, causing nearly-random results of lookups when it was used in later builds. IMPORTANT: This version fixes the bug that causes the corruption, but you will need to manually delete the IPNumberCache file in your LogAnalysisInfo folder to fix any corruption caused by past builds.
  • [by request] Fixed a bug where links from emailed views would sometimes lead to the Administrative menu login.
  • [by request] Fixed a bug where Sawmill would sometimes report a number (like 32) as the error message, when it really meant to say, "you're out of disk space."
  • [by request] Fixed a bug where Sawmill would not handle IIS logs correctly in some cases if they contained large numbers of null characters (as IIS logs often do). This could result in the log after the null-character log being skipped, or included in the previous day's statistics.
  • [by request] Brought Sawmill for MacOS 9 up to date. Version 6.3.5 for MacOS 9 had a bug that made it completely unusable, and was promptly pulled back to 6.3.4. Version 6.3.6 never shipped for MacOS 9 at all, due to build difficulties. The build difficulties are resolved and the bug is fixed, so Sawmill for MacOS 9 is back. (Sawmill for MacOS X was not affected by either issue).

New features in 6.3.7:

  • [by request] Added support for a different variant of Firewall-1 log format.

Version 6.3.6, shipped May 29, 2002

Bugs fixed in version 6.3.6:

  • [by request] Fixed a bug where in come cases, the last slice in a pie chart would be white, when it really should have been some other color.
  • [by request] Fixed a bug where the link back to the Admin menu from the "send view by email" response page was broken.
  • [by request] Improved Squid format to handle a slight variation (an extra space at the end of each line).
  • [by request] Fixed a bug where the "server response" field was not being compared as an integer in log filters, so "response > 400" did not work.
  • [by request] Fixed a bug where Sawmill would re-process some log data it had already processed, during a database update.
  • [by request] Fixed a bug where Sawmill would frequently crash when run on Solaris, with an "Alarm Clock" error printed on the console.
  • [by request] Fixed the browser-matching algorithm to recognize Netscape 6, Mozilla, and Opera.
  • [by request] Fixed a bug where screen resolution information was not being tracked for W3C logs.
  • [by request] Fixed a bug where Sawmill could crash when displaying an error message on UNIX-style systems without a uname command.
  • [by request] Fixed a bug where the 404s view did not appear in statistics, even if was requested.
  • Fixed a bug where there were several formatting problems with a CVS-exported Sessions view.
  • [by request] Fixed a bug where the Start Over link was broken in offline pages.
  • [by request] Fixed a bug where the name of the Sawmill executable was missing in some examples in the documentation.
  • [by request] Fixed a bug where screen dimensions hits were being reported in the "Top pages" view, instead of being suppressed.
  • [by request] Fixed a bug where Sawmill could hang during FTP transfers.
  • [by request] Fixed a bug where Sawmill was incorrectly attempting to sum visitors in the "Top weekdays (C)" view.
  • [by request] Fixed a bug where certain operations (including scheduled updates) would fail when run on MacOS 9, with an Internal Error that said that use_tcp_for_dns was not set.
  • [by request] Fixed a bug where URLs ending in / were not being properly grouped into the log field hierarchy, resulting in situations where the total for the "Top URLs" view did not match the total for the "Top URLs/directories" view.

New features in 6.3.6:

  • [by request] Fixed a bug where the IPNumberCache file could become corrupted on Windows, causing Sawmill to crash when it attempted to re-process log data with DNS lookup turned on.

Version 6.3.5, shipped April 21, 2002

Bugs fixed in version 6.3.5:

  • [by request] Fixed a bug where configurations would not show up in the administrator's View Statistics menu if the "Hide in User menu" option was checked.
  • Fixed a bug where the images were broken in the documentation in CGI mode.
  • [by request] Fixed a bug where some of the links could be broken in the documentation (e.g. the Admin Menu link), if a non-standard executable name was used.
  • [by request] Fixed a bug which could cause Sawmill to hang during database builds if DNS resolution was turned on.
  • [by request] Fixed a bug where Sawmill could hang at the end of database builds in some situations.
  • [by request] Fixed a bug where Sawmill was miscomputing/misreporting the sizes of moderate-sized files (e.g. a few hundred Meg). This could result in wildly inaccurate size reports in the Browse window, and similarly inaccurate progress estimates. This bug was caused by an attempt in the previous version to fix a bug where Sawmill miscomputed the size of large files (2+ Gig). We have not been able to find a way to report both correctly, so for the moment we're favoring the small ones. Large files may have their sizes misreported in the Browse window, and the progress indicator may be off for them, but other than these cosmetic issues, they should not cause problems.

New features in 6.3.5:

  • [by request] Improved Microsoft Syslog format to track usernames and other login/logout-related fields, and to treat usernames as visitor IDs and the "action" as the page field. This makes it possible to do "clickstream" analysis of this format, which is a little weird since the analysis acts as though the log is a web log, but is nevertheless very useful because it lets you see session logins/logout and session durations for particular users, or in aggregate, as well as other session-related information.
  • [by request] Added support for a new variant of INN Log Format.
  • [by request] Improved HTTP log sources to use HTTP 1.1. This allows them to fetch log data from virtually-hosted domains, which was not possible in previous versions.
  • [by request] Improved Raptor format to track the "rule" field.

Version 6.3.4, shipped April 14, 2002

Bugs fixed in version 6.3.4:

  • [by request] Fixed a bug where certain types of log corruption could cause Sawmill to stop processing the file, or even stop processing all further files in the log source.
  • [by request] Fixed a bug where if the log data contained blank lines, sections of the data after the blank lines could be ignored.
  • [by request] Fixed support for Microsoft Exchange format, which was not handling some time fields correctly.
  • [by request] Fixed a bug where if you used compound filters (e.g. week filters), and you used a configuration password, direct links to the configuration (e.g. from email) would not work.
  • [by request] Fixed a bug where if you zoomed on a day, and then clicked "Top days," it would take you to the Overview.
  • [by request] Fixed a bug where a "size" log field would appear in the Log Fields tab of the Configuration Options, even if there was no bandwidth in the database, and would refuse to be deleted.
  • [by request] Fixed a bug where Sawmill could crash in some cases if two people were using it remotely at the same time.
  • [by request] Fixed a bug where the size of very large files (more than 2 Gig) could be reported incorrectly, as too small or even negative.
  • [by request] Fixed a bug where Sawmill could crash if a value was entered in the "Matches" field for a statistics filter, and did not match anything.
  • [by request] Fixed a bug where weekdays were sometimes in English, even if they had been translated in the language module.
  • [by request] Fixed a memory leak where Sawmill was not freeing session memory properly during HTML file generation, resulting in higher and higher memory usage as the generation proceeded. Also, if session memory was exceeded, all further generated views would have a "session memory exceeded" error, even if they weren't session views.

New features in 6.3.4:

  • Improved the "verbose" command-line option, so that individual components of the debugging output can be turned on or off separately. This allows for detailed debugging of one aspect of the output, for instance the filters or the DNS lookup, without having to see all of the other, unrelated, debugging output. Also, improved the number and types of output generated, including support for network output and improved DNS output.
  • [by request] Improved the view sidebar so the color of view button text is red, and the button is dimmed, when there is no data available for that view.
  • [by request] Added support for LSMTP log format.
  • [by request] Added support for a new variant of iptables logs.
  • [by request] Added support for Microsoft Exchange 2000 log format.
  • [by request] Added support for GroupWise log format.
  • [by request] Improved support for Bind Query log format with timestamp, to support a slight variation of the format.
  • [by request] Improved support for Microsoft Proxy log format, to track "remote port" values and report URLs better.
  • [by request] Added support for CatOS Syslog format.
  • [by request] Added support for a new variant of Cisco IOS log format.
  • [by request] Added a "Start Over" button to the view sidebar in the statistics, to clear the filters and start over viewing the statistics.
  • [by request] Added a new "Templates" language module which will eventually contain HTML templates of all pages Sawmill generates, so it will be possible to customize the look and feel of the interface. For the moment, it contains only templates for the documentation. Future versions will use templates for all pages. A new documentation look-and-feel has been implemented using this new templating feature.
  • [by request] Added an option to preprocess (or not preprocess) the headers and footers. Previous versions always preprocessed; even older versions never did; now you can choose. Preprocessing lets you include variables like the configuration name in your header, but makes it difficult to include CSS and other code that uses curly or square brackets.

Version 6.3.3, shipped April 05, 2002

Bugs fixed in version 6.3.3:

  • [by request] Fixed a bug where the Scheduler was counting months incorrectly, so a job scheduled for March would actually run in February.
  • [by request] Fixed a bug where if bandwidth was not being tracked, and the log format was specified using a regular expression, Sawmill could crash while processing log data.
  • [by request] Fixed a bug where Sawmill could crash if an invalid log format string was entered.
  • [by request] Fixed a bug where the %g field was not identified correctly in custom CacheFlow Reporter logs.
  • [by request] Fixed a bug where Sawmill could crash when processing some types of log data (including Communigate).
  • [by request] Fixed several bugs in DansGuardian log format, which prevented Sawmill from counting some lines, and from tracking some fields.
  • [by request] Fixed bug where Sawmill would report incorrect or garbled values in the Filter bar in some circumstances.
  • [by request] Fixed a bug in support for Microsoft Proxy Log Format, where Sawmill was tracking the client-to-proxy traffic instead of the proxy-to-client traffic.
  • [by request] Fixed a bug in CacheFlow Custom log formats, where Sawmill would not show complete IPs in the "Top hosts" view, even if you had asked for them.
  • [by request] Fixed a bug in CacheFlow Custom log formats, where Sawmill would not show the "Top URLs" view, if URLs were present as part of a %r log format specification.
  • Improved Sawmill's handling of corrupt compressed log data. In previous versions, Sawmill would stop processing the log source if it found corrupt log data. That meant that a single corrupt file could potentially cause Sawmill to ignore all the data in other, valid files. Sawmill now skips the the remainder of the corrupt file when it encounters a problem, but continues processing the other files.
  • [by request] Fixed a bug where it was possible to delete all log sources, which would get in a state where many configuration operations would fail with an "internal error."
  • [by request] Fixed bug where Sawmill was not handling FTP server timeout correctly. When Sawmill processes compressed log data by FTP, it downloads it first, then decompresses it from disk and processes it, and finally deletes the file. This could cause problems if the processing time for the file was so long that the ftp server timed out. Sawmill sends NOOPs to try to prevent this, but some ftp server will time out anyway. Sawmill now closes and reopens the FTP connection for every file, which may make database builds somewhat slower if you have many small files, but won't make much of a difference in most situations. By reopening the connection, Sawmill ensures that FTP timeouts will not cause the database build to fail.

New features in 6.3.3:

  • [by request] Added a "logout URL" option, to let you redirect logouts to a particular URL.
  • [by request] Added support for Microsoft Exchange log format.
  • [by request] Added support for IIS SMTP log format.
  • [by request] Added a "log_data_type" option, which is a clue to Sawmill about the type of server/device/etc. that generated the log data. This helps Sawmill make some decisions when creating the configuration, and it allows the instructions and view names to be more appropriate; for instance, it no longer talks about "hits on the site" if the log data is from a firewall/proxy/cache, and it no longer mentions .htaccess authentication when displaying the "top users" for an FTP server.
  • [by request] Added support for Gauntlet log format with yyyy-mm-dd dates.
  • [by request] Added support for Windows Syslog log format.
  • [by request] Added support for "lastmonth" filter on the command line, to show hits from last month.
  • [by request] Added support for UDP DNS. Sawmill has been using TCP to communicate with DNS servers; now you can choose TCP or UDP. UDP is more frequently used for DNS, and is generally faster, so it now is the default. Because UDP is faster than TCP, Sawmill now processes log data faster (in some cases, several times faster) than previous versions when DNS is enabled (especially if the connection to the DNS server is slow). Because some DNS servers support only UDP, Sawmill now works with a wider range of DNS servers. TCP also has a way of leaving thousands of sockets waiting to close, which can cause problems with some platforms (including Windows); UDP solves this issue.
  • [by request] Improved the log source interface to show examples of appropriate field values.
  • [by request] Added support for INN News log format.
  • [by request] Added support for WU-FTP (yyyymmdd Dates, Server Domain) Log Format.
  • Improved the progress indicator to show bytes processed and bytes processed per second.
  • [by request] Added support for regular expression statistics filters. You can now enter a regular expression, in the Edit Filters page, to have Sawmill show you all items matching that expression. For instance, if you want a view showing only error server response pages, you can create a view with a page table and a regular expression filter "$[45].." on the server response field, to show only pages resulting in 4xx and 5xx server responses.

Version 6.3.2, shipped March 24, 2002

Bugs fixed in version 6.3.2:

  • [by request] Fixed a bug where the "log source" page would generate a JavaScript error, complaining about mainform.lsu.
  • Fixed a bug where Sawmill could crash in certain (rare) circumstances, when using the "Page frame command" option.
  • [by request] Worked around an problem with Opera browser where images would take a very long time to load.
  • [by request] Fixed a bug where Sawmill could get stuck in a state where the log source URL was set as http:// internally, and everything you tried to do just printed an error "Invalid URL format: http://"
  • [by request] Fixed a bug where some older versions of Windows, and/or older versions of Internet Explorer, would try to "download" Sawmill6CL.exe instead of displaying the page from it.
  • [by request] Fixed a bug where Sawmill would use all available CPU, even when doing nothing, under MacOS X.
  • [by request] Fixed a bug where the "trusted host" feature didn't work.
  • [by request] Fixed a bug in Squid log formats where Sawmill was ignoring DEFAULT_PARENT lines.
  • [by request] Fixed a bug where if there was just one log source, and that was collapsed, then it was difficult to modify it (you had to create another, expand the first, and delete the new one). Sawmill now always expands a log source if it is the only one.
  • [by request] Fixed a bug where if you deleted a filter, field, logs source, etc., and used the Back button to go back to a page where it had not yet been deleted, Sawmill could crash when the page was submitted.
  • [by request] Fixed a bug where clicking Next> in the "Add log data" page would generate an error.
  • [by request] Fixed a bug where Create Many Configurations was not updating the log sources properly.

New features in 6.3.2:

  • [by request] Added support for CacheFlow Alternate log format.
  • [by request] Improved the <, >, <=, and >= log filter operations so they recognize IP addresses, and compare them numerically if both operands are IP addresses.
  • [by request] Added support for yyyy/mmm/dd dates.
  • [by request] Added support for Cisco VPN Concentrator log format.
  • [by request] Added support for Steel Belted Radius ACT log format.
  • [by request] Added reporting of 2-time, 3-time, 4-time, 5-time, and more-time visitors to the "Sessions (overview)" view. Added reporting of mean and median visits per visitor.
  • [by request] Added support for filtering based on a wildcard pattern; for instance, you can now show all PDF files by entering *.PDF as the page filter.
  • [by request] Added support for Netegrity SiteMinder log format.
  • [by request] Added a new option "View email subject," which lets you set the subject of emailed views.

Version 6.3.1, shipped March 18, 2002

Bugs fixed in version 6.3.1:

  • [by request] Fixed a bug where Sawmill would generate a security warning when saving the Preferences or Default Configuration.

Version 6.3, shipped March 17, 2002

Bugs fixed in version 6.3:

  • [by request] Fixed a bug where if you tried to create a directory in the Browse window, but didn't have permission to do it, it would generate a mangled page, or a partial page.
  • [by request] Fixed a bug where with certain log format (including iMail), Sawmill could run out of internal log entry objects during processing, causing processing to hang.
  • [by request] Fixed a bug where the "day-by-day" and "404s" checkboxes didn't work if there were no xrefs explicitly listed in the log format plug-in.
  • Fixed a bug where Sawmill would read data from the database incorrectly in certain unusual cases. Most often, this would result in values showing up zero in the statistics when they really weren't, but it could theoretically cause incorrect values to appear as well.
  • [by request] Fixed a bug where if you were clicking around the interface quickly, all threads could be left in "waiting" states, and the interface would stop responding (or at least, stop serving images quickly).
  • [by request] Fixed a bug where Sawmill could crash when editing scheduled items, in certain circumstances.
  • [by request] Fixed a bug where Sawmill was ignoring the last data point in the current filter (assuming it was a partial day), rather than ignoring the last data point in the entire database.
  • Fixed a bug where Sawmill was displaying the "ignoring last data point" message even in the "top weekdays (C)" view, where it doesn't ignore anything.
  • [by request] Fixed a bug where the "server response" field was not being tagged with English descriptions (i.e. "Not Found" for 404) when the log format used a format header line.

New features in 6.3:

  • [by request] Redesigned the log source interface. Previous versions had a single field where all log source values could be typed, one log source per line. The new interface returns to the style of Sawmill 6.0, by using a more user-friendly form-based interface for selecting the log source (for instance, in FTP mode there are separate hostname, username, password, and pathname fields). Multiple log sources are supported using a "yellow box" interface similar to that used in other parts of the Sawmill interface. The layout of the "Where is the log data" page has been improved by putting the documentation in a column at the right, and adding another Next button in a more convenient location. The new page also makes it clear that Browse applies only to local files. Finally, the regular expression and "process files in subdirectories" options can now be specified on a case-by-case basis, rather than applying to all log sources.
  • [by request] Added a "worm" field and view, which tracks hits from known worms (i.e. nimda, Code Red). Also added a default filter to convert all worm page values to (worm)-- this effectively removes worm hits from the "top pages" view, so they don't clutter up the "real" hits.
  • [by request] Changed the way Sawmill tracks hostnames, to make it easier to zoom in on particular hostname to see the statistics for that host. Previous versions of Sawmill tracked hostname fields hierarchically by default, with .com at the top level, second-level domains like yahoo.com available by clicking .com, and no further information available. This version now gives you the choice, when you create the configuration, of using either that approach, or of showing all hosts in a non-hierarchical view. Tracking hosts individually requires much more memory, time, and disk space, but makes it possible to zoom in on a particular hostname, if desired, to see traffic for just that host. Tracking hosts at the domain level requires fewer resources, but lacks the detail of individual host tracking. Many of the included log formats had already addresses this partially by using a "flat" field type for the host field, which gives roughly the same results as checking the box. In this version, all these have been changed back to the "host" field type, so you have the choice of flat or hierarchical when you create the configuration. Using the "host" field type also makes it possible to look up the IP addresses of that field.
  • [by request] Restored the "save my password" feature that was removed as part of the 6.2.15 authentication overhaul. It doesn't work quite the same as before-- it now fills in the username/password fields for you, rather than bypassing the login screen entirely as it did before The new method lets you easily undo the save password, or change the saved password, which was difficult or impossible before (without editing the browser cookies directly).
  • [by request] Added support for arbitrary Apache-style log format strings. If Sawmill doesn't recognize your format as one of the built-in ones, it gives you the option to enter an Apache-style log format string (i.e. the value from the LogFormat directive in the configuration file), and sets up all the fields and views appropriately from that string. This means that is is no longer necessary to create custom log format plug-ins for every Apache variant-- you can just enter the format string and use the generic "Apache Custom" plug-in.
  • [by request] Added a new view, "sessions," which lists all the sessions in a single table. The table shows the visitor id, start time, end time, duration, and page views for each session.
  • [by request] Add support for HTTP log sources; Sawmill can now download single log files from HTTP servers.
  • [by request] Improved DNS lookup on Windows and UNIX-type operating systems (including Linux), so if the DNS servers are not specified, Sawmill will get them from the operating system. In practice, this means that you shouldn't need to specify them anymore, unless you want to use a DNS server other than the default. On other platforms (e.g. MacOS 9), you still need to specify the DNS servers manually.
  • [by request] Added an LDAP authentication script (written in perl) to the Extras folder. This provides an easy way to use LDAP authentication with Sawmill, on any platform with a command line (i.e. anything but MacOS 9). The default LDAP authentication script uses LDAP groups for access control, providing access to a configuration to any users which are members of the group whose name matches the configuration name.
  • [by request] Added support for SMTP hostnames in the format hostname:port, which overrides the default port of 25.
  • [by request] Added support for "view_info" usage in log format plug-ins. Previous versions would ignore view_info values, and overwrite them with the default views. Now, if view_info is specified, the defaults are not used. Also, a value of "(defaults)" is now accepted in visibility options for a subview; it causes the default visibility options to be used for that subview, which makes it possible to have a custom view_info section which adapts properly to the available options (i.e. hits, bandwidth, etc.).
  • [by request] Improved Squid log format to track peer host, peer status, and MIME type.
  • [by request] Added support for DansGuardian log format.
  • [by request] Improved the headers/footers so they are pumped through the standard language module processor before being included in the HTML. This means that it is not possible to use configuration variables, language module variables, and language module expressions in your header/footers. For example, you can now say "{CONFIG_NAME}" in the header or footer, and it will be replaced with the actual name of the configuration being displayed.
  • [by request] Added a new preference option which controls the color of administrative header and footer bars.
  • [by request] Added a new preference option which lets you hide the Sawmill logo in administrative headers.
  • [by request] Added support for Planet-Share InterFax log format.
  • [by request] Added support for SonicWall (syslogd) log format.
  • [by request] Added support for WinGate log format with Traffic lines.
  • [by request] Improved support for W3C format logs so that if both sc-bytes and filesize are defined, Sawmill will use sc-bytes for the bandwidth. This means that it will report true bandwidth for Windows Media files now, instead of reporting the bandwidth as though all files had been fully transferred.
  • [by request] Simplified and fixed MacOS X "run at startup" instructions and method.
  • [by request] Added support for Oracle Listener log format.
  • [by request] Added support for dd/MMM/yyyy date format (i.e. where month is all caps).
  • [by request] Improved the export function to export bandwidth in bytes, rather than using "k", "M", etc. to indicate kilobytes, Megabytes, etc. This makes it easier to manipulate imported bandwidth numbers.
  • [by request] Added a new option to the Scheduler, to let you process some of the configurations (those which match a particular pattern).
  • [by request] Ported Sawmill to Sun's compiler. This means that it is now possible to compile Sawmill on a 64-bit UltraSPARC system and have it run in full 64-bit mode. Sawmill runs somewhat slower in 64-bit mode, but it can handle a larger range of values, so certain extreme datasets (i.e. more than 2 billion lines of log data) will behave better in 64-bit mode. A 64-bit SPARC binary is not currently available for download, but if you contact sawmill@flowerfire.com, we can send you a custom-built version.

Version 6.2.16, shipped February 17, 2002

Bugs fixed in version 6.2.16:

  • [by request] Fixed a bug where Sawmill was caching the schedules in web server mode, so if you updated them manually or using CGI mode, they would not be updated until the web server restarted (and if you used the web server to edit Schedules, the changes would be overwritten). Sawmill now reloads schedules when it detects that they have been modified.
  • [by request] Fixed a bug where Sawmill's authentication cookies were incorrectly formatted, and were ignored by some web browsers (in particular, IE on MacOS X). This caused a situation where Sawmill would keep asking you to log in, and would never let you do anything.
  • [by request] Fixed a bug where if command-line authentication was being used, Sawmill would let anyone log in if they entered their username and the administrative password. Sawmill now allows only the administrator to log in with the administrative password.
  • [by request] Fixed a bug where if the "automatically update if older than" option was set, it would be ignored if the statistics were viewed from the Configuration Menu.
  • [by request] Fixed a bug where clicking Browse in the "browse config" page without entering a configuration name would take you to the administrative login prompt.
  • Fixed a bug where clicking View in the "user menu" without choosing a configuration name would take you to the administrative login prompt.
  • [by request] Fixed a bug where Sawmill could sometimes crash while performing long tasks, if a web-based progress indicator was being used.

New features in 6.2.16:

  • [by request] Improved detection and handling of partial final lines in log data. Sawmill now detects when the final line is "partial"-- i.e. when it does not have a carriage return or newline at the end. When it detects this situation, it ignores that line, and "rewinds" itself so that it will look at that line again on the next update. This allows it to gracefully handle situations where the log data is growing or being uploaded in non-entry blocks, where the last log entry may be truncated.
  • [by request] Improved decompression of gzip, ZIP, and bzip files so that if Sawmill sees an error in the compression format, it treats it the same as the end of the file, and does not report an error. This allows Sawmill to successfully process partial files, so it can handle decompression and processing of files which are being uploaded as it processes them. Data is not lost-- Sawmill picks up where it last saw good data on the next update, so eventually it will always get the whole file.
  • [by request] Improved the web server threading, so a fixed pool of threads are always available to take new connections. This is more efficient because it eliminates the thread creation/destruction overhead, and it is also more robust because it is no longer possible for Sawmill to consume all system threads in extreme cases.
  • [by request] Added support for Tomcat log format.
  • [by request] Added support for Domino Error log format.
  • [by request] Added support for TAI64N date/time format.
  • [by request] Added support for QMail log format with TAI64N datestamps.
  • [by request] Added an option to show only minimal operating system information in error messages, so full details are not available to hackers.
  • [by request] Improved authentication so that if cookies are disabled, Sawmill will work as it did in 6.2.14 and before-- by carrying the authentication information forward in form variables. This allows Sawmill to be used if cookies are disabled, but some new features (i.e. hiding of administrative links in the statistics) will be disabled if cookies are disabled.

Version 6.2.15, shipped February 10, 2002

Bugs fixed in version 6.2.15:

  • [by request] Fixed a bug where if extremely complex statistics filters were used, Sawmill could crash.
  • [by request] Fixed a bug in Gauntlet log format where it would ignore data from months 1-9 of the year.
  • [by request] Fixed a bug where Sawmill always tried to call the scheduler using the "default" executable name ("sawmill") instead of using the actual name of the executable. This could prevent scheduled tasks from executing.
  • [by request] Fixed a bug where the FTP server could time out while Sawmill was processing large compressed log files by FTP. Sawmill now sends NOOPs to the FTP server every 30 seconds, to make sure it doesn't hang up.
  • [by request] Fixed a bug where in log formats that use parsing filters, Sawmill was not correctly resetting the collected fields after processing the line, so field values from earlier lines could "bleed over" into later lines.
  • [by request] Fixed a security flaw in which the AdminPassword file was created with incorrect permissions (666 instead of 600), allowing it to be written to by any user on the system. This means that a clever user (who can figure out how to encode their own admin password) can become administrator. This can be fixed trivially by changing the permissions on the AdminPassword file to remove world and group access. This version will create the password correctly with those permissions.
  • [by request] Fixed a bug where with certain log formats (including Radius), updates would fail because the log format header was so long that two different log formats were identical for the first 500 characters, so Sawmill thought they were the same (previously-processed) log data. Sawmill now checksums the first 16k of data, instead of just the first 500 bytes, which should eliminate this problem for any format with less than 16k of header data.

New features in 6.2.15:

  • [by request] Improved the "week" links in the Calendar so they show the entire week, instead of just the days of the week that are in the month containing the link.
  • [by request] Changed the order of years in the Calendar view so the most recent years are at the top.
  • [by request] Added external authentication of usernames/passwords. Sawmill can now call an external program or script to authenticate users. When it operates in this mode, users log in with a username and password, and an external program decides which configurations they are permitted to use. The external program also logs in the administrator(s). This is a much more flexible authentication method than that previous one, and it allows Sawmill to integrate with external systems like LDAP, by using a custom script.
  • [by request] Improved the authentication system to track logins using cookies. Sawmill now stores authentication information in cookies, so it knows if you're the administrator when you're viewing statistics. It is no longer necessary to login again when leaving the statistics and reentering the administrative interface. Some administrative aspects of the statistics (including the configuration link and the administrative menu link) are now also automatically hidden when the viewing user is non-administrative.
  • [by request] Improved Common Access format to allow extra spaces at the end of the log format (WebLogic does this).

Version 6.2.14, shipped February 03, 2002

Bugs fixed in version 6.2.14:

  • [by request] Fixed a bug where images would sometimes not appear correctly in icon menus.
  • [by request] Fixed a bug where Sawmill was not properly extracting the "len" (bytes transferred) field from Firewall-1 logs and other similar logs.
  • [by request] Fixed a bug in "export table," where if the table data was so large that the export showed a progress page, then the final result would be shown with MIME type text/html rather than text/csv, so it would be displayed by the browser rather than downloaded.
  • [by request] Fixed a bug where if you used Quick Start to build a very large database, Sawmill would clean up the temporary files before the build was done, resulting in a (harmless) "can't write config file" error at the end of the build.
  • Fixed a bug where "export all" exported only the visible table rows.
  • [by request] Fixed a bug where Sawmill could use arbitrary amounts of memory while processing log formats which used the "collected entries" feature. Sawmill now manages its memory better, and these formats should require only a little more than a normal non-collected format.
  • [by request] Fixed a bug where if there was a duplicate database field (for instance, if you'd just duplicated it), some configuration operations could crash.

New features in 6.2.14:

  • [by request] Added an option to sort tables alphabetically.
  • [by request] Added average sessions per day to the "Sessions (summary)" view.
  • [by request] Added support for QMail log format.
  • [by request] Added support for NcFTP Xfer log format.
  • [by request] Added support for Winproxy Common log format.
  • [by request] Sorted the session filter menus alphabetically.
  • [by request] Added support for tracking of monitor resolution and depth for web logs, using JavaScript embedded in the HTML.

Version 6.2.13, shipped January 27, 2002

Bugs fixed in version 6.2.13:

  • [by request] Fixed a bug where DNS lookup did not work for log formats which used the "collected" log entry parsing features.
  • [by request] Improved the PIX log formats so they separated hostnames from ports and hostnames from sides better, so only the raw IPs are stored in the "source host" and "destination host" fields. This allows those fields to be used for DNS lookup.
  • Fixed a bug in session information where the "Total unique hosts" number did not take the session filters into consideration.
  • [by request] Fixed a bug where if there was an empty line in the log source field, and it was not the last line, then Sawmill could crash trying to process the log source.
  • Fixed a bug in Wipro WebSecure log formats which caused them to not work when used with GMT+ time zones.
  • Fixed a bug where some memory allocated by the "Generate HTML Files" operation would never be released.
  • [by request] Fixed a bug where Sawmill would not properly skip previously-seen files if the "skip processed files on update" option was checked, and the filenames started with digits.
  • [by request] Fixed a bug where Sawmill could crash if so many filters were applied that the URL was more than 1000 characters long.
  • [by request] Fixed a bug where Sawmill would create multiple duplicate cross-reference groups if there was a duplicate database field. A handful of older log format plug-ins had duplicate database fields, including the "file type" field. Over time, this could greatly decrease the performance of the database. More seriously, in some cases it could cause database corruption on update. Sawmill no longer creates duplicates, and it now removes duplicate database fields and duplicate cross-reference groups when it finds them, so configurations which have been corrupted in this way will be repaired the next time they are opened and the configuration options are viewed; also, even if that is not done, they will dynamically correct the problems whenever the database is used, so even configurations with duplicate fields and xref groups should work properly now.
  • [by request] Fixed a bug where Sawmill could get confused if a W3C-format log (including IIS logs) switched format drastically in the middle.

New features in 6.2.13:

  • [by request] Added support for Wipro WebSecure Auth logs.
  • [by request] Added support for WebSTAR W3C log format.
  • [by request] Added a new type of session filter, which lets you show only sessions from a particular visitor id.
  • [by request] Improved Raptor format; combined Raptor and Raptor Alternate into a single format, and added detection of certain types of statistics lines.
  • [by request] Added an "export table" option to export only the visible table data.
  • [by request] Improved tcpdump log format to track icmp packets.
  • [by request] Added support for Order log format.
  • [by request] Added support for Bind Query log format with timestamps.
  • [by request] Removed "Sawmill" from some of the names of Sawmill's support files. The SawmillInfo directory is now called LogAnalysisInfo, the SawmillRoot directory is now called WebServerRoot, SawmillDefaultConfig is now called DefaultConfig, and SawmillPassword is now called AdminPassword. This changes makes it possible to truly "white-label" Sawmill, eliminating any reference to Sawmill's name in the interface or support files. (The language modules also need to be modified to remove references before the entire interface is "clear").
  • Added a new log filtering feature: variables. Variables are temporary storage locations that can be used, like variables in programming languages, to store values. They can be used, for instance, to keep track of the number of lines processed, or to count the lines since a particular line, which is useful when processing some multi-line log formats.
  • [by request] Improved database expiration so that the newly-freed space in the database is collapsed, and the database is shrunk to the minimum size, after expiration is done. Previous versions freed space in the database tables, but did not actually shrink the tables.
  • [by request] Fixed a bug in iptraf text log format where it would reject hits on the 1st through the 9th of the month.
  • [by request] Added support for CacheFlow log format.
  • Added a default filter to categorize PDF files as hits, but not page views.

Version 6.2.12, shipped January 06, 2002

Bugs fixed in version 6.2.12:

  • [by request] Fixed a bug where if you deleted the Overview, you would get "unknown view 'Overview'" when you clicked bottom-level items in the statistics. Sawmill now uses the first view when it can't find the Overview.
  • [by request] Fixed a bug which could cause the Scheduler to crash on MacOS X (and possibly other platforms) when it tried to run a scheduled task involving "all configurations."
  • [by request] Fixed a bug where if the "Allow empty log source" option was checked, and the log source was empty, the built database would be unusable even though it didn't generate an error on build.
  • [by request] Fixed a bug where on Windows, some of the Sawmill's images would sometimes be broken.
  • [by request] Fixed a bug where Sawmill would generate "unknown view" errors while generating HTML files for a configuration which contained hidden views.
  • [by request] Fixed a bug where the "active statistics view" and "view to send by email" options would show up as empty popup menus in the default configuration. They now appear as fields, so the name of the default view can be typed. We suspect, though we're not certain, that this bug is also responsible for the corruption of these two options in the default configuration file and other configuration files, as reported by several users.
  • [by request] Fixed a bug where if there were multiple "singlefilter" items in the filters list, sometimes some of them would be ignored.
  • [by request] Fixed a bug where if the password in an FTP URL contained an @ character, the "(password)" replacement would leave part of the password visible.
  • [by request] Fixed a bug where if none of the log entries were accepted, Sawmill would crash or generate a garbage screen, instead of the nice "none of the entries were accepted" screen it was supposed to generate.
  • [by request] Fixed a bug where if the entry name has been customized, Sawmill would sometimes not sort tables correctly.
  • Fixed a bug where the FAQ icons appeared in the statistics even if the "show docs links in stats" option was turned off.
  • [by request] Fixed a bug where there were no instructions for the "Top server domains" view.
  • [by request] Fixed a bug where searching the documentation in CGI mode would generate a web browser error.

New features in 6.2.12:

  • [by request] Improved the performance of database expiration. Expiration of data from the database was slow due to several inefficient algorithms, which showed up particularly strongly when there was one or more very complex field hierarchies (e.g. a very deep page/directory hierarchy). These algorithms have been rewritten to be much faster. Expiration of all data from a particularly problematic database, which took 5 minutes before, takes 15 seconds now. Expiration of 25% of the data from the same database, which took 98 minutes before, takes 55 seconds now.
  • [by request] Added a "default log date year" option, which lets you choose what year Sawmill uses for log data that does not include year information.
  • [by request] Added automatic computation of xref groups, to simplify the creation of log format plug-ins. Sawmill now automatically creates xref groups so that 1) each field is cross-references to itself, 2) the date/time field is cross-referenced to everything else if "day-by-day statistics" is checked, and 3) the page and server response fields are cross-referenced if the 404s option is checked. This means that the cross_reference_groups option can be completely empty in most plug-ins.
  • [by request] Improved the default order of the views. Grouped the "Top days," "graphs of traffic over time," "top hours," and "top weekdays" views (both field-based and computed) together. (Incidentally, many log formats which were not tracking weekdays and hours were updated so they did). Grouped the "Top domains" and "Top domains/hosts" views together. (Incidentally, many log formats which were not tracking domain descriptions were updated so they did). Grouped the "Top referrers," "Top search engine," "Top search phrase," and "Top referrer domains" views together. Grouped the "Top web browsers," "Top operating systems," and "Top spiders" views together. Grouped the "sessions summary," "paths through the site," "entry pages," "exit pages," and "time spent per page" views together.
  • [by request] Improved the generation speed of the menu of configurations that appears when selecting Open Configuration, View Statistics, or other main menu options. In one test, the previous version took 40 seconds to generate the menu; this version does it in 9 seconds.
  • Improved TaskLog so it logs error messages, and so it logs result messages after operations complete (i.e. it logs the text that appears at the bottom of the configuration page).
  • [by request] Added support for Domino Agent log format.
  • [by request] Added support for bzipped log files-- Sawmill now automatically decompresses bzipped log data on the fly.
  • [by request] Added a new type of command-line filter: you can use "-f yesterday" to show only yesterday's data.
  • [by request] Added better support for certain PIX logs which have two date/time stamps per line. Previous versions would use the first one, which did not include year information. This version uses the second one, which does include year information, so year information is now available in the statistics for those logs.
  • [by request] Improved the Squid format plug-in so it tracks visitor information based on the source IP, rather than the source host; this allows accurate visitor numbers in situations where the source host is not available. Also, added support for page view tracking.
  • [by request] Improved the iMail Alternative format plug-in to track SMTP lines, in addition to SMTPD lines that it's always tracked.
  • [by request] Added an exceptions-only variant of the Raptor log format plug-in.
  • [by request] Added support for WinGate log format.
  • [by request] Added support for 3Com Office Connect / WinSyslog log format.
  • [by request] Added support for Vicomsoft Internet Gateway format.
  • [by request] Added support for BDS FTP log format.
  • [by request] Added a "hidden views URL" option which you can use to specify a URL that the buttons for hidden views should link to. The buttons are also dimmed out in this case. When the option is empty, the buttons are omitted, as before.
  • [by request] Switched the MacOS X installer over to use Apple's installer. This makes the installation procedure more in line with standard MacOS X installers, and also fixes occasional problem with permissions in the old installer.

Version 6.2.12a, shipped January 08, 2002

Bugs fixed in version 6.2.12a:

  • [by request] Fixed a bug which prevented Sawmill from reading gzipped log data correctly.

New features in 6.2.12a:

  • [by request] Added support for SonicWALL Log Format (Kiwi syslog).

Version 6.2.11, shipped December 11, 2001

Bugs fixed in version 6.2.11:

  • [by request] Fixed a bug in session tracking where it wouldn't working if the log data didn't have the necessary fields at the start of the file. Sawmill will now bide its time until the fields appear, and then start tracking session information.
  • [by request] Fixed a bug where Sawmill did not handle the response codes of some FTP servers correctly after it terminated a download, resulting in an error message when nothing was really wrong.
  • [by request] Fixed a bug where Sawmill did not properly read gzipped log data.
  • [by request] Fixed a bug where Sawmill would not update the database properly when previously-processed log files were in the log source, and were less than 100k.
  • [by request] Fixed a bug where Sawmill could get confused about the name of the "authenticated user" field, sometimes calling it "user," which caused problems when trying to report on it.

Version 6.2.10, shipped December 09, 2001

Bugs fixed in version 6.2.10:

  • Fixed a bug where if you moved a configuration from from Windows to UNIX, and it contained multiple log sources, the first few would stop working.
  • [by request] Fixed a bug where configuration options containing # characters (like passwords) would be truncated by Sawmill when the configuration file was read.
  • [by request] Fixed a bug where if some views were hidden (using the "show" checkbox in the Stats Views tab), then the sidebar would be out of sync with the statistics, so clicking a button wouldn't show you the right view.
  • [by request] Improved the Export feature so it downloads the CSV file, rather than displaying it.
  • [by request] Fixed a bug where under certain circumstances, under MacOS 9, Sawmill would give an error about "NsNu."
  • [by request] Fixed a bug where clicking "Browse" next to the log source field would show an empty directory listing if there were multiple log sources.
  • [by request] Fixed a bug where on Windows 95 or 98, Sawmill would open a console window when it created the server process.
  • [by request] Fixed a bug where the number of documentation search matches was not shown in some browsers.
  • [by request] Fixed a bug where documentation links were broken in offline statistics, if "Running Sawmill URL" was defined.
  • [by request] Fixed a bug where if the was a > or < in a filename, and the "go" icons (red arrows) were visible, then the generated view HTML would be messed up.
  • [by request] Fixed a bug where certain types of binary corruption of the log data, on Windows would cause Sawmill to stop reading the log data when it saw it.
  • [by request] Fixed a bug where day-by-day tracking did not work with Firewall-1 logs.
  • [by request] Fixed a bug where the destination field was not being tracked properly in Firewall-1 logs.
  • [by request] Fixed a bug where clicking "Scan and Summarize Log Source" would generate an error if the database hadn't been built yet.
  • [by request] Fixed a memory leak which cause cause Sawmill to use much more memory than it should, when processing certain log formats (specifically, those that used collect_log_fields_regexp parsing filters).
  • [by request] Fixed a bug where Sawmill would not recognize that it was in CGI mode when it was accessed by certain browsers, resulting in an internal server error.
  • [by request] Fixed a bug where when communicating with certain FTP servers, Sawmill would get confused and would give strange incorrect messages.
  • [by request] Fixed a bug in Zeus Extended log format where referrer and agent information was not being properly extracted

New features in 6.2.10:

  • [by request] Improved Gnatbox format to process PASS lines, and to handle some slight format variations.
  • [by request] Improved the configuration selection interface so it doesn't show *.in2 files, which are generated by some antivirus software.
  • [by request] Added support for IPMon format.
  • [by request] Added support for Wipro Websecure Audit format.
  • [by request] Added support for Wipro Websecure Debug format.
  • [by request] Added unique visitor tracking to Squid log format.
  • [by request] Added support for Combined Proxy format (used by URLsnarf and others).
  • [by request] Added support for RealServer Error Log Format.
  • [by request] Fixed a problem where certain types of W3C log corruption could confused Sawmill so much that it skipped whole sections of the log data.
  • [by request] Added support for ZIP files. Sawmill can now automatically extract and process the contents of ZIPped log files, on the fly.
  • [by request] Added support for d/mmm/yyyy dates.
  • [by request] Added support for dd/mmm/yy dates.
  • [by request] Added a new derived log field, database field, and view called "file type" which tracks the type of the accessed file (e.g. HTML, GIF).
  • [by request] Improved the Cisco PIX log format plug-in to handle logs where there are two date/time stamps in each line.
  • [by request] Added support for portsentry format.
  • [by request] Added support for ipchains format.
  • [by request] Added support for iptables format.
  • [by request] Improved command-line (and URL) statistics filters, so you can use multiple singlefilter sections in a filter string to apply single filters to multiple fields.
  • [by request] Improved the Generate HTML feature so it uses a progress page like other long processes. This makes it possible to cancel it, and also eliminates a problem where the progress web page could get very large.
  • [by request] Improved the tabs in the configuration interface so you can click anywhere on them (not just the text), if you're using a recent browser version.
  • [by request] Improved FTP so that if you entered a URL like ftp://myserver/var/logs/* and var/logs/* didn't exist, then Sawmill would try /var/logs/* rather than reporting an error.
  • [by request] Added support for Oracle Application Server log format.
  • [by request] Added support for Postfix log format.
  • [by request] Added support for Zone Alarm log format.
  • [by request] Improved "Check All" in the Filters so it unchecks (no filter).
  • [by request] Made the menu item labels in the Administrative and Configuration menus clickable.
  • [by request] Improved the date/time filter display algorithm so it always shows them in chronological order, and it always uses a dash where appropriate. Previous versions usually used a dash when appropriate, but not always, and the order of items was somewhat random.
  • [by request] Improved pluralization and capitalization to make it language-independent. Previous versions just appended an "s" to pluralize, and generally did not capitalize (except for a few specific words). This made the plurals incorrect in some cases in English, and was not general enough to work for other languages at all. The language modules now contain regular-expression-based rules which are used to capitalize or pluralize, so these can be translated when the rest of the language module was. Field names are now capitalized where appropriate (e.g. in table headings).
  • [by request] Improved the "edit filters" interface so newly checked filters are not lost when you expand/collapse.
  • [by request] Added support for MacOS X FTP log format.

Version 6.2.9, shipped November 19, 2001

Bugs fixed in version 6.2.9:

  • [by request] Fixed a bug where the date/time graph data could sometimes get squashed so it took up only part of the available horizontal space.
  • [by request] Fixed a bug where where Sawmill would complain if the log source was empty, even if "Allow empty log source" was checked.
  • [by request] Fixed a bug where the one-time and repeat visitor numbers were not being computed on MacOS.
  • [by request] Fixed a bug where hitting return in the Save As window would take you to the Administrative menu (or do something else unexpected).
  • [by request] Fixed a bug where Sawmill could crash if it read a configuration file, language module, database info file, or similar file containing a token more than 50,000 characters long.
  • [by request] Fixed a bug where the visitors pie chart was not appearing correctly (it was appearing as a bandwidth pie chart, or it was not appearing at all).
  • [by request] Fixed a bug where on MacOS X, if Sawmill was installed in a folder with space in its name, the Scheduler would not work.
  • [by request] Fixed a bug in the UnixSendmail plug-in which would cause some sendmail logs to have almost no information reported.
  • [by request] Fixed a bug where the Page frame command, and command-based log source, did not work on Windows unless you used Samwmill6CL.exe directly.
  • [by request] Fixed a bug where Sawmill did not allow the use of "()" in regular expressions on Solaris systems, causing problems in some log formats.
  • [by request] Fixed a bug where the session filters did not persist from click to click.
  • [by request] Fixed a bug where the server hostname was not appearing in bookmark URLs (Sawmill was using the server IP instead).
  • [by request] Changed the way Sawmill represents date/times internally. The previous method was very compact, but caused problems when used in certain time zones (some dates would not appear in the calendar, the Overview would show incorrect date ranges, etc.). Databases built with older versions will be automatically converted when they are used. For most uses, this will not significantly increase the size of the database, but if you have increased the depth of your date/time field significantly (to the minute or second level), it may increase the size of your date/time index by a factor of three or four.
  • [by request] Fixed a bug where filter fields were not being translated properly when used with non-English language modules.
  • [by request] Fixed a bug where Sawmill could crash on update if one of the log files was less than 10k during the first build, and more than 10k during the update.
  • [by request] Moved the Filters bar for the page above the View buttons. This fixes a problem where if the Filters were very long, and didn't get line-wrapped due to a bug in IE, then the statistics would be "centered" under them partially or complete off to the right of the window.
  • [by request] Fixed a bug where if there was a negative date/time offset, and there was corrupt entries in the log data, then Sawmill could report dates in 1970 or 1969.
  • [by request] Fixed a bug where if there were multiple bars in the same table, the headers for the later ones would get shifted to the left.
  • [by request] Fixed a bug where Sawmill would print "Master process no longer exists; terminating server" at the end of every command-line job on Windows, even if it wasn't a server job.
  • [by request] Fixed a bug where if DNS lookup was turned on, but there was no hostname field in the log data (or there was for part of the log data, but not for all of it), then Sawmill would generate an error message during log processing.
  • Fixed a bug where if a filter description contained HTML code (for instance, if a page name contained HTML tags, and it was used as a filter), then the statistics display would get formatted wrong.

New features in 6.2.9:

  • [by request] Added an option to control the color of the view sidebar in the statistics.
  • [by request] Added support for a variant of Apache Combined log format with a server domain added as the first field.
  • [by request] Added support for Apple File Service log format.
  • [by request] Improved the "choose password" page to use better names for the buttons.
  • [by request] Improved all Common Log Format (CLF) type log formats, to handle usernames with spaces. This has been a long-term problem with CLF formats, because CLF formats are space-delimited, but (foolishly) allow spaces in usernames without quoting them. This makes it difficult for Sawmill to parse these format correctly, because the space in the middle of the username looks like the divider between fields, so fields get offset. This is finally solved in this version by adding a new option, "Common log format." When this option is true, Sawmill allows spaces in the field before the date/time field (i.e. the username field, in common log format), and uses the opening bracket of the date/time field to determine where the end of the username field is, rather than using the normal delimiters.
  • [by request] Added support for mmm/dd/yyyy dates.
  • [by request] Added support for SiteKiosk log format.
  • [by request] Added a feature to automatically remove null characters (zero bytes) from the middle of the log data, when encountered. This allows Sawmill to handle some types of log data corruption better.
  • [by request] Changed bandwidth unit for bytes to "b" rather than not showing a unit at all. For instance, 100 bytes used to be shown as "100" and is now shown as "100b".
  • [by request] Added support for Gauntlet log format.
  • [by request] Added support for Radius ACT log format.
  • [by request] Added a "first day of week" option to change the first day of the week to Monday instead of the default Sunday (or to another day of the week).
  • [by request] Improved the formatting of statistics pages so the pie chart and table don't go off the screen even if the filters header is really wide, and so the Show/Sort menus stay centered below the table, rather than being centered on the whole page.
  • [by request] Added detection of the Overture search engine.
  • [by request] Added a write-locking mechanism to the databases, so it is no longer possible to have two database builds/rebuilds/updates going at the same time. One of them will wait for the other to complete before starting. This fixes a class of problems which could occur if, for one reason or another, two or more builds/updates of the same database occurred at the same time. That would almost always corrupt the database; it no longer will.

Version 6.2.8, shipped October 24, 2001

Bugs fixed in version 6.2.8:

  • [by request] Fixed a bug in the 6.1 database converter, which could cause internal errors while converting the database.
  • [by request] Fixed a bug where tabs in filters were not being properly quoted in configuration files, possibly resulting in configuration file corruption.
  • [by request] Fixed a bug where Sawmill could crash if you had two convert_field_regexp filters in the same filter set.
  • [by request] Fixed a bug where using the Back button in the Sawmill page while creating a configuration could cause an error, "Can't write SawmillInfo/Databases/xxxx/config."
  • Fixed a bug where Sawmill was showing an incorrect list of fields that needed to be cross-referenced when it showed a Data Not Available error for the "search terms (computed)" view.
  • Fixed a bug where using the Back button in the Sawmill page while creating a configuration would lose the name of the configuration you typed.
  • [by request] Fixed a bug where Total cell of the "average time spent per page" column actually showed the average of the averages. This has been moved to the Average cell of the column, and Total is now "-".
  • Fixed a bug where, when processing logs with mm/dd/yyyy format dates, Sawmill could crash if it encountered a log entry with a date outside of the 01-12 range.
  • [by request] Fixed a bug where Sawmill could generate the cross-reference groups incorrectly, resulting in an extra group with a single field in it even when that field was already part of another xref group. This could have dire consequences when attempting to expire the database.
  • [by request] Fixed a bug where choosing an Options menu item, or a Sort menu item, or a link at the bottom of a table, while viewing a view with a view- or subview-level filter (e.g. the broken links view) would change the main filter to the view's filter.
  • [by request] Fixed a bug where clicking the configuration name at the bottom of the statistics would ask for the password, then the configuration name, and then the password again before finally letting you see the configuration menu.
  • [by request] Fixed a bug where if there were multiple date/time graphs in the same view, they would all look like the last one.
  • [by request] Fixed a bug in the Zeus Extended Log Format which caused it to not recognize files in that format.
  • [by request] Fixed a bug where Sawmill would not host-stamp its temporary configuration files correctly if the browser provided REMOTE_ADDR information but not REMOTE_HOST information. This could cause temporary configurations to collide if the same configuration was being browsed from several machines at the same time. Sawmill now looks for REMOTE_ADDR if it can't find REMOTE_HOST.
  • [by request] Fixed a bug where if there were multiple pie charts of the same type in the same view, they would all look like the last one.
  • [by request] Fixed a bug where the "weekdays (C)" view had an incorrect title for the first column of the table.
  • [by request] Fixed a bug where if Sawmill did not have permission to write to SawmillInfo/IPC (e.g. if it was being run as a different user now than when it was installed), it would crash.
  • [by request] Fixed a bug where if you created many configurations, and one of the new configuration names had the old configuration name in its name, Sawmill would consume all available memory.
  • Fixed a bug where if you changed a subview type in the Stats Views tab of the Configuration Options, the Options and Sort menu contents wouldn't change until the next reload.
  • [by request] Fixed a bug where if you ran Sawmill in web server mode, and then deleted a configuration with a database so large that Sawmill displayed a progress page during the deletion, then the progress page would never go away-- it would just keep loading forever, even after the database deletion was done. It now correctly shows the Administrative menu at the end of the deletion.
  • Fixed a bug where in some circumstances, the "by hits" sort order wouldn't work in the Stats Views tab, and subviews marked that way would be unsorted.
  • [by request] Fixed a bug in the Windows version which could cause the GUI window to stall for a very long time (sometimes minutes) before finally displaying the server URL and Use Sawmill button.

New features in 6.2.8:

  • [by request] Improved Microsoft Proxy format plug-in to handle files where the IP address is missing.
  • Changed the way scheduled tasks are done on all platforms except MacOS 9. Scheduled tasks are now called as separate command-line invocations of Sawmill. Previous versions (and the current MacOS 9 version) ran the tasks as separate threads inside the main web server process. This new method prevents any possible memory conflicts between tasks, and also allows for better error recovers in the event that a task thread terminates unexpectedly. This eliminates a problem that could occur in 6.2.6, where if a task crashed, it would crash the main server, which would get restarted, and immediately try to run the task again, crash again, etc.... Now the scheduling thread is relatively immune to the woes of the scheduled tasks, so a scheduled task should not be attempted more than once.
  • [by request] Added a new chapter to the documentation, describing how to set up a multi-user environment.
  • [by request] Improved the Browse button for the log source field, so it doesn't overwrite the existing field; it appends a new log source to the end of it. This makes it possible to Browse several times in a row to pick up all the files/folders you need.
  • [by request] Improved the "generate html files" feature so it deletes all existing files in the destination folder, before generating new ones.
  • [by request] Improved the browser detector so it can identify Java, and so it reports "unspecified" when the browser isn't specified, rather than "unspecified/unknown", and so it does not report the version number at all if it's unknown, and so it parenthesizes (and therefore omits from the statistics) any items which are completely unspecified and/or unknown.
  • [by request] Added support for iPlanet Error Log Format.
  • [by request] Improved support for Cisco IOS format, to recognize and process logs where the router is identified by hostname rather than IP.
  • [by request] Added a new date format, dd/mm/yyyy.
  • [by request] Added support for a ProxyPlus log format.
  • [by request] Added support for Bulletproof FTP log format.
  • [by request] Added support for Tiny Personal Firewall Log Format.
  • Added a new option, "Allow empty log source" which suppresses the usual error that occurs if Sawmill tries to build or rebuild a database and finds no data in the log source. When this option is turned on, Sawmill will build a completely empty database without complaint. This is useful for multi-user environments where some of the logs may be empty; it allows all database to be built error-free even if some of them are empty.
  • Improved the appearance of empty subviews, so they show a "no data" note instead of table headers without any content.
  • Improved the GNATBox log format plug-in to support "NAT: WWW" log entries.
  • [by request] Improved the documentation window so it jumps to the front whenever documentation is displayed. We tried to do this a long time ago, but the solution at that point had bad side effects, include freezes on MacOS 9, and we had to disable it. The new solution is much simpler, and hopefully will work on all platforms.
  • [by request] Added support for Netscape Messenger log format.

Version 6.2.8a, shipped October 26, 2001

Bugs fixed in version 6.2.8a:

Version 6.2.7, shipped October 14, 2001

Bugs fixed in version 6.2.7:

  • [by request] Fixed a bug in the Windows version of Sawmill running in web server mode, where once a database had been used (built, updates, viewed, etc.), attempts to erase or rebuild the database would fail with a "can't delete file flock" error.
  • [by request] Fixed two bugs in chronological sort for day of year and week of year; the sort options were not appearing in the menu, and even if you modified the configuration file to put them there, they didn't sort correctly.
  • [by request] Fixed a bug where the configuration file could become corrupted if it contained a subview with no options visible. This could occur because of a second bug, where creating a new subview when no subviews existed would generate a subview with semi-random values. Subviews with no visible options are no longer a problem, and creating a new subview creates a reasonable default subview.
  • [by request] Fixed a bug with the MacOS X version where an exception window would appear if you clicked Use Sawmill, and then quit the web browser.
  • [by request] Fixed a bug where the link to the configuration menu from the statistics did not work properly if the configuration name contained a space, and Sawmill was running in CGI mode under IIS.
  • [by request] Fixed a bug where the image was broken in the "Send view by email" response page.
  • [by request] Fixed a bug where a view's button was still shown in the statistics even if all subviews were hidden. Sawmill now hides the button if all subviews are hidden.
  • [by request] Fixed a bug in the 6.0/6.1 database converter which affected the MacOS version of Sawmill. The database index could get corrupted during the conversion, resulting in "can't read N bytes of data" errors when you tried to view the converted database.
  • [by request] Fixed a bug where Sawmill was miscomputing visitor information in some situations when the database was updated.
  • [by request] Fixed a bug where if you created a configuration with the same name as an old configuration, the database would not be erased properly.
  • [by request] Fixed a bug where Sawmill could crash in rare situations if multi-part filters were used.
  • [by request] Fixed a bug where if you clicked the "Back to Administrative Menu" button on the Delete Configuration page, it would delete the selected configuration.
  • Improved Sawmill's behavior on Windows in the case where the GUI version is used, but the command-line version (which it requires) cannot be found. Previously, Sawmill would freeze in this situation, and would have to be terminated; now, it reports an error and can be quit normally.

New features in 6.2.7:

  • [by request] Added support for a new variant of GNATBox log format.
  • [by request] Improved the command-line filtering interface to allow "-f recentdays:30" even if the date/time field isn't the first field.

Version 6.2.6, shipped October 08, 2001

Bugs fixed in version 6.2.6:

  • [by request] Fixed a bug where on Windows, Sawmill was running each scheduled item twice, because both the graphical Sawmill (Sawmill6.exe) and the "behind-the-scenes" Sawmill (Sawmill6CGI.exe) thought they were responsible for scheduling.
  • [by request] Fixed a bug where Sawmill could become confused if you used several FTP log sources. This bug would sometimes manifest itself as the error message "Invalid Passive Response."
  • [by request] Fixed a bug where using quotes (") in filter values could cause the configuration file to become corrupted to the point that it could no longer be opened.
  • [by request] Fixed a bug in the "Apache Combined Format With Visitor Last" format plug-in which prevented it from properly counting visitors.
  • [by request] Fixed a bug where the -scheduler command-line option did not work.
  • [by request] Fixed a bug where Sawmill could crash while processing log data if the log format changed in the middle.
  • [by request] Fixed a bug where on platforms where Sawmill did not have multithreading capabilities, long operations involving progress lines would cause browser errors.
  • [by request] Fixed a bug where Sawmill was not processing iMail dates correctly, resulting in empty statistics.
  • [by request] Fixed a bug where on platforms where Sawmill did not have multithreading capabilities, when using Netscape, the animated logos were broken on progress pages. The animated logos cannot be displayed under those circumstances, so they are now suppressed in this case (rather than being broken images).
  • [by request] Fixed a bug which could cause sporadic crashes when generating a database on Solaris (and possibly other operating systems), and watching the progress in a web browser.
  • [by request] Fixed a bug where the "Create Many Configuration" option did not correctly convert filter fields.
  • [by request] Fixed a bug where using Save As would erase the original database, if the database directory was explicitly specified (rather than blank, as is the default).

New features in 6.2.6:

  • [by request] (Note: this feature was actually available in version 6.2.5X, which shipped only for MacOS X). Added graphical interface to native MacOS X version of Sawmill. There has been a "Carbon" version of Sawmill for some time now, and separate native MacOS X command line version. Now, there is a native graphical interface version (a "Mach-O" version) for MacOS X. This version works similarly to the Windows version-- the graphical interface is a very simple single-window interface that runs the command-line version behind the scenes. This means that it is no longer necessary to use the Terminal program to run the native version of Sawmill-- you can run it by double-clicking the icon. The Carbon version has been retired, because it is slower and less stable than the native version. The "Classic" version will continue to be supported indefinitely, for those who want to run Sawmill on MacOS 9 or earlier.
  • [by request] Improved the way bottom-level items (for instance, days in the "Top days" view or pages in the "Top pages" view) are linked. In previous versions of Sawmill, they were not linked at all, because clicking generally means "zoom in" and there was nothing to zoom in to. Now, they are linked, and they zoom in, but they automatically switch to the Overview, rather than switching to a useless page in the same view, or a "Data Not Available" view. This makes it easier to apply limits by simply clicking the appropriate item in the appropriate view.
  • [by request] Added support for Apache Combined log format with WebTrends visitor cookies embedded.
  • [by request] Added support for yet another variant of Cisco PIX log format.
  • Improved Sawmill's behavior on all platforms in the case where the "master" process (the one with the window, on Windows or MacOS X, or the one you started from the command line on UNIX or command-line mode) terminates unexpected or is manually terminated. In previous versions, the "subordinate" process would happily continue to run forever in this situation, causing problems if you tried to start another version later. Now, the subordinate process checks regularly to see if the master process still exists, and if it doesn't, it exists.
  • Changed the name of the command-line executable on Windows from Sawmill6CGI.exe to Sawmill6CL.exe, to better emphasize that it is a command-line program, not just a CGI program. It will still work as a CGI program; only the name has changed.
  • Improved the way Sawmill6.exe works on Windows. Sawmill6.exe used to be a fully-functional version of Sawmill; it has been gradually offloading the work to Sawmill6CL.exe. This caused problems because there was some confusion about which copy of the program did certain tasks (like scheduling; see elsewhere in this document). This version completes the process-- Sawmill6.exe is now a very simple single-window program that does nothing but run Sawmill6CL.exe (formerly Sawmill6CGI.exe), and restart it when dies. It also displays version information, the server URL, and any errors generated by Sawmill6CL.exe. All the real Sawmill work is done by Sawmill6CL.exe. The error window is improved, in that it now displays all errors in a single window, instead of opening another window for each error. As a side effect of this, the distribution is now smaller, because there is only one copy of Sawmill included, instead of two.
  • [by request] Improved detection and rejection of log entries with invalid dates.
  • [by request] Improved the field/filter/etc editor so duplicating an item expands the duplicate.
  • [by request] Improved support for Firewall-1 log format. Sawmill now processes the format line at the top of the log file, and uses that to determing the location and type of each log field.
  • [by request] Improved the "generate html files" option so it always generates all views for any filter set. This eliminates the problem where most of the buttons down the left would not be clickable in offline statistics, in some cases. The default maximum depth offline has been changed to 0 for all views except the Calender (where it's 1), to reduce the default size of the offline statistics.
  • [by request] Improved support for WebSTAR log format, so it tracks the SEARCH_ARGS and PATH_ARGS options.

Version 6.2.5, shipped September 21, 2001

Bugs fixed in version 6.2.5:

  • [by request] Fixed a bug where duplicating a log filter would cause Sawmill to crash. Because of the central role of log filtering in most uses of Sawmill we decided that this was too important a bug to leave in the shipping version, so we shipped this new version immediately.

Version 6.2.4, shipped September 21, 2001

Bugs fixed in version 6.2.4:

  • [by request] Fixed a bug which could cause Sawmill to crash in various circumstances, and would cause semi random text-mutation errors in other circumstances, especially while using the web interface. In its most embarrassing variant, this bug could cause some versions of Sawmill (including the Linux version) to crash infallibly and immediately when run. Yes, we really do test this before we ship it! But we test it with the executable named "sawmill" -- changing the name to "sawmill6.2.3" was enough to bring out this bug that slipped by the whole test suite. Therefore, you can work around this particular aspect of the bug by changing the name of the program to "sawmill", but the bug causes other problems as well, so we strongly recommend you upgrade if you're running 6.2.3.
  • [by request] Fixed a bug where clicking "Back to Administrative Menu" in the Open Configuration page would open the selected configuration.
  • Fixed a bug where Sawmill could crash when using a "rewrite rules" filter.
  • [by request] Fixed a bug where views sent by email from the web interface would include "temporary" links which would eventually expire, so you would get a "session timed out" error when you clicked them.
  • [by request] Fixed a bug which could cause a server error when running Sawmill in CGI mode and attempting to send a large view by email from the CGI interface (the view got sent anyway, so this was just a cosmetic problem).

New features in 6.2.4:

  • [by request] Slightly improved the action emails to be sent after the action completes, rather than just as it starts.
  • Brought back Alpha/Tru64 version of Sawmill. This version was unavailable since Sawmill 6.2 because of hardware problems on our Tru64 build machine. It's back up and running again, so the Tru64 version is available again.

Version 6.2.3, shipped September 18, 2001

Bugs fixed in version 6.2.3:

  • Fixed a bug where some of the FAQ entries were missing from the documentation.
  • [by request] Fixed a bug where scheduled items could possibly run repeatedly.
  • [by request] Fixed a bug where the "Direct URL to View Statistics" listed in the Database Information screen would sometimes use the wrong hostname.
  • Fixed a bug where the date graph equation text was hard-wired into Sawmill, rather than being in the language module, so it could not be translated.
  • Fixed a bug where the date graph equation text says "hits per default" instead of "hits per day."
  • [by request] Fixed a bug where the "(remove)" link was missing from the filters bar of offline pages.
  • [by request] Fixed a bug where the "export" option did not export all the columns (or left them blank) in some views, including the "search terms" view.
  • [by request] Fixed a bug where if you deleted a view, the corresponding subview in any multi-subview view (e.g. Single-page summary) would have blank values in its left column.
  • Improved the database converter to handle databases from certain v6 versions which generated incorrect date/times in the databases. The converter now corrects these as it sees them. One symptom of this was that converted databases would have no clickable links in the calendar, even though there was data in the database.
  • [by request] Fixed a bug where schedule monthdays were off by one day, so an item scheduled to run on the 11th of the month would actually run on the 10th.

New features in 6.2.3:

  • Added a new log file to SawmillInfo, called TaskLog. This log is written every time a task is performance, for instance every time a database is built, or the statistics are viewed, or a view is sent by email. The log shows when the task began, when it ended, and the IP of the user running the task, if any. It shows when scheduled items are are started. Finally, it shows any error that occur, together with basic information about the error (filename and line of error location in source code).
  • [by request] Restored the graph-over-time in the cases where there is only one data point. This was removed because it's not very meaningful, but it was pointed out that it is *useful* because the single bar is clickable, and that's often the easiest way to zoom in when you're in the Single-page-summary view.
  • [by request] Added progress pages to the "send view by email" operation, so the browser won't time out even if it takes a very long time.
  • [by request] Improved the performance of one of Sawmill's internal string manipulation algorithms. This fixes a performance problem where Sawmill could take a very long time to send large views by email. This may also improve performance of other aspects of Sawmill slightly.
  • [by request] Changed the directory structure of the "generate html files" result folder. Sawmill used to create oodles of directories to keep track of the filters; now it puts all the HTML files and graphs in a single directory. The point of this is to improve performance, decrease disk usage, and work around limitations of some operating systems, which apparently don't let you have too many directories on a single volume. This also has the added benefit that complex filters can now appear in offline pages, which means that the "week" links in the Calendar view are once again visible offline.
  • [by request] Hid the password field contents for the "Password to browse statistics" command.
  • [by request] Added the "Active Statistics View" option back to the Stats Misc tab, so you can set which view to start with.

Version 6.2.2, shipped September 09, 2001

Bugs fixed in version 6.2.2:

  • [by request] Fixed a bug where Sawmill could crash if it encountered a corrupt date/time value in a log file.
  • Fixed a bug where if the log source was an FTP URL with a filename (rather than a pattern), "Scan For Matches" would show that filename whether the file existed or not.
  • [by request] Fixed a bug where the DNS progress information was incorrect, and did not add up.
  • [by request] Fixed a bug on Windows where Sawmill would restart the server if the administrator killed it.
  • Fixed a bug where Sawmill would not return a valid page when the server was killed, resulting in a web browser error page.
  • [by request] Fixed a bug where memory and threads were not being cleaned up properly after a scheduled task ran. This had two serious consequences. First, every scheduled task would eat a chunk of memory that was never freed, so memory usage would increase gradually until Sawmill crashed. Secondly, each scheduled task would count as a "running thread" even after it exited, so the maximum threads would eventually be exceeded, causing "Sawmill is too busy" errors.
  • [by request] Fixed a bug where the progress page for the Consolidate stage of database builds did not always show the correct percentage.
  • [by request] Fixed a bug where FTP URLs did not work properly if the username or password contained a @ character.
  • [by request] Fixed a bug where FTP URLs did not work properly if the username or password contained a / character.
  • [by request] Fixed a bug where non-English language modules are not being properly loaded.
  • Fixed a bug where database expiration did not affect the session data. Expiration now expires the session information, too.
  • [by request] Fixed a bug where Sawmill would not handle log source URLs correctly if they contained a / as part of the username.
  • [by request] Fixed a bug where clicking a huge number of checkboxes in the Edit Filters page (for instance, expand all then check all) could cause Sawmill to crash when you submitted the page, and would use more memory than necessary even if it didn't crash. This bug was a bug in Sawmill's general string-processing library, so it could theoretically have caused many other crashes in various circumstances.
  • [by request] Fixed a bug where session information in memory was not being properly cleaned up in some cases. This could cause problems when generating HTML files, because the session information would eventually grow so large that every new session view would exceed the session information limit, and would fail to generate.
  • Fixed a bug where Sawmill could crash while computing session information.
  • [by request] Fixed a bug where there was no way to change the three-letter month names in the language module (they're now in the Stats module as MONTH_xxx).
  • [by request] Fixed a bug where Sawmill did not auto-detect some GNATBox log files correctly.
  • Fixed a bug where the final statistics page would sometimes not show up with using Netscape-- Sawmill would just keep reloading the progress page forever.
  • Fixed a bug where Sawmill did not track bandwidth properly for some GNATBox logs.

New features in 6.2.2:

  • [by request] The Sawmill web site now includes a support forum (threaded message board) for asking questions and getting answers from the Sawmill support staff and other users. The support forum is available at http://www.flowerfire.com/cgi-bin/dcforum/dcboard.cgi , in the SUPPORT section of the flowerfire.com web site.
  • [by request] Added a link to the administrative menu from every interview page.
  • [by request] Added support for a new variant of Zeus log format.
  • [by request] Added support for date/times in the format yymmdd-hhmmss.
  • [by request] Added support for times in the format hhmmss.
  • [by request] Added support for another variant of iMail log format.
  • [by request] Added support for G6FTP log format.
  • [by request] Added a link to the Administrative menu from the documentation.
  • [by request] Added an option which attempts to limit the amount of CPU Sawmill uses during long operations. The option (max_cpu_usage_percent) is a percent between 1 and 100, and specified the amount of CPU time Sawmill may use. Sawmill pauses at intervals during processing to ensure that it uses no more than that amount, on average. This option is useful in multi-user environments where processes are not permitted to use more than a certain percent of the CPU.
  • [by request] Improved the way Sawmill shuts down the subordinate server under Windows. Normally, Sawmill6.exe starts Sawmill6CGI.exe, and Sawmill6CGI.exe does the actual web serving. Sawmill6.exe restarts Sawmill6CGI.exe if it crashes. In 6.2.1 and before, Sawmill6.exe forcibly terminated Sawmill6CGI.exe when the user closed the window, which could cause problems and sometimes didn't terminate Sawmill6CGI.exe at all. In this version, Sawmill6.exe just sends Sawmill6CGI.exe a request to terminate, and Sawmill6CGI.exe terminates when it's good and ready.
  • [by request] Added a 6.0/6.1 database converter. Databases can be converted either from the command line or from the graphical interface. In both cases, the older database must be named "{configname}-6.1" where {configname} is the name of the configuration, and it must be placed in the Databases folder in SawmillInfo before the conversion can begin. The converted database will be created in the Databases folder, and its name will be the same as the configuration name. . Since that's how databases are normally named, that means you can start using the database immediately after it is converted. Conversion does not affect the original database-- an entirely new database is created. Since this is a fairly large feature which has not been passed through our normal beta test process, this should be considered a "beta" feature, with a fairly high risk of bugginess. Therefore, you should BACK UP up your database before attempting a conversion.

Version 6.2.1, shipped September 01, 2001

Bugs fixed in version 6.2.1:

  • [by request] Fixed a bug where Sawmill was converting old-style FTP log source information to new URL-style information incorrectly (it wasn't adding a leading / before the pathname of the URL).
  • Fixed a bug where the "Pattern is a regular expression" option and the "Process subfolders" option were not shown in the Log Source tab.
  • Fixed a bug where Sawmill was not skipping previously-seen data when multiple log sources were used.
  • [by request] Fixed a bug where the label on the "Hits %" column was "{ENTRIES_LABEL_CAPITALIZED} (%)".
  • Fixed a bug where if there were multiple log sources, one of them of unknown size, Sawmill would still show a progress meter, even though it didn't know how long the operation would take.
  • [by request] Fixed a bug where Sawmill could crash during DNS lookup if both the primary and secondary DNS servers refused a connection.
  • [by request] Fixed a bug where Sawmill would generate an internal error during statistics browsing in some cases.
  • [by request] Fixed a bug where Sawmill was not tracking URLs correctly in some Cisco PIX log formats.
  • [by request] Fixed a bug where Sawmill did not work in CGI mode with certain web servers, including Xitami.
  • [by request] Fixed a bug where if you tracked broken links for a file which contained no 404s, Sawmill would generate errors when you tried to view the broken links view.
  • [by request] Fixed a bug where passwords were not being correctly saved in some cases when using FTP log sources.
  • [by request] Fixed a bug where entering an FTP URL with a username but no password would cause Sawmill to crash.
  • [by request] Removed "visitor id" from the xrefs view, since it can't be xreffed anymore.
  • [by request] Fixed a bug where if you entered an invalid view number in the Sawmill URL, Sawmill could crash.
  • [by request] Fixed a bug where the "Back to the Administrative Menu" button would not appear in the View Statistics configuration list, if the password was saved as a cookie.
  • [by request] Fixed a bug where the "Back to the Administrative Menu" button did not work in the View Statistics configuration list-- it acted like you had clicked "View."

New features in 6.2.1:

  • [by request] Changed the default view name from "Top items for the (field) field" to "Top (field)s." This makes the view names more compact and readable, but will not work in all cases, for instance fields whose names end with "s." Still, those can be customized in the language module or the configuration, so this is overall a better solution than the previous.
  • [by request] Added a new variant of WinProxy log format.
  • [by request] Improved web browser detection so that all spider hits are categorized as "(spider)" in the web browser field. This makes the web browser statistics more accurate.
  • [by request] Added a new variant of Cisco PIX / Kiwi log format.
  • [by request] Improved Apache Combined formats to allow spaces at the end of the log line.
  • [by request] Added a new log format: Cisco 827 log format.
  • [by request] Improved the "too busy" response so Sawmill retries the connection for 20 seconds before giving up (previously, it would immediately return an error if it was too busy at the time of the connection).

Version 6.2, shipped August 22, 2001

Bugs fixed in version 6.2:

  • Fixed a bug where on Solaris, sawmill was auto-restarting if you killed it with control-C.
  • [by request] Fixed a bug where bookmarks did not save the filters.
  • Added some more checks to the DNS server response processing code, to try to eliminate any remaining possibility of Sawmill crashing due to an invalid DNS response. There are still reports that DNS causes Sawmill to crash in some cases; if so, this may fix it.
  • [by request] Fixed a bug where Sawmill could hang or crash while computing session information. This only seemed to occur under Windows in CGI mode, and only for certain views, but theoretically it could have happened in other situations as well.
  • [by request] Fixed a bug where the web_server option could be set to true in a configuration file, causing Sawmill to try to start a server whenever the configuration was used.
  • [by request] Fixed a bug in IIS 4/5 log format description file, where authenticated users were not being tracked.
  • [by request] Fixed a bug where the number of "single" and "repeat" visitors were reversed in the Sessions summary report.
  • [by request] Fixed a bug where DNS lookup could crash if it got a particular type of garbled response from the DNS server.
  • [by request] Fixed a bug where DNS lookup could get in an infinite loop (freeze forever) if it got a particular type of garbled response from the DNS server.
  • Fixed a bug in Common Log Format (and similar formats) autodetection, where Sawmill would not recognize a log entry's format correctly if the size field was "-".
  • [by request] Fixed a bug where offline views were showing even if "show views offline" was turned off for that view.
  • [by request] Fixed a bug where the "show view title" option did not work.
  • [by request] Fixed a bug where one category of DNS lookups from cache was not being counted, with the result that the total attempted lookups was not equal to the sum of the succeeded from cache, succeeded from network, timed out, and failed lookups.
  • [by request] Fixed a bug where if the filters filtered out all the sessions information, then there could be a divide-by-zero error, or strange results, in the session summary.
  • [by request] Fixed a bug where if you had auto-login enabled, or you were using the loopback IP (127.0.0.1), and you clicked the configuration name at the bottom of the statistics page, you would get a configuration page which just reloaded itself every time you clicked something, rather than doing the correct action for what you clicked.
  • [by request] Enhanced the ddmmmyyyy date format to also handle dmmmyyyy dates. This fixes a bug in the support of Firewall-1 logs, where the days 1-9 of the month were rejected from the log data.
  • [by request] Fixed a bug where log field names were not being translated properly when they appeared in log format description files, resulting in strange error message and crashes.
  • [by request] Fixed a bug where Sawmill would generate an error message if there were trailing carriage returns at the end of the "create many configurations" field.
  • [by request] Fixed a bug where the progress bar did not show up properly if the log data was more than 2 Gig.
  • [by request] Fixed a bug where Sawmill could crash while processing certain log files.
  • [by request] Fixed a bug where Sawmill could crash if, after a database build was complete, you clicked the browser's Back button to go to a progress page where you had shown or removed the Details.
  • [by request] Fixed a bug where use of the page_frame_command option could cause Sawmill to crash, and/or to display spurious characters above the statistics. The Samples on the Sawmill web site showed both symptoms.
  • [by request] Fixed a bug where the "Visible" and "Visible Offline" checkboxes for a subview had no effect.
  • [by request] Fixed a bug where Sawmill would sometimes hang or get incorrect data on FTP downloads, especially on Windows.
  • [by request] Fixed a bug where if there were no page views accepted in the log data, then trying to access session-related views would generate an error.
  • [by request] Fixed a bug in Firewall-1 support where it did not process some logs' date/time fields correctly.
  • [by request] Fixed a bug where if IIS logs contained #Date lines, but with multiple days between them, Sawmill would put all the hits only on those days explicitly mentioned in the #Date lines.
  • [by request] Fixed a bug where some random fields appeared at the top of the Database Info screen, and clicking Back To Configuration Page from there did not work properly.
  • Fixed a bug where empty fields would sometimes be counted as hits on the "root" item of that field, rather than as hits on "(empty)".
  • [by request] Fixed a bug where if you entered a "view statistics password" and then clicked Next, you would have to enter it again the next time you clicked something (but if you entered the password and hit return, you would not).

New features in 6.2:

  • [by request] Added a secondary DNS server option; Sawmill falls back on it if the first one refuses the connection.
  • [by request] Added tracking of the "server domain" field for Apache Combined format logs.
  • [by request] Improved error reporting for DNS. Sawmill no longer halts the build with an error if the DNS servers stop responding part way through the build; the lookups just fail until the servers start responding again. It will generate an error, however, if the very first request does not get a response; it assumes that the DNS servers are set up incorrectly in that case.
  • [by request] Added an option to export CVS data from the command line, using "sawmill -rfcf configname -cm evd -asv viewname".
  • [by request] Added auto-restart of Sawmill on Windows. Sawmill now automatically restarts itself if it crashes on Windows. This feature has existed for a while on UNIX versions (including MacOS X), and is finally now available on Windows. This is implemented internally by Sawmill starting another copy of itself, using Sawmill6CGI.exe. This has the side effect that there is no communication between the Sawmill window and the server anymore so reports of threads active, threads processed, and other "Advanced Options" are no longer available in the Sawmill window. If you miss these, email sawmill@flowerfire.com and we'll add them to the Web Server Administration page.
  • [by request] Added a "passwords" field to the log source screen. This field, which is "protected" in the sense that it shows asterisks or bullets instead of what is actually typed there, can be used to enter the FTP passwords separately from the plainly-visible text in the Log source field. Sawmill automatically substitutes passwords from the password field for occurrences of "(password)" in the log source field when the form is submitted, and then extracts them again when the page is shown. Multiple passwords can be entered by separating them with commas.
  • [by request] Implemented "rewrite rules" along the lines of those used by Apache. These are rules contained in a text file which Sawmill uses (through the Log Filters interface) to convert field values. The rules are simple now (just regular expression replacement), but future versions will implement more of the Apache mod_rewrite mechanism, as it is requested.
  • [by request] Added a new variant of Kiwi/PIX format.
  • [by request] Added support for multiple log sources; just enter one per line in the log source field, and Sawmill processes them sequentially.
  • [by request] Added a checkbox to the configuration creation interview which lets you choose if you want a 404s view in your statistics.
  • [by request] Spent some time optimizing critical sections of code, yielding slight (or sometimes, significant) log processing performance increases.
  • [by request] Added WinRoute Mail Log Format.
  • [by request] Added an option to skip previously-seen files during updates, by checking the filename. This is much faster than the normal method, which checksums the beginning of the file data, especially in FTP mode, where in some cases the whole file must be downloaded just to check if it needs to be processed. But this method will not work in cases where one or more of the log files is growing, or where files are being concatenated as part of rotation, or where several files in the log source have the same name (the standard checksumming method will handle all those cases).
  • [by request] Improved the progress page's layout, so it's more attractive and somewhat more functional.
  • [by request] Reworked the "log source" option to make it simpler to use. Instead of choosing from a menu and filling in several fields, you now enter the log source as a single value in on field. You can enter a pathname with or without wildcards, an FTP URL with or without wildcard, a command enclosed in backticks, or "stdin". The "Where is the log data" layout has also been improved, and a new "Show Examples" button shows several examples of common log sources.
  • [by request] Fixed session information tracking so the configuration contains a variable describing whether session information is tracked or not. This allows log processing to be slightly faster when session information is not tracked, and fixes a bug where session views appeared even if session information was not supposed to be available.
  • [by request] Added a new view showing the top year/month/days (a hierarchical date/time view).
  • [by request] Added a new option to ignore format lines while processing log data. Previously, if you defined a custom log field, it would disappear if your log data had a format line in it.
  • [by request] Added a new "Sessions Filter" option which applies to the session-related views. This works similarly to the Filters, but it applies to the session information, and the filter types are different. At this point, there is one filter type: you can filter the session information to show only sessions that hit a particular page. This is very handy, for instance, if you want to see how people got to your "sales" page-- you can set that page as a session filter, and look at the paths through the site view, to see where they entered and what they did before they hit the ordering page (and where they went afterward).
  • [by request] Added support for a new variant if Cisco IOS Log Format.
  • [by request] Reworked the way visitor information is tracked. Versions of Sawmill from 6.0 to 6.1.8 tracked visitor (unique host) information by cross-referencing the visitor id field against all other fields, treating it like any other field, and precomputing the number of unique hosts by examining the number of visitor ids cross-referenced to each field. This worked, but was highly inefficient because the many cross-references caused the database to be very large. More importantly, the previous visitor tracking method caused data to be poorly balanced across database segments, so it was possible to have very large segments (which resulted in very high memory usage). Sawmill 6.2 tracks the visitor information in a separate structure outsite of the main data of the database, where it can be optimized and compressed much better. The result is that processing speed is 50% faster, final databases are between 1/20 and 1/10 the size, and memory usage is greatly reduced (database segments are balanced). Because the format of the database has fundamentally changed in this version, all databases must be rebuilt from scratch-- old databases cannot be used.
  • [by request] Improved the appearance of the Calendar, and made the calendar fonts customizable.
  • [by request] Added a "one-time visitors" number to the Sessions summary view.
  • [by request] Added support for "fw logexport" style Firewall-1 logs.
  • [by request] Added an "export" link to the table which, when clicked, generates the table data as a CSV format text file. This can be easily imported into Excel or other programs.
  • [by request] Limited the number of simultaneous threads to 4 (you can customize the number). This eliminates a class of crashes which could result from rapidly clicking somewhere-- after a few clicks, Sawmill will refuse the connections until the existing threads are done processing.
  • [by request] Improved the configuration list page so you can double-click a configuration name to open/view/etc. it.
  • [by request] Removed the "Detail level" option from the "Create Configuration" interview. The "Full Detail" choice was too tempting-- everyone was turning it on, in spite of the warning about huge disk and memory usage. The option has been removed-- those few who truly need that much cross-referencing can easily add it manually in the Database Xrefs tab.
  • [by request] Added a "spider" log field and database field, and a "Top spiders" view. This view tracks the search engine spider hits. Like the "search engine" field, the "spider" field is computed using spider matching information in a user-editable file (SawmillInfo/Spiders) so you can add new spiders if you know of them.
  • [by request] Improved file management during database builds. Sawmill 6.1.8 and earlier would create many, many database segment buffers during builds, sometimes overwhelming the operating system it was running on. This version uses fewer segments to begin with, and also combines all buffers for a segment when possible, to greatly reduce the number of files used.
  • Added a ?open URL option (ex. http://{server}:8987/sawmill?open) which lets you type in the name of a configuration to open, and then opens it.
  • [by request] Added a "Current processing speed" indicator to the configuration status page, showing the speed at which the most recent chunk of log data was processed. Previous versions showed "Processing speed" which was the average processing speed over the whole analysis; this has been renamed to "Average processing speed."

Version 6.1.8, shipped July 31, 2001

Bugs fixed in version 6.1.8:

  • [by request] Fixed a bug where Sawmill would report an error that there was "no data in the log source" when the database was updated and there was no *new* data, when it should really only have reported that when the database was built.
  • [by request] Fixed a bug where certain types of DNS server responses were not handled correctly, resulting in many "failed" network lookups that should have succeeded.
  • [by request] Fixed a bug where tab-separated W3C-format logs were not being processed correctly.

Version 6.1.7, shipped July 30, 2001

Bugs fixed in version 6.1.7:

  • [by request] Fixed a bug where long statistics-viewing operations would never show the final page, if the browser was Netscape.
  • [by request] Fixed an error where Communigate Pro log file destination addresses were not being tracked properly.
  • [by request] Fixed a bug where Sawmill could deadlock during DNS lookup, if the log entry pool size was not large enough to hold all the simultaneous DNS lookups.
  • [by request] Fixed a bug, introduced in 6.1.6, where the (edit) Filters link in the statistics would generate an error.

New features in 6.1.7:

  • [by request] Enhanced FTP log source by allowing an optional port to be specified as part of the hostname.
  • [by request] Added "Starting day" and "Ending day" items to the Overview.
  • [by request] Added support for Microsoft ISA log format.

Version 6.1.6, shipped July 24, 2001

Bugs fixed in version 6.1.6:

  • [by request] Fixed a bug where newly-created scheduled items would not work unless you opened them and closed them again.
  • [by request] Fixed a bug where the session information was incorrect-- many of the sessions were being discarded incorrectly. This resulted in underreporting of values in all the session-related views, including the Sessions (Summary), the paths through the site view, the entry pages view, the exit pages view, and the time spent per page view.
  • Fixed a bug where Sawmill could crash when opening a subview in the Stats Views editor.
  • [by request] Fixed a bug where the REMOTE_USER feature was in the wrong part of the interface-- it was in the configuration, rather than in the Preferences. This basically meant that it didn't work unless you edited the SawmillDefaultConfig file manually.
  • [by request] Fixed a security issue where anyone could see a complete list of available configurations by entering a random username and clicking View.
  • [by request] Fixed a bug where the configuration password would be reset whenever Next was clicked on the "Do you really want to destroy all the data in the database" page.
  • [by request] Fixed a memory leak where Sawmill would not properly free memory used while computing session information.
  • [by request] Fixed a bug where Sawmill was not tracking bandwidth properly in log formats whose size field was not named "size."
  • [by request] Fixed a bug where clicking the Cancel button could cause a crash.
  • [by request] Fixed a bug where Expire Database would generate an internal error in some cases, if the database had date/time information to the hourly level and if daylight savings time was active.
  • [by request] Fixed a bug where Sawmill would crash every time it tried to generate a progress page in MacOS ACGI mode.
  • [by request] Fixed a bug where clicking "Scan and Summarize Log Source" could cause an error.
  • [by request] Fixed a bug where scheduled items did not run correctly on the MacOS version of Sawmill.
  • [by request] Fixed a bug where the "by hits" sort item in the Sort menu of the subview editor showed up as "by defaults" instead, making it impossible to choose "by hits" as the default sort.
  • Fixed a bug where the "time spent per page" view had a row for *every* page in the database, even if the page had no time spent on it (and even if it wasn't actually a page, but an image file).
  • Fixed a bug where non-page-view hits were being included in session information.
  • Fixed a bug where with certain types of IIS logs, Sawmill would get confused about which field was the page. That could cause several types of confusion, including incorrect session data and internal errors.
  • [by request] Fixed a bug where choosing a configuration name containing square brackets would cause problems while browsing. Sawmill no longer lets you use square brackets in configuration names.
  • [by request] Fixed a bug where Sawmill for MacOS would sometimes open a "SIOUX" window, double its menu bar, and start acting strangely.
  • [by request] Fixed a bug where dates of the format "mmm dd yyyy" were not being processed correctly-- they were being flagged as "corrupt," resulting in a "no log entries found" error.

New features in 6.1.6:

  • Improved feedback from Scheduled items-- Sawmill now sends an email to the action email address (if any) when an error occurs while processing a scheduled item, reporting the error message.
  • [by request] Improved the filter/field/view editing interface so it automatically scrolls to the one you just expanded or collapsed.
  • [by request] Added a new option to extract the date from log filenames.
  • [by request] Improved Communigate and Communigate Pro log formats so they extract the date from the log filenames when available.
  • [by request] Changed the default "database freshness" value to 1000000. This basically ensures that during a normal database build, the database will be consolidated only at the end (rather than every hour, as in previous versions). This means that the log data won't be incrementally available as the database is being updated or built (it will be browsable, but the new data won't appear until the end), but it also will speed up database builds, which is generally more important. The option can be modified, of course, for those who need "real-time" statistics.
  • [by request] Added an option to limit the amount of memory used to view session-related statistics. By default, Sawmill limits session memory usage to 10 Meg, which is enough to compute and display a moderate sized site's session information. However, for large sites, more memory may be required. With very large sites, previous versions of Sawmill could use arbitrary amounts of memory generating session statistics, causing crashes or performance problems. This version will no longer use more than the specified maximum; session views which would use more will display an error message instead.
  • [by request] Improved Sawmill's guess of the Temporary Directory URL to use the HTTP_HOST environment variable for the URL hostname, when available, rather than the IP number of the system running Sawmill. This is a better guess because on servers with separate IP addresses for each domain, the first IP address may not be the one for the site where Sawmill is installed, but HTTP_HOST will always be right.
  • [by request] Removed the "Sawmill Info Directory" and "Sawmill Temporary Directory" options from the Preferences. There are almost no circumstances where you need to change these options, and changing them can cause serious problems with the operation of Sawmill if you don't know what you're doing. For power users, they can still be changed by manually editing SawmillDefaultConfig.
  • [by request] Rewrote the DNS lookup algorithm. The previous one was a third-party library, and didn't work on MacOS 9 or some other platforms, and was buggy; the new one was written by us, so if there are bugs in it, we'll at least be able to fix them easily. The new algorithm seems to be roughly the same speed as the old one, or perhaps a little faster. Unlike the old library, the new one works on *all* platforms, including MacOS 9, so we finally have fast DNS lookup on MacOS 9 systems. This new algorithm should also fix the problem with DNS on Windows which were reported by several customers.
  • [by request] Added support for SecureIIS log format.
  • [by request] Enhanced the Cisco IOS log format to track route and ICMP information.
  • [by request] Enhanced the Cisco PIX log format to handle yet another variant.
  • [by request] Improved the m/d/y and d/m/y date formats to accept four-digit dates. Modified the WebSTAR log format to use d/m/y date format, so four-digit-date WebSTAR log files are now supported.
  • [by request] Added a new filter type which contacts a TCP/IP-based server to perform arbitrary filtering of one or more fields. This filter is very powerful, as it allows you to do arbitrary queries of external databases during log processing (for instance, to look up the full name from the username, and track it in the Sawmill database) by creating a small filter server script.

Version 6.1.5, shipped July 08, 2001

Bugs fixed in version 6.1.5:

  • [by request] Fixed a bug with IceCast log format, where Sawmill was not processing referrer or agent information correctly.
  • [by request] Fixed a bug with Common Error log format, where Sawmill was not processing date/time information correctly.
  • [by request] Fixed a bug where double-clicking the "Build Database" button, or trying to view the statistics for a building database using the same instance of Sawmill that was build it, would crash Sawmill.
  • [by request] Fixed a bug where you couldn't delete a duplicated database field because Sawmill complained that it was in use by a view.
  • [by request] Fixed a bug where plus (+) characters were converted to spaces when they were entered in any field.
  • [by request] Fixed a bug where date/time values could get corrupted while the log data was read, if they were in certain formats.
  • [by request] Fixed a bug where the the automatic update, which occurs when statistics are viewed and the database hasn't been updated in 24 hours, was not skipping over previously-seen data properly.
  • [by request] Fixed a bug where Sawmill would hang on some types of FTP server response.
  • [by request] Fixed a bug in NetPresenz log format where it would not properly process log files from 2000 or 2001.
  • [by request] Fixed a bug where log entries without URL in Cisco PIX logs would show up as "http://(empty)(empty)" in the log data, rather than as "(empty)".
  • [by request] Fixed a bug where "Generate HTML Pages" would not include some of the links and pages it should have (for instance, links from the Calendar).
  • [by request] Fixed a bug where Sawmill was not handling logs properly when they switched format partway throughout the log data.
  • [by request] Fixed a bug where accented characters (upper-ASCII) in language modules could cause a crash when viewing the Log Filters tab.
  • [by request] Fixed a bug where if you Browsed, clicked a red arrow, clicked your browser's Back arrow, and then tried to zoom in on something, it would ignore the last click and pretend like you had clicked the red arrow again.
  • [by request] Fixed a bug where in IIS logs without an uri-query field, Sawmill would report all pages as having ?(parameters), resulting in incorrect elimination of images and other non-page views.
  • [by request] Fixed a bug where Sawmill would crash if you fed it a WebSTAR (or WebTen) log file with a !!FORMAT COMMON_LOG_FORMAT line in it.
  • Fixed a bug where database expiration did not work properly if there was no visitor information in the database.

New features in 6.1.5:

  • [by request] Added support for W3SVC64 log format.
  • [by request] Added support for VICOM Gateway log format.
  • [by request] Added support for Checkpoint log format.
  • Added several new lines to the progress page which appear if DNS lookup is enabled-- these lines show the number of DNS lookup attempts, successes, failures, and timeouts.
  • [by request] Added support for Apache Error Log Format.
  • [by request] Added support for Miva Access Log Format.
  • [by request] Added support for dates of the format d/m/y and m/d/y (i.e. with leading zeros omitted for the month, day, and years).
  • [by request] Omitted the red arrows on lines in the referrer table which are clearly not URLs (ex. "(internal referrer)" or "(no referrer)").
  • [by request] Added a default log filter to convert "-" usernames to "(not authenticated)" so they are suppressed by default in the "Top authenticated users" field.
  • [by request] Added support for Winproxy log format.
  • [by request] Added support for Cisco IOS log format.
  • [by request] Added support for Communigate log format.
  • [by request] Added support for Communigate Pro log format.
  • [by request] Added support for SIMS log format.
  • [by request] Added support for Microsoft Proxy logs with dd/mm/yy dates.
  • [by request] Added support for IIS 3 logs format with dd/mm/yy-style dates.
  • [by request] Added default filters to categorize .swf and .js files as hits but not page views.
  • [by request] Added support for a different variant of tcpdump logs (one which includes the interface on each line).
  • [by request] Changed the behavior in the case that a log files does not have a size field, but theoretically allows bandwidth (ex. an IIS log without a SC-BYTES field). Sawmill used to generate an error message if you selected bandwidth tracking in this case; now it allows it, though unless the log format changes partway through he data to add a size field, reported bandwidth will be 0.
  • [by request] Changed the "View to send by email" option to be a dropdown menu, rather than a text field, so you can choose the view you want to send from a menu rather than having to type its name exactly correctly.
  • [by request] Added support for dates in the format m/d/yyyy (i.e. dates with 4-digit years where leading zeros are omitted).

Version 6.1.4, shipped June 15, 2001

Bugs fixed in version 6.1.4:

  • [by request] Fixed a bug where Sawmill was sending Action emails for every statistics view, with subject "Sawmill: ACTION_view_statistics".
  • [by request] Fixed a problem where clicking an "info" or "?" icon when running Sawmill on MacOS 9 and browsing it with Internet Explorer running on the same server would result in Internet Explorer hanging.
  • [by request] Fixed a bug where dates in the statistics could be off by one day, in certain situations where daylight savings time was active.
  • [by request] Fixed bug where "info" links did not work in some cases (such as while creating a configuration).

Version 6.1.3, shipped June 14, 2001

Bugs fixed in version 6.1.3:

  • [by request] Fixed a bug where duplicating a log filter in the Log Filters tab could cause Sawmill to crash.
  • [by request] Fixed a bug where referrer log analysis would try to analyze "operating systems" and "web browsers," even though that information is not available in referrer logs.
  • [by request] Fixed a bug which could cause an Internal Error during statistics browsing.
  • [by request] Fixed a bug where corrupt date/times in log entries could result in statistics reported from January 1, 2000 or January 1, 2001.
  • [by request] Fixed a bug where certainly types of log data (including some IIS logs) could cause Sawmill to report a large number of "(empty)" hits.
  • [by request] Fixed a bug in WebSTAR format support where Sawmill was not properly extracting the server domain in some cases.
  • [by request] Fixed a bug in the "apparent root" option where statistics were sometimes not bound to the apparent root branch of the database.
  • [by request] Fixes a bug in the internal hierarchy tracking code which could cause various errors, including directories appearing in the "top pages" view.
  • [by request] Fixed a bug where MacOS error windows could not be closed.
  • Fixed a bug where Expire Database would sometimes generate an error about the database being corrupt, when it really wasn't.

New features in 6.1.3:

  • [by request] Added support for iMail log format.
  • [by request] Changed the FTP password field to use bullets or asterisks, rather than displaying what is typed in the browser.
  • [by request] Changed the View Menu so it appears at the top *and* the bottom of the page, rather than one or the other.
  • [by request] Left justified the more/fewer/all/10 links in the tables, so they're visible even if the table is very wide.

Version 6.1.2, shipped June 08, 2001

Bugs fixed in version 6.1.2:

  • IMPORTANT: Due to a build problem, Sawmill 6.1.1 for some platforms did not have fix all the bugs it claimed to-- it was somewhere between 6.1.0 and the intended 6.1.1. This version fixes all the bugs mentioned below for 6.1.1.
  • [by request] Fixed a bug where Sawmill did not work properly under Apache on Windows-- it would complain about a security violation during statistics viewing.
  • [by request] Fixed a bug where Sawmill was not processing logs correctly in some cases when they had quoted fields.

New features in 6.1.2:

  • [by request] Added support for Rumpus log format.
  • IMPORTANT: Due to a build problem, Sawmill 6.1.1 for some platforms did not have all the advertised features-- it was somewhere between 6.1.0 and the intended 6.1.1. This version includes all features mentioned below for 6.1.1.
  • [by request] Improved support for GNATBox log format-- added support for long dates.
  • [by request] Improved support for tcpdump log format-- added support for a slight variant.

Version 6.1.1, shipped June 06, 2001

Bugs fixed in version 6.1.1:

  • [by request] Fixed several bugs with the page_frame_command-- it was not detecting properly when the command did not exist, and it was generating garbled error messages in some cases, and it was inserting the header code before the tag rather than before the tag.
  • [by request] Fixed a bug where if the configuration name contained a space, and the session timed out, then Sawmill would generate an error when you tried to view the statistics or go to the administrative menu from that page.
  • [by request] Fixed a bug where folders and drives containing apostrophes (') could not be browsed.
  • [by request] Fixed a bug where virtual domains were not being tracked correctly in WebSTAR logs.
  • [by request] Fixed a bug where if the filters rejected all log entries, Sawmill would sometimes generate a strange "the configuration is not in memory" error instead of the proper "No valid log entries found" error.
  • [by request] Fixed a bug where Sawmill could crash on Windows when "filters checkboxes and menu" was selected.
  • [by request] Fixed a bug where log entries with corrupt dates and times (including empty lines) would show up as 01/Jan/2001 hits.
  • [by request] Fixed a bug where Sawmill was not using quite the right syntax for SMTP MAIL messages, which some SMTP servers complained about.
  • Fixed a bug where DNS lookup debugging information was bring printed on console.
  • [by request] Fixed a bug where Generate HTML Files generated blank pages.
  • [by request] Fixed a bug where DNS lookup was not working on MacOS 9.
  • [by request] Fixed a bug where the multithreaded log reading options were not appearing properly in the Configuration Options-- they can now be found in the Database Params tab.
  • [by request] Fixed several bugs in UNIX Sendmail format which resulted in most log entries being misreported in some logs.
  • Fixed a bug where the response page for "Send View By Email" was misformatted.
  • Fixed a bug where selecting the empty row in the Options menu (which was a difficult thing to do, but could be done with some effort) would generate a JavaScript error.
  • [by request] Fixed a bug where Sawmill would sometimes generate an Internal Error during database builds.
  • [by request] Fixed a bug where the "go" links (red arrows) on URLs would get garbled if the URL contained < or > characters.
  • [by request] Fixed a bug where Sawmill was not properly handling the responses from certain FTP servers, resulting in hangs or crashes.
  • [by request] Fixed a bug where clicking Details while using the "Add Log Data To Database" feature could cause Sawmill to crash.
  • [by request] Fixed a bug where files and folders on the Windows desktop were not working properly when Browsed-- folders were appearing as files and files were not being entered properly in the field when they were clicked.
  • [by request] Fixed a bug where the administrative password was not working to automatically bypass the configuration password prompt page.
  • [by request] Fixed a bug which could cause crashes when turning on the "filters checkboxes and menu" option in the Windows version.
  • [by request] Fixed a bug where Sawmill would sometimes crash just after a scheduled job finished running.
  • Moved the Language Module option to the Preferences, where it belonged.

New features in 6.1.1:

  • [by request] Improved reporting of server domain (virtual server) in several log formats where is was recognized but not tracked.
  • [by request] Improved GNATBox support to handle more variations of the format, and to process WWW and FILTER log entries (instead of just NAT).
  • [by request] Added a "repeat visitors" value to the sessions overview.
  • [by request] Improved Cisco PIX log format to track accessed URLs better.
  • [by request] Enhanced the "Send View By Email" and "action email" features to allow a comma-separated lists of email addresses, to specify multiple recipients.
  • [by request] Improved the error message you get if you try to set up a "action email email" without setting an SMTP server or an return address.
  • [by request] Improved the email message sent by "action email" to show the action in the subject, and to include a better-format message body.
  • [by request] Added a brief note to the bottom of the "Where is the log data" page, explaining that gzipped files will be automatically decoded.
  • [by request] Improved the error message that occurs when Sawmill is restarted during a long operation ("Progress Page Not Found").
  • [by request] Added support for IAS log format.
  • [by request] Improved the Sawmill Error page so in the debugging traceback, it doesn't show full pathnames and drive names. This eliminates a problem where some users were getting confused when some users were getting confused because the traceback referred to disks and pathnames that did not exist on their own system.
  • [by request] Added an "average visitors per day" value to the Overview.
  • [by request] Improved Sawmill's behavior when you click a documentation link while the documentation window is open but behind the current window-- Sawmill used to display the requested documentation page but not bring the window to the front; now it brings the documentation window to the front.
  • [by request] Added a documentation page describing language modules, localization, and text customization.

Version 6.1.0, shipped May 28, 2001

Bugs fixed in version 6.1.0:

  • [by request] Fixed a bug where progress pages were being displayed incorrectly on Netscape 4.x.
  • [by request] Fixed a bug the Options menu had some meaningless option in the "traffic over time" view (including "totals row").
  • Fixed a bug where the configurations menu in a Scheduling item was a long list, rather than a popup menu as it was supposed to be.
  • [by request] Fixed a bug where Sawmill was not processing logs correctly when they used Macintosh-style line endings (\r).
  • [by request] Fixed a bug where Sawmill would generate an error if you entered a value containing a quote (") into a field in the configuration options.
  • [by request] Fixed a bug where configurations appeared in the User menu, even if they had "Don't show this configuration in the User menu" checked.
  • [by request] Fixed a bug where the sort order for a subview in the Stats Views tab was not being saved properly.
  • [by request] Fixed a bug where clicking the Administrative Menu button in the Scheduler did not save the changes.
  • [by request] Fixed a bug where the administrative password did not work when entered at a "view statistics password" prompt.
  • [by request] Fixed bug where hitting the return key on the ?browse page would go to the administrative menu instead of to the statistics.
  • [by request] Fixed bug where if there was no "overview" view, the links on the calendar were broken.
  • [by request] Fixed a bug where Sawmill could crash under some circumstances while processing log data.
  • [by request] Fixed a bug in Microsoft Proxy Format support, where certain non-HTTP proxy logs would not be recognized or properly processed.
  • [by request] Fixed a bug where Sawmill would sometimes prompt repeatedly for the statistics-browsing password.
  • [by request] Fixed a bug in the Cisco PIX support, where bandwidth transferred was not being tracked properly in some cases.
  • [by request] Fixed a bug where the 95th percentile translucent shading was not showing up properly in reverse-chronological date graphs.
  • [by request] Fixed a bug where the statistics page would sometimes not appear correctly after a lengthy computation involving progress pages.

New features in 6.1.0:

  • [by request] Added multithreaded log processing. Sawmill can now run multiple simultaneous threads of execution as it reads the log data, each thread handling a different section of the processing (log parsing, log filtering, database feeding, database consolidation). This allows database builds to run significantly faster on multiprocessor systems.
  • [by request] Greatly improved DNS lookup. Sawmill now looks up many IP numbers at a time (even in single-threaded mode) -- this eliminates problems in previous versions where log processing would completely stop during DNS timeouts, resulting in extremely slow processing speeds. Log processing with DNS lookups is hundreds or thousands times faster with this version than with previous versions.
  • [by request] Greatly improved the way database builds and database viewing interact. Previous versions of Sawmill would refuse to let you view a database if it was being built. Sawmill now lets you view a database at any time, including in the middle of the build. You can reload repeatedly, for instance, and watch the numbers grow as Sawmill processes the log data. This also makes true real-time statistics possible-- you can use a command log source which "tails" your log file (or you can set up a schedule to update continuously), and the statistics will always show up-to-the minute numbers.
  • [by request] Added a new mode in the Scheduler where you can select "all configurations" from the menu to have it perform the specified operation on all configurations, in sequence. For instance, if you have 1000 configurations, you can now update them all at midnight with a single Scheduler entry-- Sawmill will run the updates one after the other, starting at midnight. This simplifies scheduling for ISPs and similar situations, and also increases efficiency by running jobs end-to-end, eliminating the need to space them widely to prevent overlap.
  • [by request] Added a new "Create Many Configurations" feature. This feature lets you create many configurations based on an existing configuration, and is ideal for situations like those found in most ISPs, where there are hundreds or thousands of configurations, all very similar. Configuration options containing the configuration name are automatically converted in the new configurations to contain the new configuration name, so by judicious choice of the configuration names, this can be used to fully automate the creation of many configurations with similar structure (for instance, it can be used to point each configuration at its own log data). Further per-configuration customization can be done by entering configuration options for each configuration. All information is entered in a single large text field, making it easy to save the setup for later modification and regeneration.
  • [by request] Improved memory management of the memory write buffer. Sawmill now processes logs faster for the same amount of memory write buffering, especially for very large databases. Sawmill also now uses as little as half the requested write buffer memory (and as much as all of it), so on average, it will use less memory for a database build than earlier versions.
  • [by request] Added a new data type, "sessions," in addition to the previous four (hits, page views, bandwidth, and visitors). Session information appears in session-related views like the entry pages time-per-page views. Previously, these were shown as "visitors," but that was not really accurate, because sessions are not the same as visitors (sessions take repeated visits into account, and visitors do not).
  • [by request] Added a "help" link to the filters editor, leading to the Log Filters documentation page.
  • [by request] Added support for Software602 log format.
  • [by request] Improved the View Statistics operation so that if the database isn't built when you try to view the statistics, the database will be automatically built before the statistics are viewed. Previous versions hid the View Statistics button when the database was not yet built, but that could be circumvented using a "view" URL, and doing so would cause a not-too-friendly error.
  • [by request] Improved the View Statistics operation so that if the database hasn't been updated recently (the exact time is configurable), Sawmill updates it first, and then shows you the statistics.
  • Improved field/filter/etc editors to hits the "show all" and "hide all" icons when there is nothing in the list.
  • [by request] Greatly improved the page that appear when no valid log data is found during a database build.
  • [by request] Improved GNATBox format support to allow a slightly differently formatted log entries.
  • [by request] Added a comment to several database-related tabs in the configuration options, pointing out that changes don't take effect until the database is rebuilt.
  • Enhanced online documentation-- it now includes all the information that used to be in the online FAQ page.
  • [by request] Added a note to the statistics when parenthesized items are omitted from a table.
  • Added little purple question mark (?) icons to some places in the interface. Clicking them leads to FAQ entries which answer common questions that you might have at that point. For instance, clicking the ones next to the "(parameters)" part of a page table shows the FAQ entry explaining how to enable page parameters.
  • [by request] Added support for SHOUTcast 1.8 log format.
  • Improved error reporting in the case that someone enters an FTP or HTTP URL in the log filename field. Sawmill does not currently understand URLs, so FTP log sources need to be specified in several pieces (hostname, username, etc.) using an FTP log source. Sawmill does not currently support HTTP download at all.
  • [by request] Added the "session timeout" and "max session duration" information to the sessions overview table.
  • [by request] Added a new variant of Cisco PIX / Kiwi format, which includes the year in the date.

Version 6.0.12, shipped May 10, 2001

Bugs fixed in version 6.0.12:

  • Fixed a bug where the labels on the Single-page Summary page were missing in some cases when using Generate HTML Files mode.
  • [by request] Fixed a bug where if there was a directory in the LogFormats directory (which shouldn't normally happen, but could with some types of file system browsers), Sawmill would generate an error while trying to autodetect the log format.
  • [by request] Fixed a bug where Expire Database could fail with an internal error if the database was tracking visitor information.
  • [by request] Fixed a bug where the filters were not appearing at the top of the "Single-page summary."
  • Fixed a bug where some files in the database were not being erased when the database was rebuild, causing there to be unnecessary files in some databases, so they used more disk space than necessary.
  • [by request] Fixed a performance problem where the Consolidating stage of database builds took much longer than it had to for large databases. The Consolidating stage is now much, much faster in some cases.
  • [by request] Fixed a bug in the filters display where session-related views (which support only date/time or page filters) were allowed to have other types of filters without any complaints, but the filters were being quietly ignored. Now the filters are loudly ignored-- a red message in the filter bar warns that some of the filters are not in effect for the current view.
  • [by request] Added new "Use built-in web server" preference which controls whether the built-in server is run. Previously, that was controlled by setting the "server hostname" to something, but that was problematic because some users wanted to set the default hostname to something, but not always start the server. "server hostname" has now been decoupled from the option that starts the server.
  • [by request] Fixed a bug where several elements were not available in the time per page and entry/exit page views, including remainder row, average rows, total row, and more/less/all links.
  • [by request] Fixed a bug where the Administrative menu would sometimes not display properly on Netscape 6.
  • [by request] Fixed a bug where statistics tables were unnecessarily narrow when viewed with Netscape 6.
  • [by request] Fixed a bug where the page after a series of progress pages could be garbled.
  • [by request] Fixed a bug where Sawmill would crash if you entered an invalid hostname (for instance a URL) in the FTP server field.
  • Fixed a bug where clicking the "Back to Administrative Menu" button in the View Statistics selection screen would view the statistics instead of going back to the menu.
  • Fixed a bug where if you were using the "page frame command" option to wrap your statistics in a custom interface, the "edit filters" link did not work.
  • [by request] Fixed a bug where the "save password" feature was not working properly with Netscape.
  • Fixed a bug where some of the images in the web interface would sometimes be broken.
  • Fixed a bug in the Alpha/Linux version of Sawmill where visitor information was being computed incorrectly.
  • [by request] Fixed a bug where if you turned off "show doc links in stats", the "info" icons would disappear in the administrative interface for that configuration.
  • [by request] Fixed a bug where Sawmill was not accepting certain valid responses from slightly unusual SMTP servers, causing an error when it tried to send mail.

New features in 6.0.12:

  • [by request] Added a "visitors" column to the "time per page" view.
  • [by request] Added a new "Maximum session duration" option. All sessions longer than this are rejected from the session data (though not from the rest of the database). This reduces the problem of large cache servers appearing as single long sessions when they are in fact collections of sessions routed through the same cache.
  • [by request] Added several new search engines to the list recognized by Sawmill, including MSN Search, Dogpile, and AOL Search.
  • [by request] Improved the algorithm which decides how many database segments to use. The previous algorithm always chose 10; this one now chooses between 10 and 80, depending on the complexity of the database detail level. This reduces a problem which could occur when there was a really huge amount of data being processed, and full detail was turned on-- by splitting the database into only 10 segments, each segment could grow to several hundred megabytes in size, and the build process could run out of memory while consolidating the segments. Large, complex databases should use 1/8th the amount of memory now with the default settings. The option can also be controlled manually if desired.
  • [by request] Improved the (more info) links from the single-page summary view so they preserve the filters.
  • [by request] Added a link from the statistics page to the configuration page (if you haven't used the "save my password" feature, you will need to enter the admin password to use the link).
  • [by request] Improved the appearance of the "paths through the site" view, by eliminating the dangling corners on some lines, and compacting it a bit.
  • [by request] Improved the functionality of the "paths through the site" view by scrolling the page to keep what was just clicked visible.
  • [by request] Added a "browse to folder" button on the Browse page, so you can enter the folder/directory name and go straight there.
  • [by request] Replaced the Show and Hide menus with a single "Option" menu contain actions like "show hits column" and "hide visitors pie chart." This simplified the interface and makes it easier for experienced users to show/hide items, because they always in the same position in the menu.
  • [by request] Extended the limit on the number of divider characters for a field from 2 to 3, so for instance the page field can now be divided on /, ?, or &.
  • [by request] Brought back the "bookmark" feature, which was present in v5 but was disabled in previous versions of v6. Also improved the bookmark feature to make the bookmark URLs shorter and easier to read.
  • [by request] Changed the default number of database segments so it is between 10 and 80 (instead of always 10) depending on how complicated the database detail is. This reduces a problem where processing a log of log data with "full details" and all checkboxes checked could require huge amounts of memory.
  • [by request] Added support for IceCast log format.

Version 6.0.11, shipped May 04, 2001

Bugs fixed in version 6.0.11:

  • Fixed a bug where Sawmill was printing "Content-type: text/html" when used in command line mode.
  • Fixed a bug where the "graph over time" could be missing from the top of the "Single-page summary" when there was no visitor information in the database.
  • Fixed a bug where the database could be left in a "modifying" state after a successful build, preventing access for 60 seconds.
  • [by request] Fixed a bug where Expiring the database could cause an Internal Error.
  • [by request] Fixed a bug where the progress bar for the Consolidate stage of a database build would sometimes be pegged at 100% for the whole stage.
  • [by request] Fixed a bug where if you opened a view in the Stats View tab of the Configuration Options, and the database was not yet built, you would get an error. Sawmill now lets you open views and subviews, though you can't edit the filters for them until the database is built.
  • [by request] Fixed a bug where the "Log entry name" option did not work.
  • [by request] Fixed a bug where it was possible to delete a database field when there were subviews that depended on it. Sawmill no longer lets you do that-- you must delete the subview first, and then delete the database field.
  • [by request] Fixed a minor bug where Sawmill was creating many more temporary directories than it had to.
  • [by request] Fixed a bug where if you let a configuration build for a while, and used Sawmill to do other things in the mean time, then when you clicked Details in the progress page you would get an error.
  • [by request] Fixed a bug where the links in emailed views were broken.
  • [by request] Fixed a bug where the MacOS version would create several unnecessary GIF files in the Sawmill folder when it was run.
  • Fixed a bug where using the "Add Other Log Data To Database" feature would permanently change the log source for the configuration to the log source entered for the add operation.
  • Fixed a bug where if an error occurred in the middle of generating some types of pages (ex. the "database detail" page), the error would not be displayed properly.
  • [by request] Fixed a bug where the more/fewer/all item appeared in the Show/Hide menus of the "graphs over time" view, even though it didn't do anything there.
  • [by request] Fixed a bug where Sawmill was not properly releasing memory it used while generating the "paths through the site" and other session views, causing it to eventually run out of memory.
  • Fixed a bug in MacOS ACGI mode where an large blank window would appear.
  • [by request] Fixed a bug in MacOS ACGI mode where Sawmill would complain that the server was too busy.
  • [by request] Fixed a bug where some operations did not work properly when using Sawmill with IE 4.5 on MacOS-- clicking a folder or disk in the Browse window did not work, and choosing a different log source type would start a Browse operation, and clicking Apply Filters after editing them in the View Options would go to the configuration page.
  • [by request] Fixed a bug where in some circumstances, Sawmill could crash while accessing an FTP server.

New features in 6.0.11:

  • [by request] Added a default log filter to the IIS log format description, so that URL query parameters are copied to the end of the page field, allowing for easy analysis of parameters.
  • [by request] Added support for mm/dd format dates.
  • [by request] Added a new view, and a new subview in the "Single-page view," showing 404's (broken links).
  • [by request] Added support for Snort log files.
  • Improved the performance of session views like the "paths through the site" view, in web server mode. Sawmill used to reread and reprocess the session information for every click; now it keeps it in memory with the rest of the active database, so session statistics generate much faster.
  • [by request] Improved Sawmill's response when a session timed out. Previously, it would generate a cryptic error page stating that a particular ugly-named configuration file could not be found. Now it generates a much nicer-looking page explaining that the session has timed out, and giving two options (go to the Admin menu or view the statistics again from the start).
  • Added a $notsupported option to log format descriptor files so that log formats which do not support a particular type of data (for example, day-by-day data or page views) can declare it. Sawmill will no longer show the options which are not supported for the format. All built-in formats have been updated to include the option where appropriate.
  • [by request] Added descriptions to the server responses, so for instance now instead of 404, it says "404 (Not Found)".
  • [by request] Added arrows to the field/filter/etc editors, to make it possible to change the order of items easily.
  • [by request] Added "expand all" and "collapse all" icons to the filter/field/etc editors, to make it possible to expand or collapse all items in a single click.

Version 6.0.10, shipped April 27, 2001

Bugs fixed in version 6.0.10:

  • [by request] Fixed a bug where Sawmill was not properly communicating with some FTP server, including ProFTPd.
  • [by request] Fixed a bug where with some web servers, including IIS, Sawmill was generating statistics in a way that confused the server/browser into trying to download the HTML as a file, rather than displaying it.
  • [by request] Fixed a limitation where Sawmill was reporting all connecting hostnames as "unknown." It now looks them up properly.
  • [by request] Fixed a minor bug where Sawmill would generate directories it didn't need during command-line database builds and updates.
  • [by request] Fixed a bug where the "autologon" feature did not work in CGI mode.
  • [by request] Worked around a bug in IE 5 where IE would repeatedly submit large forms. The result of this was that you would sometimes be dumped back to the administrative menu while browsing the configuration options. The workaround should prevent this, but you still may see forms reload up to three times if you're using IE 5.5.
  • [by request] Fixed a bug where large form submissions (including Log Filter, Statistics Fonts, and some other types of form submissions) would fail, or load twice, or even go to the wrong page, when used with Internet Explorer 5. This is really a bug in IE 5, not Sawmill, but Sawmill now works around it so it won't be a problem.
  • [by request] Fixed a bug where Sawmill was not properly autodetecting the log fields in log files which had the fields defined in the headers, and had headers more than five lines long.
  • [by request] Fixed a bug where Sawmill was not properly detecting username and server response fields in some types of log formats, including Quicktime Streaming Server.
  • [by request] Fixed a bug where in databases tracking large amounts of bandwidth (more than 4G transferred) and visitor information, Sawmill could generate an error saying that a database segment was corrupted.
  • [by request] Fixed a bug where visitor percentage information was not appearing in some views.
  • [by request] Fixed a bug where the "Back to Administrative Menu" button in the Licensing page took you back to the Licensing page instead.
  • [by request] Fixed a bug where some parts of the Configuration Options were not appearing when viewed with Netscape.
  • [by request] Fixed a bug where clicking twice on a link could cause an internal error.
  • [by request] Fixed a bug where the Browse feature did not work correctly when browsing or clicking folders whose names contained spaces.
  • [by request] Fixed a bug where Sawmill could crash while processing FTP directories containing many files.

New features in 6.0.10:

  • [by request] Added a "concatenate" log filter which can be used to concatenate the values of two fields together.
  • [by request] Added explicit support for Quicktime Streaming Server log files-- they were supported before but you had to know to choose IIS4 Log Format. Now they are supported directly as "Quicktime Streaming Server Log Format."
  • [by request] Added support for HTTP/1.1 in Sawmill's built-in web server. Previous versions only supported 1.0. HTTP 1.1 allows multiple requests to be carried over a single connection, increasing the speed at which images are loaded into the pages.

Version 6.0.9, shipped April 22, 2001

Bugs fixed in version 6.0.9:

  • [by request] Fixed a bug where some of the Log Filter options were missing when using Sawmill via Netscape.
  • [by request] Fixed appearance of sidebar buttons in Netscape.
  • [by request] Fixed a bug where WU-FTPd dates were being processed incorrectly (the month was always January).
  • [by request] Fixed a bug where if you edited filters in the Stats Views tab, then click Apply Filters, Sawmill would prompt for a password.
  • [by request] Fixed a bug where if Sawmill was running in CGI mode on a system without a hostname configured, it would not require an administrative password.
  • [by request] Fixed a bug where if you were reading multiple files from an FTP log source, and you deleted some of the files and updated the database, files which had already been processed would be re-added to the database.
  • [by request] Fixed a bug where the "Send View By Email" button would generate a Server Error (after correctly sending the email) when used in CGI mode.
  • [by request] Fixed a bug where the visitors % column was always filled with dashes.
  • [by request] Fixed a bug where the "operation" field was not being correctly extracted from some IIS log files.

Version 6.0.8, shipped April 18, 2001

Bugs fixed in version 6.0.8:

  • [by request] Fixed a bug where Update Database could generate an internal error.
  • Fixed a bug where 95th percentile information was available even when there was no bandwidth in the database, and it didn't make sense to show it.
  • Fixed a bug where with certain combinations of limits, the Show/Hide menu selections would lead to the administrative login page.
  • Fixed a bug where date/time graphs would randomly fail to appear if there was no hits information in the database.
  • Fixed a bug where the vertical tick marks in date/time graphs were slightly off.
  • Fixed a bug where deleting a database field which was alone in its xref group did not delete the xref group. This could cause infinite looping during database builds.
  • [by request] Added a "create a new subview" icon to the view/subview editor. This fixes a problem where if you deleted the only subview of a view, you could not create a new subview for that view.

New features in 6.0.8:

  • [by request] Added an optional parameter to the ?user URL mode so that only configurations beginning with that parameter will be shown. For example, http://myhost:8987/sawmill?user+tom will show the User menu, but the menu will only contain configurations whose names start with "tom".
  • [by request] Improved the "please choose a configuration" page to use a multiline selection list rather than a popup menu, for easier selection.
  • [by request] Enhanced the 95th percentile information. Sawmill can now compute 95th percentile information based on arbitrary sample intervals (it used to be limited to 1 second, 1 minute, 1 hour, 1 day, etc.)-- you can now compute it based on 5-minute samples, for instance. Sample size can be modified "live"-- so you can quickly recompute the 95th percentile information based on different sample sizes by selecting a new sample size from a menu. Samples are also plotted translucently on the bandwidth graph, and the 95th percentile value is plotted as a red line.

Version 6.0.7, shipped April 15, 2001

Bugs fixed in version 6.0.7:

  • [by request] Fixed a bug where Sawmill was rejecting all log entries in IIS log files where the date was listed in the header but not in each entry.
  • [by request] Fixed a bug where Sawmill could hang at the licensing page in CGI mode.
  • [by request] Fixed a bug where Sawmill could crash while submitting the "Database Xrefs" tab.
  • Fixed a bug where FTP log source of gzipped files did not work properly with some FTP servers.
  • Fixed a bug where certain types of FTP server response could hang Sawmill.
  • Fixed a bug where in CGI mode, there would sometimes be a spurious "Content-type: text/html" line between the progress lines and the contents.
  • [by request] Fixed a bug where Sawmill was treating all log entries as coming from the same day, in an IIS log with the date listed in the header and also listed in each log entry.
  • [by request] Fixed a bug where Sawmill would not browse properly into directories whose names contained spaces.
  • Fixed a bug in the MacOS/Carbon version of Sawmill where the cursor would sometimes remain a watch after initialization was done (rather than becoming an arrow).
  • Fixed a bug in the MacOS 8/9 and Carbon versions of Sawmill where under certain circumstances involving unexpected activity from the browser, the built-in web server could hang.
  • Fixed a bug where if when you clicked Apply Limits after editing the filters in the Stats View tab, Sawmill would prompt for a password.
  • Fixed a bug where tables would sometimes contain items with zeros in all columns.

New features in 6.0.7:

  • Improved CGI/IE mode progress, so Sawmill no longer repeats the progress line over and over-- it only adds dots to the existing line until the progress description changes. This can greatly reduce the amount of progress output that accumulates at the top of the page.

Version 6.0.6, shipped April 11, 2001

Bugs fixed in version 6.0.6:

  • [by request] Fixed a serious counting bug where in some databases with visitor information tracked, Sawmill would incorrectly count the number of hits, page views, bytes transferred, and visitors.
  • Fixed a bug where hitting return in the "Please name this configuration" page would take you to the Administrative Menu, rather than to the Configuration Page.
  • [by request] Fixed a bug where Sawmill was creating the SawmillInfo directory in the current directory, rather than in Sawmill's own directory as in previous versions.
  • [by request] Fixed a bug where the Log Source menu did not work in the Configuration Options or in the Default Configuration.
  • [by request] Fixed a bug where clicking the Back to Administrative Menu button did not save changed in Preferences or Default Config.
  • [by request] Fixed a bug where there was a Colors tab in the Preferences screen that didn't do anything.
  • [by request] Fixed a bug where the Security Mode was not working properly.
  • [by request] Fixed a bug where web browser and operating system information was not being processed correctly in W3C-based formats (ex. IIS 4+).
  • [by request] Fixed a bug where Sawmill did not communicate correctly with certain FTP servers.
  • [by request] Fixed a performance bug where for large databases, the Consolidating stage of the build could take an extraordinarily long time. That stage, which could be 75% of the build time for a large database, has been sped up by a factor of 100 or more.

New features in 6.0.6:

  • [by request] Changed the default permissions Sawmill uses in a UNIX environment when creating files, to eliminate possible security problems which could arise if a Sawmill administrator used the default permissions and did not modify them to make them secure. The new permissions are tighter, and never include world write access.

Version 6.0.5, shipped April 08, 2001

Bugs fixed in version 6.0.5:

  • [by request] Fixed a bug where Sawmill would hang fetching an FTP log source if there was nothing in the specified directory.
  • [by request] Fixed a bug in the statistics HTML which could cause problems with older versions of Netscape.
  • Fixed a bug where only the first 10 hours of the day appeared in the "Single-page summary" view.
  • [by request] Fixed a bug where items in tables would be omitted if any of the values (hits, bandwidth, etc.) were zero, even if there was data in the other values. As a side-effect, this also caused crashes when viewing the date/time graph.
  • [by request] Fixed security hole where it was possible to gain access to the administrative menu without using a password.
  • [by request] Fixed a bug where the SawmillAccessLog file would sometimes be truncated.
  • [by request] Fixed a bug where Sawmill would hang fetching an FTP log source if there was nothing in the specified directory.
  • [by request] Fixed a bug in the statistics HTML which could cause problems with older versions of Netscape.
  • [by request] Fixed a bug where some log formats, including Raptor, were not being processed correctly.
  • [by request] Fixed a bug where web-GUI database builds would sometimes not properly release write access to the database, resulting in errors when trying to browse or rebuild the database.
  • [by request] Fixed a bug where labels could be missing or absent on emailed views if command-line limits were used during sending.
  • Fixed a bug with some log formats where some of the sidebar buttons were not clickable.
  • [by request] Fixed a bug where Sawmill could crash while erasing the database.
  • [by request] Fixed a bug where Sawmill could crash while viewing statistics for a log with no date/time information.
  • [by request] Fixed a bug where under some circumstances, the tabs on the Configuration Options page would show broken pages or no data at all.

New features in 6.0.5:

  • [by request] Simplified the CGI installation instructions.
  • Alphabetized the list of log formats.
  • Added Show/Hide menus to the bottom of multi-subview views like the Single-page summary view.
  • [by request] Added Common Error log format.
  • [by request] Added a new date/time format: yyyymmddhhmmss.

Version 6.0.4, shipped April 03, 2001

Bugs fixed in version 6.0.4:

  • Fixed a bug where with some browsers, in CGI mode, the expand/collapse icons did not work in the paths-through-the-site view.
  • Fixed a bug where with certain types of v5 configuration files, Sawmill was not parsing logs correctly.
  • [by request] Fixed a bug with v5 configurations and databases, where Update Database would fail or crash.
  • [by request] Fixed a bug where under certain circumstances, Sawmill would crash while generating the configurations list.
  • [by request] Improved Sawmill's handling of corrupt date/times in log files-- Sawmill now rejects log entries whose date/time fields are corrupt.
  • [by request] Fixed a security flaw where it was possible for a clever hacker to gain access to the Administrative menu without using a password.
  • [by request] Fixed a cosmetic error in the MacOS 9 version where the About window said "Sawmill 5."
  • [by request] Fixed a bug which could cause crashes when the statistics were viewed.
  • Fixed a bug where the operating system and browser type were not being computed correctly in some cases.
  • Fixed a bug in the MacOS version where clicking the URL link in the "Show advanced information" section of the Sawmill window caused an error.
  • [by request] Fixed a bug where if visitor counting was turned on, but there was no visitor id log field, or no visitor id database field, or if the visitor field was not cross-referenced to everything else, Sawmill would give a "corrupt database" error on database build.

New features in 6.0.4:

  • [by request] Simplified trial licensing. You no longer have to get a trial key from the flowerfire web site-- Sawmill will generate one for you if you click the "Try Sawmill for 30 Days" button in the licensing page.
  • [by request] Added support for the short form of IIS 3 log format.
  • [by request] Added support for Shoutcast log format.
  • Created a "Carbon" version of Sawmill for MacOS. This version will run on MacOS 8/9 with the Carbon libraries, or on MacOS X. On MacOS X, it will use the Aqua interface for the Sawmill window and menus.

Version 6.0.3, shipped March 28, 2001

Bugs fixed in version 6.0.3:

  • Fixed a bug where every time you created a new configuration, Sawmill would regenerate the log format descriptor files.
  • Fixed a bug where Sawmill did not correctly auto-detect some types of Microsoft Proxy logs.
  • Fixed a bug where Sawmill did not process certain types of logs correctly, including Microsoft Proxy and IIS3.
  • Fixed a bug where in certain circumstances, generating a table could cause a divide-by-zero error. and IIS3.
  • [by request] Fixed some bugs in the view elements, where some were available when they didn't make sense, especially in the sessions summary view.
  • Fixed a bug where the "number of visitors" column was missing from the entry and exit pages views if visitor information was not tracked in the database, even though the information is independent of visitor information and could be displayed.

New features in 6.0.3:

  • Re-enabled the "read log data from the output of an arbitrary command" feature, which had accidentally been disabled.
  • [by request] Added a new option and view element, "Show/Hide menus affect all views." This is a checkbox which may itself be shown or hidden using the Show/Hide menus. When it is checked (the default), Show or Hide menu selections affect all subviews; when it is unchecked, they affect only the current subview. In previous versions of Sawmill, Show/Hide menu selections always affected only the current subview; this makes it easier to make changes affecting many views (and it works more like people expected it to).
  • Added a date/time graph to the "Single-page summary" view.

Version 6.0.2, shipped March 27, 2001

Bugs fixed in version 6.0.2:

  • [by request] Fixed a bug where under some circumstances, views sent by email would have incorrect or missing labels in the tables.
  • [by request] Fixed a bug where the rows of some tables were taller than they should have been.
  • [by request] Fixed a bug where Sawmill would sometimes not detect the end of a session in FTP mode, and would hang forever waiting for the end.
  • [by request] Fixed a bug where Sawmill would not allow certain kinds of multi-line FTP responses (it would generate an error when the server sent them).
  • [by request] Fixed a bug where Update Database did not work under some circumstances with a FTP log source (it would re-read already seen data).
  • [by request] Worked around a bug in Netscape 6 where it would not accept Sawmill's default charset, so it was generating an immediate error when you tried to access Sawmill's web interface.

New features in 6.0.2:

  • [by request] Improved performance for log processing, especially for default settings (minimal database), and especially on BSD/OS.
  • [by request] Eliminated a common cause of licensing woes by stripping off leading and trailing spaces in entered license keys.
  • Added a new type of log filter, replace_after, which replaces everything after a specified string with another string. This is used by default to remove page parameters, for improved performance over the filter that used to do that, which used a regular expression.

Version 6.0.1, shipped March 24, 2001

Bugs fixed in version 6.0.1:

  • [by request] Fixed the 95% percentile calculation, which was computing incorrect results.
  • [by request] Fixed some bugs in the conversion of v5 configuration files.
  • [by request] Fixed a few cosmetic bugs in the documentation.
  • [by request] Fixed a bug which could cause "unable to compile regular expression" errors on BSD/OS, MacOS X, and possible other platforms.
  • [by request] Fixed a bug where the Sessions Summary view reported incorrect numbers.
  • [by request] Fixed a security issue where the documentation viewing tab would print what was searched for without validating that it didn't contain malicious HTML (like JavaScripts).

New features in 6.0.1:

  • [by request] Improved the appearance of the statistics by using more compact fonts and layout, especially for the views sidebar.
  • [by request] Added a 95% calculation to the date/time graph, and a line to show where the 95% mark is.
  • [by request] Added support for Pardeikes' Welcome Plugin Log Format.

Version 6.0, shipped March 21, 2001

Bugs fixed in version 6.0:

  • Fixed a bug where the Show/Hide/Sort links in the calendar view lead to the Overview.
  • Fixed a bug where the Show/Hide menus contained unreasonable options in the Calendar view, and the Sort menu appeared when it shouldn't have.
  • Fixed a bug where editing the log filters from the web GUI could cause unintended parallel edits in the log parsing filter.
  • Fixed a bug where the date/time graph could be incorrectly shaded/colored if there were a lot of data points.
  • Fixed a few bugs in the language modules related to searching documentation.
  • [by request] Fixed a bug where hitting return in the Search field of the documentation would take you to the admin menu.
  • Fixed a bug where clicking a link in one view could take you to another view if the other view also reported data based on the same database field.
  • Fixed a bug where choosing "only bottom-level items" would not show all bottom-level items-- it would only show a few.
  • Fixed a bug where the page footer (info bar, etc.) was not appearing when a "Data not Available" error occurred.
  • Fixed a bug where some of the bars were not clickable in the date/time graph view.
  • Fixed a bug where entering an invalid pathname as the Temporary Directory in CGI mode could cause Sawmill to freeze.
  • [by request] Fixed a bug where the Search Documentation feature would generate a server error in CGI mode.
  • [by request] Fixed a bug where clicking on the "Click this icon to create a new ..." icon would not create a new anything.
  • Fixed a bug where command-line filters which used item names rather than numbers (ex. -filter nofilter,nofilter,.com) did not work properly.
  • [by request] Fixed a bug where updating a database would double the number of hits on all old data!

New features in 6.0:

  • [by request] Added four new show/hide items, to show or hide the domain descriptions next to hostnames, the weekdays next to dates, the extra data in an overview view, and the views sidebar.
  • [by request] Changed what used to be the "top years, months, and days" view to "Top days," and turned on "only bottom-level items" so it shows only days, and changed it to order by hits instead of chronologically, so it really is showing the "top" hits.
  • Added a new view called "Top pages" which shows the top pages (not directories)-- this is the same as "Top pages and directories," except that it has "only bottom level items" shown.
  • Renamed "top ten lists" to "Single-page summary" to make it a little more descriptive, and to try to differentiate it from all the other "top" views.
  • Changed the default number of rows in most views from 10 to 20, but left the default number of rows for the subviews of the single-page summary view (formerly top ten lists) at 10.
  • Added a "hits graph equation" item to the Show/Hide menus, to turn the hits equation graph and text on or off in the date/time graph view.
  • Turned off truncation and auto-multi-lining of long pathnames in the "paths through the site" view, because it looked ugly. Pathnames will be truncated anyway by the edge of the window, without any harm.

Version 6.0fc8, shipped March 17, 2001

Bugs fixed in version 6.0fc8:

  • [by request] Fixed a bug where clicking a link in the statistics page generated by Quick Start would rebuild the database.
  • [by request] Fixed a significant design flaw in the field/filter/etc editor, where it was not immediately obvious how to create a new field/filter/etc. There is now an item at the bottom of the list which reads "Click this icon to create a new {whatever}."
  • [by request] Fixed a bug which could cause crashes in some of the session-based views (and the top ten view) in some circumstances.
  • [by request] Fixed a bug where viewing the "Top days of the week (computed)" or "Top days of the week, average (computed)" views caused in internal error.
  • [by request] Fixed a bug in the SPARC Solaris 8 version where any attempt to access the Sawmill server generated a "unable to create thread" error.
  • Fixed a bug where top-level domain names (ex. .com) were not clickable in the "top hosts and domains" table.
  • [by request] Fixed a bug where .net addresses were identified as "Niger."
  • [by request] Fixed a bug where the more/fewer/all controls still weren't showing up in the "paths through the site" view, in spite of two previous attempts to fix that.
  • Fixed a bug where the view menu was appearing even when it was supposed to be hidden.

New features in 6.0fc8:

  • [by request] Added RealServer log format.
  • Greatly improved memory management in certain hierarchical operations, including those used to generate the calendar and the session-based views. This greatly speeds the generation of those views (especially the session ones), and also eliminates some memory management problems which were causing instability, especially on MacOS and Windows.

Version 6.0fc7, shipped March 12, 2001

Bugs fixed in version 6.0fc7:

  • Fixed a bug where command-line language_module_pathname options were being ignored.
  • [by request] Fixed a bug where updating the database could zero out some existing data.
  • Fixed a bug where if a new configuration was created with the same name as an existing one, and if the old database took so long to delete that a progress page appeared, then the progress pages would reload forever.
  • [by request] Fixed a bug where if there were subdirectories in SawmillInfo/Configs, Sawmill could crash while generating the list of available configuration.
  • [by request] Fixed a bug where Sawmill would hang if the specified directory on an FTP site was empty.
  • [by request] Eliminated with top line of the progress page, which was frequently wrong anyway.
  • [by request] Fixed a bug where the "Open Configuration," "Delete Configuration," etc. pages did not work in CGI mode (they caused server error message).
  • [by request] Fixed a bug where if a log format had a date field an a time field but no date/time field, selecting day-by-day structure would cause in internal error.

New features in 6.0fc7:

  • [by request] Add a Show/Hide option (hidden by default) called "parenthesized items." When the option is hidden, parenthesized items in tables are hidden, and do not appear in totals or pie charts. This improves the information presented in tables like the "search phrases," where previously the top item, dwarfing all others, was usually "(no search phrase)". That item is now omitted, so the real search phrases are more easily seen and compared in the bar charts and pie charts.
  • [by request] Added a default filter to web log formats, which converts internal referrers (referring from your own site) to "(internal referrer)". That works together with the new "parenthesized item" feature to omit internal referrers from the referrers table. However, Sawmill cannot figure out which referrers are internal without knowing the domain name, so you need to manually edit the filter for it to work properly.
  • Optimized "top ten" view by caching the session, and not recomputing it for every subview.
  • [by request] Added links to the online Samples to a few relevant places in the documentation.
  • [by request] Dimmed the more/fewer/all link text when it's inactive.
  • [by request] Improved the appearance of the views sidebar by making the view links buttonlike, and making the colors a bit milder.
  • Improved the display for the "entry pages," "exit pages," etc. tables to use the dim gray "(default page)" label rather than including a black {default} at the end of the page name.
  • Added trial-version licensing.
  • [by request] Improved Log Filter documentation; added some examples.
  • Improved progress page during multi-subview view generated (like Top Ten) to report which subview it's generating. Previously it reported how far along it was on the current subview, which was almost worthless.
  • [by request] Improved detection of operating systems, by including better reporting of Windows 2000 (which was previously reported Windows NT 5.0), and a variety of fringe operating systems.
  • [by request] Improved the consolidation step of database building, by adding better progress and optimizing some I/O operations to make it faster, especially on MacOS.
  • Improved the output of Show Matches when there are no matches.
  • Improved the "if you see the red arrow then your directory is set up right" page to use a larger Sawmill logo instead.

Version 6.0fc6, shipped March 06, 2001

Bugs fixed in version 6.0fc6:

  • [by request] Fixed a bug where Sawmill was not handling a value of "default" for cross-reference groups properly in v5 configuration files.
  • [by request] Updated the Options section of the language modules and documentation. Went through every option and modified it as necessary to bring it up to date with current version. In many cases (several dozen), the option was obsolete and was removed entirely. In several other cases (most notably view_options), the documentation was brought up to date with the current features.

  • [by request] Updated the Documentation section of the language modules. Went through every option and modified it as necessary to bring it up to date with current version.
  • [by request] Fixed a couple bugs in the Generate HTML Files option which could cause incorrect output or crashes.
  • [by request] Searched through the language modules looking for all obsolete variables; removed them to streamline compact the language modules.
  • [by request] Fixed a bug where if you create a configuration, then built the database, and then created the configuration again (with the same name), the database would still be there, and could cause problems if the new configuration had a different structure than the old one. Sawmill now deletes the database, if any, when a new configuration is created.
  • [by request] Fixed a bug with the administrator password. If you entered the wrong password, and then clicked the "Save my password" checkbox, OR if you reset the password to something else after having saved it, then Sawmill would immediately generate a password error whenever you attempted to access it, without even prompting you to enter it. This has been fixed by eliminating the password error page for administrator login; instead, an incorrect password takes you back to the password entry page, with an error message at the top, so you can try again.

New features in 6.0fc6:

  • [by request] Added support for tcpdump log format.
  • Replaced every case of hard-coded English in the source code with a language module variable. Hard-coded English makes its way into the code during betas as features are added and tested on a trial basis; at the end of each beta test, the surviving new features' text gets folded into the language module. So almost all of the v5-era text was already language-moduled, but much of the new text (ex. the new editors, new view names, new error message, and much more) was now. As of this version, *all* known cases of English text have been moved to the language module, so it is now possible to create foreign language versions of Sawmill v6. However, because this involved hundreds of small changes, it is possible that one or more of the changes was done incorrectly, resulting in incorrect text or even program logic errors. Please be on the lookout for any unusual errors or wordings in the text displayed by Sawmill.
  • [by request] Added the "visitor id" field to the database fields list. Previous versions omitted the field because it's an internal field that cannot be edited or deleted, but that made it difficult to count fields when trying to build command-line filters. The field is now visible in the list, but it is uneditable and includes a parenthetical note describing what it is.

Version 6.0fc5, shipped February 23, 2001

Bugs fixed in version 6.0fc5:

  • Fixed a bug where duplicating a database field could result in a crash.
  • Improved memory management during item optimization; with some datasets it could take hundreds of megabytes just to display one view; now it takes just a few Meg.
  • Fixed a bug which could theoretically cause crashes whenever a database was accessed.
  • Improves interactivity of MacOS version while generating views for deep-hierarchy fields (previous versions would "freeze" until the view was generated).
  • Fixed a bug where the "nnn more items" rows of computed tables would sometimes incorrectly show a range of values.
  • Fixed a bug where the "visitor id" database field values were random. Usually this was just confusing (for people trying to read configuration files), but sometimes it could also generate illegal value errors.
  • Fixed a bug where expiring the database would result in incorrect visitor totals.
  • Fixed a bug where Sawmill could garble a progress page, or even crash, while generating progress pages for a long operation in web server mode.
  • [by request] Fixed a bug in the MacOS 9 version where Sawmill used a lot more memory than it needed to (sometimes hundreds of megabytes).
  • Fixed a bug where the filter summary at the top of pages was not correct for emailed views.
  • [by request] Fixed a bug where the "filter checkboxes and menu" statistics page option was not working correctly.
  • [by request] Fixed the instructions, which were not appearing at all.
  • [by request] Fixed a bug where "singlelimit" was not working properly for non-numerical values.
  • [by request] Fixed a bug where "recentdays" was not working properly.
  • [by request] Fixed a bug where you would get an error if you tried to process all of the log files in a directory which had subdirectories, and "process subdirectories" was off.
  • [by request] Fixed a bug where the top node of the "paths through the site" view could not be collapsed.
  • Fixed a bug where the compute search terms view data was not being computed properly (the table was empty).
  • Fixed a bug where expiring did not work properly on databases with visitor information.
  • Fixed a bug where the label on compute search terms column was "date/time."
  • Fixed a bug where the "paths through the site" view would sometimes have extra line in it, extending down below the last node of a branch.
  • Fixed a bug where the "paths through the site" view contained items in the Show/Hide menu which didn't make sense, and included a Sort menu even though there was nothing to sort. Also eliminated the "tool bar" item from the Show/Hide menu, since the toolbar doesn't exist anymore.
  • Fixed bug where custom language modules were ignored and overwritten by the English module.
  • [by request] Fixed a bug where Sawmill could crash if you built a database, then updated it, then expired it, and then updated it again.
  • Fixed a bug where if there was no data in the database, the Overview would give nonsensical results.
  • Fixed a bug where a database expiration followed by an update could result in an incorrect hierarchy for some fields.
  • [by request] Fixed a bug where if Update Database skipped over previously-seen data, it would mess up the existing progress display.
  • [by request] Fixed a bug where using the Browse button in the Configuration Options section of any configuration would take you to the Preferences at the end, and corrupt the default configuration.

New features in 6.0fc5:

  • [by request] Added two digits of precision after the decimal point in the bandwidth display.
  • Added Google to the list of recognized search engines.
  • Added per-view filters in addition to the per-subview filters that already existed (but didn't work), and added an option at the subview level to use the view's filters, and at the view level to use the page's filters. All views are subviews are created using that option by default. That means that it is now possible to edit the filters for any page, even the Top Ten views or a custom view, and have the page filters apply to the data in the view (previously, multi-subview views always had their own filters, and ignored the page filters). But you can also override the page filters at either the view or the subview level, if desired.
  • [by request] Added "more info" links to the names of subviews in the statistics, pointing to the view corresponding to that subview, if any. For instance, the "top ten pages and directories" subview now has a "more info" link pointing to the "Top pages and directories" view, where more detail and finer controls are available.
  • [by request] Added a "View" sidebar to the statistics pages. The sidebar replaces the Views menu, and contains the names of all the views. If desired, the View menu can be restored using an option, and/or the sidebar can be hidden.
  • [by request] Added a new date format: mmdd.
  • [by request] Added support for WS_FTP log format.

Version 6.0fc4, shipped February 10, 2001

Bugs fixed in version 6.0fc4:

  • [by request] Changed the Windows and Macintosh icon to the Sawmill 6 logo (it was still the Sawmill 5 logo).
  • Fixed a bug where certain views (ex. the weekdays or weekday averages views) could cause a crash when viewed.
  • [by request] Fixed a bug where the database consolidating step was extremely slow.
  • [by request] Fixed a bug where data was only being stored in every fourth database segment, resulting in inefficient use of the database storage.
  • [by request] Fixed the progress page to show progress during the database consolidation stage.
  • [by request] Fixed a bug where the Expire Database button image was not drawing correctly.
  • [by request] Fixed a bug where Expire Database was not correctly eliminating subitems in a page-views-only database.
  • [by request] Changed the name of the button on the password page from Finish to Next.
  • Fixed a bug where you could get an error "Unable to write file" while Sawmill was checking the log file format.
  • [by request] Fixed several bugs in FTP Log Source support.
  • [by request] Fixed a bug where Browsing in the Preferences would return you to the wrong page after you finished browsing.
  • [by request] Fixed a bug where viewing the Database Info page could cause a crash.
  • Fixed a bug where the progress page for Scan Log Source was incorrectly reporting that it was adding data to the database.
  • Fixed a bug where the Scan Log Source result did not properly report the number of accepted log entries.
  • Fixed a bug where the Scan Log Source result did not properly report the logs filenames for the current log source.
  • [by request] Fixed a bug where the database date range was not being tracked correctly.

New features in 6.0fc4:

  • Improved the command-line filter options a bit, to allow for partial lists (if there aren't enough comma-separate items, they are assumed to be unfiltered), and also added some more user-friendly versions of the special keywords, including nofilters to have no filters on any field (was allroots), and nofilter to have no filters on a particular field (was root).
  • [by request] Added backward-compatibility with v5 configurations and databases. That means that you can now move v5 configurations to the Configs directory, and move the databases to the Databases directory, and Sawmill will be able to use them as though they were v6 databases. Because of the significant differences between v5 and v6, some features (like visitor information from v5 databases) are not available in v6, some of the layout options are ignored, and a default set of views/subviews are created. Once the configuration has been imported, however, you can use it like a v6 configuration, modifying views and other options as you wish.
  • Improved progress during database builts to track the progress of the index generation, which usually is fast enough not to require progress, but can take quite a while on certain types of databases and datasets.
  • Improved progress during database expiration to track the progress of the "deleting unused subitems" step, which can take a long time if the indices are very complex.
  • [by request] Added a new date format: d/mm/yy.
  • [by request] Added a new time format: h:mm:ss.
  • Improved the expire_database_data option to allow absolute expiration dates (ex. 12/Jan/2001:00:00:00) in addition to days-past expiration dates.

Version 6.0fc3, shipped January 27, 2001

Bugs fixed in version 6.0fc3:

  • Fixed several bugs where Sawmill was not working properly in CGI mode.
  • [by request] Fixed a bug where Expiring the database was not working properly.
  • Fixed a bug where the All Options table contained some unevaluated language module variables (ex. DIRECTORY_WORD).
  • Fixed a bug where the .GIF, .JPEG, etc. filters were case sensitive, so they did not correctly detect all cases.
  • [by request] Fixed a bug where the progress display (time elapsed) was not working properly in multithreaded log reading mode.
  • Fixed a bug where the progress display was incorrect if some of the log data was gzipped.
  • [by request] Fixed a bug where "Update Database" would sometimes re-read previously seen data.
  • Moved the "previously seen data" tracking information from a configuration option in the config file of a database into a separate "seendata" file in the database. This eliminates problems that could occur when there was a great deal of previously seen data in a large number of files.
  • Fixed a bug (introduced in fc1 or fc2) where Sawmill was converting domain suffices (ex. .nl) to names (ex. Netherlands) very, very wrongly.
  • Fixed a bug where the Overview was reporting the hits, rather than the page views, in the "page views" entry,
  • [by request] Fixed a bug where generating the date/time graph (or using Generate HTML Files) would cause an error or a freeze.
  • [by request] Fixed an error where the label for "search phrases" in the overview was mistakenly "Types of web browsers."
  • Fixed a bug where Sawmill was not tracking session information properly.

New features in 6.0fc3:

  • Changed the logos to Sawmill 6 logos in the MacOS and Windows versions of Sawmill.
  • [by request] Added an option to omit the Sawmill credit (ad?) from the Information Banner.
  • [by request] Added Microsoft Media Server as a directly supported log format. Changed W3C log format to IIS4 log format, since both formats are W3C, but it's friendlier if Sawmill figures out which is which without asking (as it would have to if generic W3C were one of the options).
  • Eliminated the "omit interior nodes" option for database fields. The performance penalty for using that option was just too high to make it worth while, especially when there was cross-referencing involved.
  • [by request] Improved the command description pages in the documentation to be more easily readable by novice users.
  • [by request] Added a chapter to the documentation talking about creating a custom log format.
  • Improved the Overview to show not just page views and bandwidth, but also hits and visitors.
  • Improved log processing performance by bringing back superitem caching.
  • [by request] Fixed a bug where the "save password" option only worked until the end of the current session. Also, changed the text of the checkbox to make it clearer that it's not an autofill feature, but an autologin feature.
  • [by request] Improved the "delete log field" operation to make it smarter. The previous version added intelligence so you couldn't delete a log field if a database fields used it; this lets you do that *if* there is another log field by the same name (i.e. if the log field you are deleting is a duplicate).
  • Reworked the way Sawmill chooses the database structure. Previous versions always included visitor information by default, and always cross-referenced date/time to everything, even in Summary mode. It also always tracked hits, bandwidth, and page views by default. In this version, visitor information and date/time cross-references are off by default, as are hits and bandwidth. You can turn any of these on in the Structure page of the Create Configuration interview, but they're off by default, so the default database is very simple (especially in Summary, where is has no cross-references and tracks only page views). This makes the default database much smaller and faster to build.

Version 6.0fc2, shipped January 13, 2001

Bugs fixed in version 6.0fc2:

  • Fixed a bug where clicking on a + or - in the paths-through-the-site view would take you to the calendar, rather than expanding/collapsing.
  • Fixed a bug where Sawmill could chew up all available memory if a path or URL field in a log entry contained a $ followed by a number.
  • [by request] Fixed a bug where processing or browsing some large log files would cause an internal error ("temp sstring pool too small").
  • Fixed a bug in Squid Log Format where it would reject some lines.
  • [by request] Fixed a bug where the Windows version of Sawmill would sometimes not compute the list of available IP addresses correctly.
  • [by request] Fixed a bug where clicking Cancel during a database build would crash Sawmill.
  • [by request] Fixed a bug where the session-related subviews would still be added to the top ten list view even when session information was not available.
  • [by request] Fixed a bug where the Log Source menu in the default config didn't work.
  • [by request] Fixed a problem where when configurations were displayed in an HTML menu (for instance, on Open or Delete), any errors found would be dumped to stdout. This caused ugly error message on console in UNIX, and caused big console windows to appear on Mac.
  • Fixed a bug where the "Pattern is regular expression" checkbox appeared in all log source pages, not just the ones where it made sense.
  • [by request] Fixed a bug where with some browsers on some operating systems (for example, IE 4.5 on MacOS), clicking the tabs in the default configuration would go straight to the Admin menu.
  • Fixed a bug where the Browse buttons did not work properly in some cases.

New features in 6.0fc2:

  • [by request] Added support for a new variant of Squid Log Format.
  • [by request] Improved Squid Log Format support to track the hostname field.
  • Extended the Raptor Log Format to autodetect logs created by the Raptor Security Gateway.
  • [by request] Added a new date format "mmm dd yyyy".
  • [by request] Greatly expanded the scope of the Cisco PIX log formats. Added a new one ("Cisco PIX, Cisco Syslogd Log Format"), and also improved the existing ones to track all common types of log entries, not just "Accessed URL" entries.
  • [by request] Added another log format, "Raptor Log Format (Alternate)" to handle a variant of the Raptor log format Sawmill already supports.

Version 6.0fc1, shipped January 07, 2001

Bugs fixed in version 6.0fc1:

  • Fixed a bug where clicking a tab in the configuration options would take you back to the configuration page. This only happened under certain circumstances (maybe only under MacOS?).
  • Fixed a bug where if you deleted all of the filters or fields, you could never create one again.
  • Fixed bug where the date graph labels at the bottom were being drawn in the wrong place.
  • [by request] Added support for the Common Log Format files created by WebSTAR.
  • Fixed temporary file cleanup, which wasn't working in some cases.
  • Fixed a bug where adding or deleting a log field would mess up the cross-references.
  • Fixed a bug where deleting a log field which was used by a database field could cause problems; Sawmill no longer lets you do that.
  • Fixed a bug where deleting a database field could screw up the cross-references.
  • Fixed a bug where deleting a log field which was used by a filter could cause problems; Sawmill no longer lets you do that.
  • Fixed the icons for the page view to show directory and file icons appropriately.
  • Fixed a bug where search engines and search terms were not being properly detected.
  • Fixed IPTraf log format, which was not parsing log lines properly.
  • Fixed a bug which could cause crashes in web server mode, on the second click or later while viewing statistics.
  • [by request] Fixed a bug where saving the admin password with a cookie did not work properly in some cases.
  • [by request] Fixed a bug where the View Documentation button was broken.
  • [by request] Fixed a collection of typos, caused by a global-search-and-replace gone bad, where every occurrence of "statistics." was replaced with "statistits.cs.".
  • [by request] Fixed a bug in the Windows version of Sawmill, where subdirectories starting with the letter 'n' were not handled correctly.

New features in 6.0fc1:

  • Switched the Scheduler over to the new editor (the one already used by filters and fields).
  • [by request] Add a new configuration mode "send_view_by_email" and associated options. This mode can be used from the command line, from the scheduler, or directly by clicking the Send View By Email button on the configuration page. The mode sends a specified statistics view by email, using HTML formatting and embedded images to make the email look exactly like a normal Sawmill statistics page. You can specify the SMTP server, the destination email address, the return address, and the view to send. You can also set the running_sawmill_url option to point to an installed and running copy of Sawmill, so that links in the email message will work by opening a browser window with the appropriate data (by default, the URL is blank, and the email does not contain links).
  • [by request] Add an option to automatically send email to a specified address (using a specified SMTP server) whenever an action completes (ex. when a database finishes building).
  • [by request] Made the lifespan of temporary files an option (temporary_files_lifespan), so you can customize how long they last, or all temporary files immediately if you want.
  • [by request] Added a link to Troubleshooting to every error message, and also a link to sawmill@flowerfire.com.
  • [by request] Disabled the feature that scans ports until it finds one that works. Sawmill will now start ONLY on the specified port. This prevents it from starting successfully on an unexpected port; it will fail instead now, with an error.
  • [by request] Improved web server mode in UNIX versions so that Sawmill starts a second copy of itself to run the server, and restarts that copy if it terminates. This provides some of the robustness of CGI mode by ensuring that the server is always running, even if Sawmill crashes, runs out of memory, or terminates unexpectedly for any other reason.
  • [by request] Improved the hostname hierarchy so IP numbers are grouped left-to-right (ex. 23. > 23.45. > 23.45.67. > 23.45.67.89), while other hostnames are grouped right-to-left (ex. .com > .flowerfire.com > www.flowerfire.com).
  • Added a "default page" tag after the / in default page items in tables (ex. the / page in a normal web site), so make it clearer that the item does not refer to the / directory, but to the / *page*.
  • [by request] Changed the SawmillPassword file permissions to 644 by default.
  • [by request] Added "Check All" and "Uncheck All" buttons to the field limits editor.
  • [by request] Updated the documentation for Statistics Pages to match version 6.
  • [by request] Added chapter numbers to the documentation.
  • Improved/restored the "search terms" subview, which had been "left behind" by the view/subview improvements. The "search terms" view is now based on the "search phrases" field, rather than the referrer field, which is generally a good thing, since the referrer field is often not of sufficient depth to include the search terms, but the "search phrases" view always has them. The subview is also now integrated into the new statistics "look-and-feel," rather than having an old-style table look.
  • [by request] Added .png and .css filters to the default set for formats that had .gif, .jpg, and .jpeg filters.
  • [by request] Improved the Delete Configuration menu option so it also deletes the associated database.
  • [by request] Added support for UMC pathnames in Windows (etc. \\hostname\share\dir\etc).

Version 6.0b16, shipped December 11, 2000

Bugs fixed in version 6.0b16:

  • [by request] Fixed a bug where Generate HTML Files would crash or generate an Internal Error.
  • [by request] Fixed a bug where changing the sort order could cause a crash.
  • Fixed a bug where clicking (edit) to edit the limits, and then editing a limit, and then returning to the statistics, would lose the active statistics view.
  • Fixed a bug which could causes divide-by-zero exceptions when displaying a view where there was one item in the table and the Adjusted Averages row was turned on.
  • [by request] Fixed a bug which occurred on some Sawmill platforms, where the date/time graph equation was incorrect.
  • [by request] Fixed a bug in the Wand Audit format where the date/time information was not being tracked correctly, resulting in crashes.
  • [by request] Fixed a bug which occurred on some Sawmill platforms, where the date/time graph equation was incorrect.
  • [by request] Fixed a bug where Sawmill would crash if the statistics were viewed and there was no date/time field in the database.
  • [by request] Fixed a set of large memory leaks. This version probably isn't leak-free, but it's certainly better.
  • Fixed a bug where increasing or decreasing the number of rows in the table could cause Sawmill to crash.
  • [by request] Fixed a bug where if the database was larger than 4 Gig, the progress page memory indicator would "wrap" around to 0.
  • Fixed a bug which could cause the sessions page (or the initial top-ten page) to crash.
  • [by request] Fixed a bug where the Add Log Data option was not working.
  • Fixed a bug where the "Process subdirectories" option was sometimes not a checkbox, and sometimes appeared twice.
  • Fixed a bug where Update Database was crashing.
  • [by request] Fixed a bug where the "page views" for the "other items" row was always 0.
  • Fixed a security flaw where you could use ?menu to get to the Limited menu, and then use Back To Admin Menu to get to the admin menu, without ever entering a password.
  • [by request] Fixed a bug where the Browse buttons were not working in the Default Configuration editor.
  • [by request] Fixed a bug where the Log Format menu in the Options contained the wrong format, and even viewing it manged the configuration file.

New features in 6.0b16:

  • [by request] Improved the host hierarchy so IP numbers are structured left-to-right (ex. 206.221.233.20 < 206.221.233. < 206.221. < 206. < root), but resolved hostnames are still structured right-to-left (ex. www.flowerfire.com < .flowerfire.com < .com < root). This makes a lot more sense than the old way, where everything was right-to-left, even IP numbers.
  • [by request] Made log field values optionally case-insensitive. When this is turned on, all items which differ only by case are grouped into a single item (ex. hits on index.html, Index.html, and INDEX.HTML all count as hits on the same item). Most fields are like this by default now. In those rare cases where case-sensitivity is required (ex. certain UNIX sites where differently-cased versions of filenames are used), this can be turned on or off on a field-by-field basis. This new feature also eliminates the need for the default "convert to lowercase" filters, so those have been removed.
  • [by request] Added support for SiteCAM Log Format.
  • [by request] Added support for ShareWayIP Log Format.
  • [by request] Improved NetPresenz log format support; fixed some bugs and added support for two variants: one with 24-hour times and European dates, and another with 12-hour times and American dates.
  • [by request] Added support for Serv-U FTP Log Format.
  • [by request] Added a checkbox next to the admin password to let Sawmill save your password as a cookie. This fixes an oft-complained-about inconvenience where you have to re-enter the administrative password when you come back from the statistics. There did not seem to be a secure way to carry the administrative password through a statistics-browsing session, as the original plan had been, but this should solve the same problem in most cases, and has the added benefit of freeing you eternally from the password (not just eliminating the stats-to-menu password; this also eliminates the anything-to-menu password). On the downside, this is less secure, since anyone using the same workstation (or same instance of the browser) can get to the admin menu. Like many security issues, it's a convenience tradeoff, and it's up to the user to decide.
  • [by request] Improved the text of the "Filters" bar at the top of the statistics page, to make it more "English."
  • [by request] Improved the text of the view and subview names, to make them more "English."
  • [by request] Multithreaded the log reading process. This is a major new feature which required massive modifications throughout the Sawmill source code. Log processing is now broken down into stages: splitting the log data into lines, parsing the lines into fields, filtering the fields, adding the accepted entries to the database, and consolidating the database segments. When multithread_log_reading is on, all stages run simultaneously, potentially with multiple threads of execution per stage. This results in a 50%+ performance increase on a two-processor system; results for larger numbers of processors are not yet in, but will presumably be even better. Performance is generally worse with this option on a single-processor system, so the option is off by default.

    One situation where single-processor systems benefit is if they are doing DNS lookup. Each log filtering thread can potentially look up an IP address at the same time, resulting in significant speed improvements when using large numbers of threads (DNS lookups often stall, leaving Sawmill doing nothing for very long periods; in multithreaded mode, it will continue to work during stalls). Preliminary results show a speedup of 70% when processing a DNS lookup configuration on a single-processor system.

    This feature is still "experimental"; due to the massive modification that went into it, and the inherent difficulty multithreaded programming. At this point, you can expect more crashes and freezes if you use this option. Later versions will become more stable as we test and debug it more, and it should be a "final" feature by the time version 6 ships.

  • Improved database buffering. Sawmill now uses a different technique for database buffering, reducing memory usage and disk usage, and improving speed. The "disk write buffer" has been eliminated (Sawmill still buffers to disk, but the final database is larger than the buffer, so there is no need to limit the size of the disk buffer), and the memory write buffer is more efficient.
  • [by request] Improved error reporting of incorrectly-defined filters.
  • [by request] Improved WUFTP log support to track the "completed" field.
  • [by request] Restored and improved the "overview" view. The overview shows an "executive overview," including information on total hits, bandwidth, page views, visitors, days of data, and averages. A link shows a few more "computed" statistic. This is now the view that appears second, after clicking Calendar.
  • [by request] Added a "calendar" view as the first thing you see when you get to the statistics. You can use that to easily zoom in on any year, month, week, or day. All database fields are cross-referenced to date/time by default, to allow for this behavior.
  • [by request] Improved installation documentation.
  • [by request] Added IPTraf log format.
  • [by request] Added CiscoPIX/Kiwi log format.
  • [by request] Added Zeus Extended log format.

Version 7.0b15, shipped July 18, 2004

Bugs fixed in version 7.0b15:

Version 6.0b15, shipped November 01, 2000

Bugs fixed in version 6.0b15:

  • [by request] Fixed a bug where Update Database would sometimes add log data to the database multiple times, instead of skipping over previously-seen data.
  • Fixed a bug where one file of the database was sometimes not properly written and closed, so updating the database later could result in an error, "segment not long enough".
  • [by request] Fixed a bug where in Windows CGI mode, the Browse feature was not working properly when browsing pathnames containing spaces.
  • [by request] Added back support for Netscape log format, which got lost between 6.0b13 and 6.0b14.
  • Fixed a bug where processing a log containing more than 4 Gig of bandwidth could cause an error "the database header is corrupt."
  • [by request] Fixed a major problem with the field and filter editors. For debugging purposes, editing was "temporarily" turned off in those editors, but it was accidentally left off for the final 6.0b14. The result was that none of the editing controls worked. This has been fixed.
  • [by request] Fixed a bug in the filters where "if A then B else C" filters were getting mangled in the filter editor. They worked fine, but they were not displaying correctly.
  • Fixed a small bug in the shading of pie charts which made it somewhat harder to see the size of some slices.

New features in 6.0b15:

  • [by request] Added preliminary support for GNATBox logs.

Version 6.0b14, shipped October 30, 2000

Bugs fixed in version 6.0b14:

  • [by request] Fixed a bug where specifying the full pathname of SawmillInfo directory in SawmillInfoDirLoc could cause Sawmill to generate security errors.
  • [by request] Fixed bug where choosing a different type of log source during Create Configuration would cause an error.

New features in 6.0b14:

  • Changed all of the "list-style" options (ex. database/log fields, filters, etc.) to use the same layout in the Options editor (the layout introduced with Views).
  • As an added perk to the new log format approach, every log format includes a regular expression log_format_autodetect_regular_expression which is used to auto-detect the log format. If any of the first five lines of the log data match the regular expression, that format is used. This means that all log formats, including custom formats, are auto-detected (previously, only CommonAccess, ApacheExtended, CommonReferrer, Netscape, IIS, and WebSTAR were auto-detected).
  • [by request] Reworked the way custom log formats work. All log formats (including the built-in ones) are now specialized configuration files residing in the LogFormats subdirectory of SawmillInfo. Log format configuration files include the format name, date/time format options, special format options (like treat_square_brackets_as_quotes), log fields, database fields, cross-referencing information, log parsing filters (if necessary), and log filters. This makes it much easier to define a custom log format (you can copy an existing format file and modify it with a text editor), and custom formats show up in the menu along with built-in ones.
  • [by request] Changed the syntax of the database_field_info in configuration files. All database fields are now described in a single large configuration option. Did the same thing with log_field_info, log_filters, cross_reference_groups, and log_parsing_filters. This eliminates the field numbers, filters numbers, and some problems associates with them (a feature requested long ago), and also opens the door for better custom formats (see below).
  • [by request] Changed the way the USER_AGENT (browser) information is tracked and reported. The "browser" field (which does not generally appear in the statistics, but is used to derive other fields) has been renamed "agent," the "browser OS" derived field has been renamed "operating system," and the "browser type" and "browser version" fields have been combined into a single "web browser" field which works in a more reasonable way-- zooming in on "Netscape" shows you which versions of Netscape they used, etc.
  • Changed the interface for the Database structure options. They are split into three categories now: Database Fields, Database Cross-Referencing, and Database Structure. The Database Fields editor is now like the View/Subview editor, including icons and a more compact interface.
  • [by request] Changed the configuration file format of the database_field_info option. Instead of having one database_field_info option for each field, there is now a single database_field_info option which describes all the database fields. As a side effect of this, database fields are no longer numbered, so it's easier to add a field to the configuration by hand; you don't have to renumber the rest of the fields.

Version 7.0b14, shipped July 17, 2004

Bugs fixed in version 7.0b14:

Version 6.0b12, shipped September 13, 2000

Bugs fixed in version 6.0b12:

  • [by request] Fixed a bug in the MacOS version where Sawmill would repeatedly attempt to open the SawmillAccessLog file, and would generate an error when the later attempts failed.
  • Fixed "total time spent on page" view, which was sometimes wrong, and sometimes crashed.
  • Fixed bug where the session summary view was counting hits wrong.
  • Fixed a bug where the date/time graph would not appear, or would cause a crash, if there was no hits information.
  • Fixed a bug where computed weekdays and hours views were not computing page views and visitors correctly, and were sometimes crashing.
  • Fixed expiration of license, so that licenses really do expire when they say they do.
  • [by request] Fixed the weird way web_server_port used to work. web_server_port no longer in any way controls whether a server gets started; that is controlled by the value of server_hostname. web_server_port just says what port to use. A server is started if server_hostname is non-empty, or if Sawmill is started from the command line with no options.
  • Fixed a bug where default configuration options and preferences were not being properly saved.

New features in 6.0b12:

  • [by request] Greatly simplified the main window for the Windows and MacOS versions, so it's just the logo, the Use Sawmill button, and an "Advanced options" checkbox. Clicking Advanced expands the window to show all the other info: the URL, IP address, threads,etc.
  • Reworked the stats_page, stats_page_by_number, get_hits, and get_hits_by_number options to be a little more reasonable. They were artifacts from older versions of Sawmill. They have been replaced with "filters", and option which accepts a list of filters, and active_statistics_view, an option which has been around a while, and specifies the current view. This makes some parts of Sawmill's internals cleaner, and makes the command-line interface a bit more intuitive.
  • [by request] Added view editing. Views are individual HTML pages of the statistics. By default, there is one view per database fields, ex. "page statistics," and then a bunch of special views, ex. "exit pages"). With previous versions, that was what you got, except for some control over omitting views and view options. With this version, views are set up the same as always by default, but can be edited to create new ones or to change their names. More importantly, views can contain multiple "subviews." Subviews roughly correspond to an old-style view; they have a type, (ex. "page statistics"), a name, a set of filters, and a set of view options. The upshot of all this is that you can create views containing arbitrary combinations of pie charts, graphs, and statistics tables from all parts of your statistics. For instance, you could define your first view (the first one you see) to show you a date/time graph at the top, then pie charts showing bandwidth usage and visitors to the entire site, then pie charts showing similar usage for one directory, then a table showing the page views from referrer Yahoo, etc. The possibilities are endless. Later versions of Sawmill will contain more elaborate view/subview templates, but for now the default looks just like it always has, with one subview per view.
  • Added several new types of view options: "totals row", "remainder row," and "more/fewer/all links" let you turn on or off the Total row in tables, the "123 more items" row, and/or the "more rows," "fewer rows," and "all rows" links. "only bottom-level items" works just like the "flatten" link used to, but can be controlled on a per-view basis. Bookmarking has never been very useful, and is even less so now that view templated can be edited, so that link has also been eliminated, eliminating the entire bottom toolbar (which contained the bookmark link and the views link).
  • [by request] Using the new view/subview infrastructure, added a new default view: "top ten lists." This view shows the top ten items of each field, and a few other subviews that lend themselves to top ten (ex. entry pages).
  • [by request] Greatly improved the message that appears if you try to view statistics when you don't have enough cross-referencing.
  • [by request] Rearranged the layout of statistics pages slightly: moved pie charts to the left and made them smaller, restructured the Overview page in two columns, and changed the normal number of rows in a table to 10. This allows all views to fit in a normal non-scrolling browser window.
  • [by request] Added an category of options to customize the font fact, size, color, and attributes of any text on the statistics pages. The defaults are somewhat different from the previous look (most notably, sans serif fonts are used heavily).
  • Disabled individual hits view by default, since it's so slow and of less value now that there is good session information available.

Version 6.0b11, shipped August 15, 2000

Bugs fixed in version 6.0b11:

  • [by request] Fixed a bug where non-English language modules were not being properly loaded.
  • [by request] Fixed a problem where corrupt date/times would result in lines labelled "midnight - midnight" in the hours of day table.
  • Fixed bug where the sort, show and hide menus did not work with some versions of Netscape.
  • Fixed a bug where the filter text sometimes showed the wrong field.
  • [by request] Fixed a bug in the processing of MSProxy log dates, which resulted in a "no data matches the limit" message in date/time views.
  • Fixed some minor cosmetic bugs in the display of date/time graph bars.
  • Fixed a bug where configuration files were written with the full English names of log file formats, rather than the short "command line" names. This can cause problems if the English names change, or if a different language is used.
  • [by request] Eliminated the "Fully load database segments" option. The option could be useful for very small databases, but with any real-world dataset, it ended up using so much memory that Sawmill ground to a halt or crashed.
  • [by request] Fixed a bug where the Update Database feature was not correctly recognizing when it had seen data before in some cases, and was re-adding it to the database, resulting in doubled numbers.

New features in 6.0b11:

  • Added licensing keying to Sawmill. License keys are semi-human-readable license strings like beta-2user-exp03152001-0000a-87be, which encode the version type (Beta, in this case), the number of users (2), the expiration date (3/15/2001), a unique license identifier (0000a). The rest is a checksum. This is useful because now a single Sawmill binary can act as both Trial and Full version, so upgrading from Trial to Full is just a matter of going to the Licensing page and entering the new license (no more installing Full and copying the data from Trial, or the executable from Full). This also provides auditors with a way to check for unlicensed versions, and provides a small amount of anti-piracy protection.
  • [by request] Added support for IAS log format.
  • Improved log parsing to support arbitrary log field dividers (instead of just whitespace, comma-space, or pipe).
  • [by request] Reworked the interface completely. Added "woodlike" graphical buttons on the main menu and the configuration page. Changed all title bars to be more "woodlike" in appearance, and somewhat 3D. Changed the interview button bar and the documentation toolbar to look similar.
  • Completely worked the "xref icons" feature. The feature was becoming unwieldy because of all the new log formats supposed, with all the new field types. It had become almost impossible to create a unique icon for each possible field type. Instead, Sawmill now includes a new feature where you can turn on a column of check boxes down the side of a table; you can click on any checkboxes you're interested in, and choose from a menu at the bottom whicview you want to see the selected items in. This serves the same purpose as the xref icons, and adds the additional benefit that multiple limit items can be applied to the same field simultaneously.
  • [by request] After getting half-a-dozen complains about the phrase "Field Limits" (some recommending that we use "Filters" instead), and seeing several customers call them "Filters" anyway, we decided to go with the flow, and changed the official name of the Field Limits to "Filters." That introduces a new problem, since now there are Log Filters and Filters, and they are not the same thing at all. Still, we hope this will be less confusing, overall.
  • [by request] Added support for Firewall-1 log format. Improved Sawmill's support for formats, like Firewall-1, where there is no date field, but there is a time field. Sawmill currently reports all hits as occurring on 01/Jan/2000 in that case.
  • [by request] Added a legend to the date/time graph view, to make it clear which graph is tracking what.
  • [by request] Enhanced data tracking to include "hits," "page views," "bandwidth," and "visitors." Previously, Sawmill tracked only "page views," "bandwidth," and "visitors," though "page views" could be configured to represent total hits by removeing the image filters. Sawmill now tracks the two separately; filters determine whether a hit is a page view or just a hit, and the two get tracked side-by-side in the database. New columns in the tables, a new pie chart, and new date/time graphs show the new information. As a side effect of this, the date/time graph code has been reworked, and no longer overlays multiple bar charts or cumulative graphs on the same area; this makes the graphs larger but eliminates the problem where the 3D perspective made it difficult to determine graph values of the deeper graphs. Sawmill now analyzes *every* line of the log by default, rather than quickly discarding image hits, so it's slower than before at reading log data. However, it's giving better information. The old speed can be restored by telling it not to track hits or bandwidth.
  • [by request] Along the same lines as the new 4-ply tracking feature, Sawmill now tracks bandwidth in a more reasonable manner. Previous versions tracked bandwidth only for files which were accepted by the filters, so bandwidth totals did not include the bandwidth resulting from image file hits (which is generally the larger part of the bandwidth). Sawmill now tracks bandwidth accurately; even if you don't ask for "hits" information, it still processes each log line and includes it in the bandwidth information, so directories and other items show true bandwidth numbers.
  • [by request] Added tracking of the earliest and latest log entry date/times in the database. They now appear in the info bar.
  • Split the info bar horizontally into functional cells, to improve aesthetics a bit.
  • [by request] Improved the "item name" column of tables to split the name across several lines (up to 4, customizable) if it exceeded the maximum number of characters for a single line.
  • [by request] Added an "average" row to the tables, showing the average of each column in the table (including hidden ones). Added an "adjusted average" row which currently shows the same as "average" but will eventually show the average with the top 5% removed.
  • [by request] Enhanced the IIS Log Format support to track server hostnames and server IP addresses.
  • Significantly reduced the disk space used by the visitor id indices (and the host indices, if it includes bottom-level items, which it no longer does by default). This is important because Sawmill keeps the entire index in memory while it's accessing a database; small indices mean a smaller memory footprint. With the new way of doing sessions, the visitor id tends to be easily the largest index; this cuts the size of the index in half or better, when all the hosts are IP numbers. When only some of them are IP numbers, the gain will be less.
  • [by request] Added a new filter type which runs a command-line command, gives the field names and values as command-line parameters, and stuffs the result into a specified field. This allows for the creation of more sophisticated filters than ever, using the programming language of your choice.
  • Improved Common Access Log Format and Apache Extended Log Format to treat their second field as a server domain field, and include it in the database (non-cross-referenced). In some cases, the field is actually the password field, but many servers put the server domain there, and it's very convenient to have access to it from Sawmill.
  • Restructure and reworked the entire administrative interface. The new interface is cleaner and more attractive, and addresses some problems with the old one (ex. it was not clean that you should click Menu to get back to the configuration page menu).
  • Changed the way session data is tracked. Session information is now tracked as the log is read, and is stored separately from the main part of the database. This has the huge advantage that the date/time field no longer has to be down to the level of seconds in order for session information to be available, so the date/time hierarchy is much smaller, as is the database itself; also, the date/time, page, and visitor_id fields no longer have to be cross-referenced to have session information. This eliminates the need for the Summary with Session Info option; session information is almost free in terms of processing power and disk space, so it is included in every database, even "Summary." Session information is roughly as fast as it was before. The main downside of the new way of doing session information is that the session views can no longer have field limits on any fields except the date/time and page fields.
  • [by request] Enhanced the stats_page option to add variants for applying a single limit easily, or for limiting the date/time field to the past few days.
  • [by request] Added a command_line_password option, useful for specifying a statistics viewing password in an URL
  • [by request] Added a generate_image option, which causes Sawmill to generate the date/time graph image in response to a view_statistics URL, rather than the HTML statistics page containing it. This is useful for embedding the graph in an HTML page.
  • [by request] Added JavaScript code to automatically select appropriate fields on various pages ( the password fields, and log pathname fields, and the configuration name field), to eliminate the need to select it manually by mouse or by tab.

Version 6.0b10, shipped July 14, 2000

Bugs fixed in version 6.0b10:

  • [by request] Fixed bug on Windows where a customized server hostname would not appear in the popup menu.
  • [by request] Fixed a security flaw, where any user with access to Sawmill by web browser could view the first part of the beginning of any file on the system where Sawmill was installed, as long as the Sawmill program itself had access to the file. This was particularly serious because in combination with the other security flaw fixed in this version, this could be used to determine the administrative password from any remote computer.
  • [by request] Fixed a serious security flaw, where anyone with access to the SawmillPassword file (i.e. access to files on the Sawmill machine) could figure out the administrative password with a bit of hacking. This has been corrected; the password can no longer be guessed from the contents of the SawmillPassword file.
  • [by request] Fixed a security flaw, where any user with access to Sawmill by web browser could view the first part of the beginning of any file on the system where Sawmill was installed, as long as the Sawmill program itself had access to the file. This was particularly serious because in combination with the other security flaw fixed in this version, this could be used to determine the administrative password from any remote computer.
  • [by request] Fixed a bug where Internet Explorer was bring mis-identified as Netscape in some IIS logs.
  • Fixed a bug where you could delete a log field without deleting the database field that depended on it, resulting in multiple strange "unknown"-type log fields. Sawmill now refuses to let you delete a log field until you have deleted its database field.
  • [by request] Fixed a bug which could cause crashes if two threads were attempting to read the same database at the same time.
  • [by request] Fixed a bug where Sawmill could crash if one of the first few log lines contained more than 100 spaces.
  • [by request] Fixed bug where the pie charts in some computed views had clickable slices (which they shouldn't have had), and clicking them caused a crash.
  • [by request] Fixed bug where the Overview section of the configuration page mis-reported the database disk usage.
  • [by request] Fixed several small errors in the documentation (mostly typos).
  • Fixed several bugs in the Generate HTML Files feature (it had broken pretty thoroughly with all the new statistics page stuff). Also changed the view menu to be a popup menu in offline mode (like it is in online mode).
  • [by request] Fixed the English filter descriptions, which were in a horrible state, having been left half-complete a few beta ago. The filters should now have reasonable English descriptions.
  • Removed the link to the administrative menu documentation from the User menu.
  • Fixed a bug where the instructions sometimes referred to the wrong field.

New features in 6.0b10:

  • Added a very powerful filter type which matches a field against a regular expression, and then sets another field to a specified value, where the value may contain variables like $1 and $2, which refer to parenthesized sections of the original regular expression match.
  • Added a new default filter which chops off everything in the page field after the ?, replacing it with "(parameters)". This reduces the database size, but you'll want to delete that filter if you need to see the parameters.
  • [by request] Eliminated password prompt in the case that Sawmill is running on localhost or 127.0.0.1 (the loopback interface). In that case, remote access is not possible, so password security is not necessary.
  • Significantly reduced the disk space used by the database subitems indices, which contain the structure of the data hierarchy. This is important because Sawmill keeps the entire index in memory while it's accessing a database; small indices mean a smaller memory footprint.
  • Compacted date/time items internally (they are now represented with five bytes, rather than 20 as they were before); this should result in a 3-4 time reduction in the size of the date/time items file (one of the two major pieces of the date/time index, which is generally the largest index of all the fields).
  • [by request] Added two new configuration options (reject_entries_before and reject_entries_after), which "bracket" the log data, so that hits outside the specified boundaries are automatically rejected during log reading.
  • [by request] Enhanced search engine support. There is now a file called SearchEngines in SawmillInfo (created there if it doesn't exist) which describes the supported search engines. It can be edited to support more. Sawmill now uses regular expressions to detect search engine URLS and parse out the parameters, allowing it to support a larger range of possible query formats. The "search phrases" view, the "search terms" view, and the new "search engines" view all use this new mechanism for determining the search engine and search terms information.
  • [by request] Added a new "search engine" log field, derived from the referrer. The field shows the names of the search engines which contributed hits.
  • [by request] Added a link to the Sawmill web site to every documention page.
  • [by request] Added "Expand All" and "Collapse All" buttons to the Field Limits editor.
  • Extended the Show/Hide menu feature of the statistics pages (and of the Stats View configuration page) to control *all* elements of a statistics page. You can now use that feature to independently turn on/off the instructions, the field limits bar, the view menu, the sort menu, the toolbar, the info bar. You can even use the Hide menu to turn off the Show/Hide menu itself, though you'll have to use your browser's Back button to get it back if you do that.
  • [by request] Improved detection of simultaneous build or build/reads of the database. Sawmill now complains if you attempt to build a database, or access a database, while another copy of Sawmill (or another thread in the current copy) is modifying (building, updating, or expiring) the database. Previous versions did not check that very well, which could cause crashes if two users tried to build the same database at the same time, or if someone tried to browse a database while it was being rebuilt. This feature is also somewhat intelligent, in that if Sawmill crashes or is terminated while building a database, the next access will notice that (if at least one minute has passed), and will take access from the crashed process.
  • [by request] Added a "Create Directory" button and field to the Browse page, so you can create directories directly from Sawmill while browsing.
  • [by request] Added support for BIND Query logs, BIND Security logs, and BIND Response-Checks logs.

Version 6.0b9, shipped June 08, 2000

Bugs fixed in version 6.0b9:

  • [by request] Fixed a bug where Sawmill would generate a strange error (integer expected for lfi) when it tried to process a log file containing a format= line with a quoted field is did not recognize.
  • Fixed a bug where Sawmill was not reporting the visitor percentage or totals properly (it was showing -'s even when the information was available).
  • [by request] Fixed support for the LISTSERV logs, which was broken (crashing, and computing dates wrong).
  • [by request] Fixed bug where drive icons were not appearing properly in Browse view.

New features in 6.0b9:

  • Reworked the way statistics page element visibility is handled. The new mechanism is more general, and makes it easier to add new views with different types of data in the future. The primary visible change is that the configuration interface for changing view element visibility is different now. This change also breaks compatibility with previous v6 configurations. If you have an older v6 configuration that you really need to import into this version, please send to to sawmill@flowerfire.com and we will convert it manually.
  • [by request] Added a new view "overview," which shows the total hits, bandwidth, and visitors, contains a bit of descriptive text, and includes links to all other views. The overview is now the first view users see.
  • [by request] Added a new view "time spent on each page," which shows the amount of time visitors spent on each page, at any time in any session. The view shows both the total time visitors spent on a page, and the average time.
  • [by request] Added support for WebSTAR Proxy format.
  • [by request] Added support for dates in the mm/dd/yyyy format.
  • [by request] Improved Sawmill's handling of certain types of corrupt log files. Sawmill used to generate an error and stop processing when it encountered large blocks of garbage in the middle of the log data; now it ignores the garbage and tries to keep going.

Version 6.0b8, shipped April 29, 2000

Bugs fixed in version 6.0b8:

  • Improved the individual hits view by removing visitor information (which was appearing as empty lines with file icons).
  • [by request] Fixed a bug in MacOS Browse feature where the top level items did not appear in the list.
  • Fixed a bug in Sendmail log support where the first few date/times read could be corrupted.
  • [by request] Fixed a bug where Sawmill would not properly compute the hour of day or day of week when processing some types of logs (including Sendmail).
  • [by request] Fixed a bug where the number of rows in "day of week statistics" and "hour of day statistics" views defaulted to 20, rather than 7 and 24.
  • [by request] Fixed a bug which could cause a crash when creating a configuration with Multithread Log Reading turned on.
  • [by request] Fixed a problem with Solaris 2.7 versions of Sawmill which caused then to crash at startup in some cases.
  • Fixed a bug where many of the Show/Hide statistics options were "Obsolete Option."
  • Fixed a bug where the bandwidth and visitor view elements were not available.
  • Fixed a bug where the Sort menu did not contain "chronological" and "reverse chronological" even when it should have.

New features in 6.0b8:

  • Improved view menu so the session-based views do not appear when session information is not available.
  • Improved the "entry points" and "exit points" views. They are now integrated better into the Show/Hide infrastructure, with all four "visitor" elements available to be shown or hidden. The Sort menu, which is useless in these views, is omitted (here and in all session-related views).
  • Improved Sawmill's default choice of database structure so it always includes visitor information in every xref group (even in Summary mode). This means that every view now includes visitor information (previously, only the page/datetime/host/referrer views showed visitor information).
  • [by request] Added an extra page to the Create Configuration and Quick Start interviews, asking for a broad description of the amount of detail desired in the database. Choosing "Summary" creates a database with no cross-referencing, which is therefore very small and fast. "Summary with Session Information" adds the minimal cross-references necessary to get session information (page/datetime/visitor). "Full Detail" gives "normal" cross-references. This lets novice users create slim configurations without having to delve into the cross-references. The default is "Summary" to ensure that the first attempt at statistics doesn't generate a multi-gigabyte databse which takes hours to create.
  • [by request] Enhanced the MacOS Browse feature to include desktop items in the top-level view.
  • [by request] Re-enabled Raptor proxy log support, which was disabled a while ago when it became difficult to support it with Sawmill's log processing features. Sawmill's log processing features have advanced tremendously since then, and supporting it with the new mechanism was fairly easy.

Version 6.0b7, shipped April 12, 2000

Bugs fixed in version 6.0b7:

  • Fixed a bug which would cause an Internal Error (xref_group.cpp:97) in all sorts of situations (generally, the second time you tried to generate a statistics page).
  • Fixed a bug where error windows were not being displayed correctly in some cases on Windows.
  • [by request] Fixed a problem with the x86/Solaris 2.7 version of Sawmill which caused it to crash at startup in some cases.

New features in 6.0b7:

  • [by request] Improved Browse feature so it shows desktop items in Windows.
  • [by request] Added an easier way to browse directly to a particular view of a particular configuration: use sawmill?view+configname+limit in the URL; ex. sawmill?view+myconfig+/dir1/ .
  • [by request] Improved Browse feature so it shows disk names in Windows.
  • [by request] Added preliminary support for LISTSERV logs.

Version 6.0b6, shipped April 07, 2000

Bugs fixed in version 6.0b6:

  • [by request] Fixed a bug where under some circumstances, Sawmill would generate an internal error (itemnums.cpp:415) when trying to generate a statistics page. This could particularly happen if unusual referrer limits were in place.
  • [by request] Fixed a bug where domain descriptions would appear as "unknown" if the hostname had any uppercase letters in its domain.
  • Fixed a bug where Sawmill would crash under certain circumstances when you clicked Create Configuration or Quick Start, or when Sawmill tried to display a progress page while rebuilding the database.
  • Fixed a bug where Sawmill would immediately generate an error if it was run as a CGI program in a read-only CGI directory.
  • Worked around a bug in Internet Explorer 5.5, where auto-reloading pages weren't auto-reloading, and uncached pages were being cached.
  • Improved auto-detection of Common Access Log Format and Apache Extended Log Format; Sawmill was not correctly identifying the format of certain valid logs of those format.
  • Fixed a bug where the blue triangle icons in the Edit Limits page were not being correctly created in some cases.
  • [by request] Fixed a bug where the "log filename" log field would always be created as field 1, even though it's derived, and derived fields are supposed to come after all other fields. Generally, that didn't hurt anything, but in some cases, especially when building a database from the command line or the scheduler, it would mangle the field hierarchies, resulting in incorrect Field Limits displays.
  • Fixed a bug where if the configuration name contained a space, clicking More Info on a progress page would give an error.
  • Improved the way English filter descriptions are generated, to make it simpler and more fool-proof. This also fixes an infinite loop that could happen while generating some very complicated regular-expression filter descriptions.
  • Fixed bug where "chronologically" and "reverse chronologically" sorts did not work for some views.
  • [by request] Improved support for Cisco Pix format, so it correctly processes a larger range of variations.

New features in 6.0b6:

  • Improved handling of SawmillInfo directory. Sawmill no longer needs the SawmillInfoDirLoc file to tell it where SawmillInfo is; it looks for it in its own directory, and in /etc/sawmill (on UNIX). SawmillInfoDirLoc is needed only if SawmillInfo is not in Sawmill's directory.
  • [by request] Changed the page Sawmill returns when it can't auto-detect the log format, to include a better description of what's going on, and to use ApacheExtended as the default, rather than Custom.
  • [by request] Changed Sawmill so it does not attempt to look up the hostname of the browsing computer for each access, on UNIX and Windows. This eliminates a common cause of slow page loads, and also eliminates the only reason Sawmill ever has for using the network under normal circumstances. So by default, Sawmill no longer requires that the network be up, so it won't dial up every time you try to use it if you have a dial-up connection.
  • [by request] Added a link to progress pages to refresh the page if it doesn't auto-refresh (auto-refresh seems to be somewhat broken in IE 5.5).
  • [by request] Added support for UNIX Sendmail logs.
  • Improved the choice of available views, sort, default visible and available view elements, and database structure in the case that there is no visitor information available.
  • Improved the default visible and available view elements and sort in the case that there is no bandwidth information available.
  • [by request] Converted hostnames to lowercase after looking them up with DNS.
  • Changed the Log Source selection page to use a popup menu mechanism (rather than radio buttons with the More Options link). This is cleaner, and provides more visual feedback that there are other options available.
  • [by request] Added several new Log Source options. The Log Source menu now lets you get the log data from a single local file, all files in a local directory, some files in a local directory (using a regular expression), a single FTP-accessible file, all files in an FTP directory, or some files in an FTP directory (using a regular expression). The "all files in a local directory" option is new, as are the multi-file FTP options.
  • [by request] Added automatic gunzipping of files fetched by FTP (the previous implementation could only read raw log data from an FTP file).

Version 6.4b6, shipped November 03, 2002

Bugs fixed in version 6.4b6:

  • [by request] Fixed a bug where Sawmill could crash while processing certain types of log formats, including iMail logs.
  • [by request] Fixed a bug where Sawmill could crash under certain circumstances, in particular when you click "Make Data Available" when running Sawmill on Windows. In theory, this could cause crashes on any platform, under various circumstances, but it was especially noticeable (and reproducible) on Windows.
  • [by request] Fixed a bug where the syslog format menu would not appear if there were multiple possible formats, and the first one was a generic format.
  • [by request] Fixed a bug where the bookmark URL would sometimes get mangled.
  • [by request] Fixed WebServerAccessLog logging so it reports the IP address correctly (as it did in 6.3.x).
  • [by request] Fixed a bug where if there were two formats detected, and one of them was a "Generic" format, Sawmill would (correctly) throw the Generic out of the matching list, but would still prompt for a choice with a list containing just one item. It now assumes you want the only one available.
  • [by request] Fixed a bug where Sawmill and IE were using different timeouts, causing clicks to result in broken pages if the click was between 10 and 35 seconds after the previous click.
  • [by request] Corrected date format for Centrinity First Class

New features in 6.4b6:

  • [by request] Improved the Session Filters editor. The filters no longer display arbitrarily-long menus-- instead, they display the first 1000 options, and you can enter options manually. The rest of the menus can be shown with a click, and the limit (1000) is customizable in the Stats Sizes tab.
  • [by request] Added support for Eims Mail SMTP log format
  • [by request] Improved NetScreen log format to track event types.
  • [by request] Improved GeoIP country lookup to work when DNS lookup is on. Previously, DNS lookup would resolve the IPs, preventing GeoIP from computing countries. Now, Sawmill preserves the IP for GeoIP's use, even if it's resolved, so countries are computed the same regardless of DNS settings.
  • [by request] Added support for Interscan Viruswall Virus log format
  • [by request] Improved handling of situation where the user enters something like logs/* as the log filename, and checks "pattern is a regular expression." * isn't a valid regular expression, but many users don't know that, and confuse regular expressions with wildcard expressions. Sawmill used to give a long-winded error message in this case, talking about wildcard expressions and regular expressions, but we frequently get "bug reports" containing this error message, so that wasn't working either. So Sawmill now tries the "regular expression" as a wildcard expression if it isn't a valid regular expression. In other words, entering logs/* and clicking "pattern is a regular expression" will now work-- it will analyze all files in logs/.
  • [by request] Added support for DecludeSPAM log format
  • [by request] Added red arrow icons to *all* URLs in tables (so you can go to that URL by clicking), not just referrer URLs.
  • [by request] Added support for FlashFXP log format
  • [by request] Added IIS/CGI installation chapter to documentation
  • [by request] Added support for WarFTPAlt log format
  • [by request] Reworked the PIX/IOS filter to be more flexible. Rather than having a huge list of pre-defined line formats, it now has a huge list of pre-defined log *segment* formats. So if new data is present that isn't expected, or if segments are put together in unexpected ways, or if some of the parts are missing, it will still grab whatever it can from each line. Since there are apparently an infinite number of PIX/IOS formats out there, this is necessary to provide something like universal compatibility.
  • [by request] Improved Cisco VPN Concentrator format to extract user bandwidth from "disconnect" lines.
  • [by request] Added date support for bpftTraflog
  • [by request] Added support for tcpdump With Interface Alternate
  • [by request] Added support for bpftTraflog
  • [by request] Improved Declude format to handle more fields.

Version 6.5b5, shipped July 21, 2003

Bugs fixed in version 6.5b5:

  • [by request] Fixed a bug in Microsoft Exchange 2000 log format to accept only if message processing ends
  • [by request] Fixed a bug where senders were not reported in McAfee WebShield log format
  • [by request] Fixed a bug where the Sort menu of the "Session visitors" view did not contain an option to sort by duration, and did contain a few options that did not make sense.
  • [by request] Fixed a bug where CGI mode did not return pages on Windows.
  • [by request] Fixed a bug where applying a session filter, and then removing non-session filters, could generate an error about "unknown session ID".
  • [by request] Fixed a bug where in some cases, session information would not be tracked, even when it was enabled in the configuration.
  • [by request] Fixed a bug which could cause crashes while analyze some log data with the GeoIP database.
  • [by request] Fixed a bug where items would disappear from the Task List after 10 minutes.
  • [by request] Fixed a bug where filter values containing commas could cause errors.
  • [by request] Fixed a bug where the 95th percentile calculation could crash if there was just one data point in the graph.
  • [by request] Fixed bug where attempting to sort the "Individual sessions" view chronologically would generate an "internal error"

New features in 6.5b5:

  • [by request] Added support for Intel NetStructure VPN Gateway log format
  • [by request] Added support for Exim log format
  • [by request] Improved Bulletproof "sessions" log format to handle multiple download lines
  • [by request] Added support for Yamaha RTX log format.
  • [by request] Added support for SpamAssassin log format.
  • [by request] Added support for Shorewall log format.
  • [by request] Added a new "skip most recent file" feature which skips the log file which has been most recently modified, when processing log data. This is useful when attempting to optimize log processing performance by skipping previously-seen files by filename-- in IIS and other situations, we don't want to process (and later skip) the most recent file because it's still being built.
  • [by request] Added support for Arcserve NT Log Format
  • [by request] Enhanced the "allow empty log source" option to skip non-existent directories and files in the log source (files which were there when the build began, but disappeared during the build), without an error.
  • [by request] Added support for Bulletproof/G6 FTP format with dd/mm/yyyy dates and 24-hour times.
  • [by request] Added support for Helix Univeral format.
  • [by request] Improved/fixed support for Microsoft Media Server, to track all fields automatically.
  • [by request] Added support for Cisco Access Register log format
  • [by request] Improved PIX/IOS format to handle a different line format.
  • [by request] Added support for Watchguard Firebox Export header format.
  • [by request] Improved Bulletproof FTP log format to handle a variant where RETR lines are replaced by "started download" lines. Also improved bandwidth tracking to handle KB notation.
  • [by request] Improved Watchguard format to accept multiple spaces between fields
  • [by request] Extended NetScreen Traffic Log format to handle either spaces or arrows (->) between some fields.
  • [by request] Added support for Watchguard SOHO log format
  • [by request] Improved NetScreen Traffic Log to handle translated addresses and ports, when present.
  • [by request] Added support for htdig log format
  • [by request] Added support for Windows Event Log Format (ALTools export)
  • [by request] Added support for PostWorks IMAP format
  • [by request] Added support for PostWorks POP3 format
  • [by request] Added support for PostWorks SMTP format
  • [by request] Improved Merak POP/IMAP format to support USER lines.
  • [by request] Added support for tcpdump log format with no command-line options.
  • [by request] Added automatic transfer of "summary" filter value to a "session" visitor id filter when it seems appropriate. Sawmill now looks through all normal statistics filters to see if there is any field which is a single filter item, where that item is a known visitor id value. If there is such a filter, and if there are no session filters active, the session filters are set to show only sessions for that visitor id. This provides a simple way to apply visitor id session filters through the view corresponding to the visitor id field ("Top visitor domains/hosts", "Top users", or whatever it happens to be for a particular log format.
  • [by request] Added an omit_database_fields option (intended for use in log format plug-ins) which allows some log fields to be explicitly omitted from the database fields, in formats where the log and database fields are automatically generated.
  • [by request] Added visitor_name option, which makes it possible for the term "visitor" to change for formats (like proxy servers) where the term "visitor" isn't appropriate. This option is used when creating log format plug-ins.
  • [by request] Improved IIS Web log format plug-in to use the new "generic W3C" features, so its log and database fields are generated automatically.
  • [by request] Added support for Netscape Extended log format.
  • [by request] Added support for Firewall1Webtrends logformat
  • [by request] Added support for Netwall logformat
  • [by request] Tightened autodetect expression for Generic CSV
  • [by request] Improved handing of FTP log source where hostname is entered as hostname/path.
  • [by request] Added a new $notsupported option which removes the entry/exit page views and other session-page-related views, for formats where there isn't really a concept of a "page" in the session information.
  • [by request] Added support for Blue Coat IM log format.
  • [by request] Added new "session logout regular expression" option, intended for use when creating a log format plug-in for a format which has session information and has a "logout" operation in the session information (unlike most web logs, which have no logout operation, requiring the use of a timeout to determine session endings). This allows for more accurate tracking of sessions for these types of logs, including Cisco VPN logs.
  • [by request] Improved session information by eliminating duplicate sequential page views in a single session, so reloading a page (or having the browser reload it) will not result in multiple events in the session displays.
  • [by request] Improved progress page to show major and minor operations in progress, even when "Details" is collapsed.
  • [by request] Improved detection of W3C and other log formats with format headers, so many more field names are converted to their "human readable" equivalents, and database fields and views are set up automatically for those fields, even if the fields are not explicitly mentioned in the log format plug-in.
  • [by request] Added automatic download of the GeoIP database file when it is first needed.
  • [by request] Improved the convert_field_map Log Filter type so it can handle maps stored in text files.
  • [by request] Enhanced country/region/city detection so region is reported by its FIPS name, rather than its FIPS code.

Version 6.4b5, shipped October 18, 2002

Bugs fixed in version 6.4b5:

  • [by request] Fixed a bug where if the "page" field was not deep enough, session information would be incorrect. This was not usually a problem because the page field is usually very deep (9 levels).
  • [by request] Fixed incompatibility with OmniWeb which would result in empty pages when forms were submitted.
  • [by request] Fixed a bug where the "visitors" column did not use thousands separators in its numbers.
  • [by request] Fixed the "Kill Server" button, which stopped working around 6.4b2.
  • [by request] Added support for Eurodate SYSLOG
  • [by request] Fixed a bug where CSV export would fail if the configuration name contained a space.
  • Fixed a bug where if you used a Syslog and Generic log format plug-in, and clicked the "visitors" checkbox, Sawmill would generate an error.

New features in 6.4b5:

  • [by request] Improved PIX/IOS plugin to track message code field.
  • [by request] Added a WebTrends Extended Log Format plug-in that can work with any syslog.
  • [by request] Implemented a generic Cisco VPN Concentrator log format, which works with any syslog server.
  • [by request] Improved the generic Unix Syslog plug-in to handle a slight variant.
  • [by request] Added support for CentrinityFirstClass (mm/dd/yyyy) log format.
  • [by request] Added a GeoIP database to Sawmill, so it can compute countries from IP addresses. This used to include a "Top countries" view in most configurations.
  • [by request] Improved the "collected log entries" feature of Sawmill so it would recycle collected log entries after a certain number of lines without activity. Previous versions would continue to keep the collected entries in memory for the entire log processing run, which is theoretically necessary in case a related log entry at the very end of the data, but in practice is not necessary, because associated entries tend to clump very close together. In cases of very large datasets, an infinity growth option was impractical, requiring several gigabytes of RAM in some examples. By default, Sawmill now recycles log entries after 10,000 lines of inactivity; this can be customized, and recycling can be disabled to get the old behavior.
  • [by request] Added support for Interscan Proxy log format.
  • [by request] Made filters clearer when active, and added a more prominent brief blurb about filters and views in the Overview, which appears when Filters are active. This is an attempt to make the usage of Filters and Views clearer to beginning users, in particular to make it clear that after clicking an item to apply a Filter, you can click a View down the left to see more information about that item.
  • [by request] Added support for Cisco Router log format.
  • [by request] Added support for Declude log format.
  • [by request] Added support for Bulletproof/G6 log format with mm/dd/yy dates.
  • [by request] Added support for Groupwise Web log format.
  • [by request] Improved FTP log source entry so if you enter *both* a URL and other fields (like username), the values entered in the fields will override those in the URL. Previous versions ignored all fields except the URL, if a URL was specified, so if you entered ftp://mysite.com/ as the URL, and entered the username/password/pathname below, they would be ignored, and anonymous FTP would be attempted.
  • [by request] Added support for Novell Border Manager logs
  • [by request] Added red-arrow icons to entry/exit page views, when "Server Root" is set.
  • [by request] Improved generic Cisco format to handle FW-3 and FW-4 lines better.
  • [by request] Improved Cisco Generic plug-in to correctly detect files that consisted entirely of %FW lines.
  • [by request] Fixed a bug where if the log format matched one or more syslog formats, but didn't match any other formats, Sawmill would show an empty menu of available formats.
  • [by request] Fixed a bug where images did not appear correctly in the progress page generated during a "send view by email" operation.
  • [by request] Fixed a bug where progress pages would reload forever during a "send view by email" operation, never actually showing the final result page, even though the email had been sent.
  • [by request] Split Netscreen format apart from its syslog-dependent section, so it will work with any supported syslog now.
  • [by request] Improved Netscreen format to handle a slightly variant of the format.
  • Split Firebox format apart from its syslog-dependent section, so it will work with any supported syslog now.
  • [by request] Extended support for generic Cisco logs
  • [by request] Added support for whatsUp syslog
  • [by request] Added an "Average bytes transferred per second" item to the Overview.
  • [by request] Added support for SymantecWebSecurity logs
  • [by request] Improved the syslog/nonsyslog split so the syslog format menu is only shown when a "require syslog" format is selected.
  • [by request] Improved format screen so the custom format string fields are only shown when the associated custom format plug-in is selected.
  • [by request] Improved log format detection so "Generic" formats like W3C Generic are only offered as a possibility when no non-Generic formats match.
  • [by request] Added support for GENERIC_IOS Debug IP Packet Detailed Logs.
  • [by request] Improved the log formats list so "General" log format (like General W3C, previously called Generic W3C) only appear if there are no other matches.
  • [by request] Extended support of ServU FTP logs.
  • [by request] Improved handling of DNS server failures; Sawmill now switches to the secondary as soon as it sees the first primary failure. If the secondary also fails at some point, it tries the primary again, etc.
  • [by request] Created generic Unix Syslog log format plugin.
  • [by request] Improved full-Sawmill-URL generation (used to generate bookmark URLs) to look at the SERVER_NAME and SCRIPT_NAME environment variables and use them intelligently in the URL.
  • [by request] Added support for Interscan Email Logformat
  • [by request] Added support for SNORT Portscan logs
  • [by request] Extended support of Cisco PIX KIWI IOS logs
  • [by request] Extended support of Cisco PIX UTC logs
  • [by request] Added "reverse alphabetical" as a statistics table sort order.

Version 6.0b5, shipped March 9, 2000

Bugs fixed in version 6.0b5:

  • [by request] Fixed a bug where under some circumstances, Sawmill would crash, or generate an Internal Error, while browsing statistics. This could be brought on by unusual database structures, but could also theoretically occur with the default structure.
  • [by request] Fixed a bug where some "turn-down" controls (ex. "more options" in "Where is the Log Data?") would generate an error when used.
  • Fixed a bug where the table pie chart legend squares would not always show all the colors shown in the pie charts.
  • [by request] Changed the way Sawmill loads itself into a local Browser on Windows (the Use Sawmill button). The new mechanism should work better across a wider range of systems.
  • [by request] Added an "Are you sure" prompt when the Sawmill window is closed on Windows. Users were closing it to get it out of the way, without realizing that closing the window shuts down Sawmill completely.
  • [by request] Changed Sawmill to default to 127.0.0.1 as its IP address when it can't find another. This fixes a problem where on some systems in CGI mode, it couldn't find its own IP address, and was generating an error.
  • [by request] Improved Sawmill's behavior in out-of-memory situations; it shouldn't crash as easily when it runs out of memory now.
  • [by request] Fixed a bug where expanding/collapsing field limits would discard any changes you had made.
  • [by request] Fixed a bug where the MacOS version would quickly consume all its memory when doing a database create or update via AppleScript.
  • [by request] Fixed bug where the MacOS version showed the wrong number of active threads when one of the threads was started by AppleScript.
  • [by request] Fixed a serious counting bug were Sawmill would misreport the number of hits on items, often by a wide margin.
  • Fixed a bug where Sawmill would parse referrers wrong in Apache Extended format if the referrers contained spaces.

New features in 6.0b5:

  • [by request] Improved the way links in tables, pie charts, and bar graphs work. Previously, clicking on a link like "index.html" would apply that as a limit to the page field, but would keep you in the "page statistics" view, which resulted in a confusing page containing only a single item: index.html. Sawmill now attempts to switch to a view with useful information in it. Failing that, it does not link the item at all.
  • [by request] Added complete instructions to every statistics page. The instructions describe what the current view is all about, and also provide general instructions about using the statistics pages. The instructions can be expanded and collapsed in three stages, to show nothing, just a small blurb about the current view (the default), or full information about the statistics page.
  • Split the Filters into two sets. The "Parsing Log Filters" are used to parse logs, and include such things as the colon-to-slash translator for WebSTAR pathnames, and the multiline log entry collection filters for NetPresenz logs. The normal "Log Filters" work as before. Most log formats do not using parsing log filters, and you generally won't have to deal with them, but in some special cases, if you have to define a custom log format, they can be handy. They are run before the normal filters, and only entries accepted by the parsing log filters make it to the regular filters.
  • Improved offline HTML generation mode ("Generate HTML Files") by adding two options, one to eliminate all cross-referencing from offline HTML files, and another to eliminate all bottom-level links (ex. links on HTML pages in the page statistics view) from offline files. These two together greatly reduce the number of files generated, at a cost in flexibility.
  • Improved display of visitors slightly; Sawmill displays "-" instead of "0" when there is no visitor information (ex. when the field is not cross-referenced to the visitor id field).
  • [by request] Added support for NetPresenz Log Format.
  • Added support for hmm time format (ex. 5:32, 18:33).
  • Improved the host hierarchy. IP numbers are now omitted by default, and when not omitted, they are all groups directly under IPNumbers, rather than hierarchially grouped under IPNumbers. This reduces the size of the database. The old structure was optimal for visitor statistics performance, but the new visitor mechanism doesn't use the host field anymore, so it can be restructured in a more reasonable way.
  • [by request] Improved linkage on Solaris so Sawmill will now run on systems which do not have libstdc++ installed.
  • [by request] Reworked the way statistics element visibility (ex. whether there's a column for hit %, or a visitors pie chart), row/depth limits, and sorting works. A new configuration category, "Stats Views," is a huge table letting you customize 20-some items for each view. You can now control, for instance, the number of rows in a table, which pie charts are visible, and how it's sorted, on a view-by-view basis. This provides a lot more control over how views look, and perhaps just as important, it lets Sawmill choose reasonable defaults for each view, rather than trying to pick defaults which work across all views (which has not turned out to work very well).
  • Changed the "Sort" toolbar to a "Sort By" popup menu. You choose from the menu and click Change Sort to change the sort order of the table or graph. This is more space-efficient, allows more detailed descriptions of the sorts to be used, makes it easier to have different possible sort sets for each view, and makes it easier to add new sort types in the future.
  • Changed the "Show/Hide" toolbar into "Show" and "Hide" popup menus. You choose from a menu and click Show or Hide to show or hide an element of the view. This is more space-efficient, allows more detailed descriptions of the view elements to be used, makes it easier to have different types of view elements for each view, and makes it easier to add new element types in the future.
  • [by request] Changed the View, Sort, Show, and Hide menus to automatically reload the page when you select something.
  • [by request] Removed the Field Limits Editor from the statistics page. It is now on its own page, which you can get to by clicking "edit" in the field limits summary bar. This makes the statistics page look cleaner. The Field Limits Editor has also been changed cosmetically to look nicer, now that it has some room to grow (instead of having to be as small as possible so it doesn't use too much statistics page room).
  • [by request] Added a blurb to the "Please enter the Sawmill Password" page to make it clear that this is the password that the user chose when they first installed Sawmill (and not, for instance, a license key provided by Flowerfire).
  • [by request] Added a new "row numbers" view element (visible by default) which shows a small number to the left of every row, for easy counting, and easier "comparing notes" with other people looking at the same report.
  • [by request] Added a new "offline depth" view option, so you can control, on a view-by-view basis, how many levels deep the offline statistics pages go.
  • [by request] Added a new "offline max rows" view option, so you can control, on a view-by-view basis, how many rows each offline statistics table has.
  • Changed the cross-referencing icons so that unavailable icons are omitted rather than dimmed. This saves horizontal space, and eliminates a lot of nearly useless icons.

Version 6.4b4, shipped September 24, 2002

Bugs fixed in version 6.4b4:

  • [by request] Fixed bug where Sawmill was not communicating correctly with some FTP servers, including UNIX servers, resulting in hung FTP transfers.
  • [by request] Improved behavior when no log format is selected; Sawmill redisplays the "choose log format" screen with a red prompt now, rather than showing an error screen.
  • [by request] Fixed a bug where exported session tables sometimes contained
    .
  • [by request] Added "total session page views" and "average page views per session" to the Sessions Summary view.
  • [by request] Improved support for iMail log format, so it picks up more information, and reports more accurately.
  • [by request] Improved Squid log format to accept resolved hostnames as well as IPs.
  • [by request] Fixed a bug where updating a database using "safe update" could cause an error on Windows.
  • [by request] Fixed a bug where file types were not detected properly if there were URL parameters.

New features in 6.4b4:

  • [by request] Added support for Centrinity FirstClass logs
  • [by request] Added checkbox to view/subview filter editor so "incremental" filters can be specified from the GUI.
  • [by request] Added support for syslog-only log format plugins, and "generic" log format plugins. Previous versions of Sawmill supported separately each combination of logging device and syslog server. This could get very ugly, with Cisco PIX as the classic example-- 6.3b3 supported 15 versions of Cisco PIX, all similar but using different Syslog servers. Now, there is a "generic" Cisco PIX plugin, and a simple Kiwi Syslog plugin that is not tied to any particular device, and Sawmill lets you choose both of them, and combined them to support Cisco PIX logged through Kiwi (if only one "generic" and only one "syslog" format match, it will be chosen for you automatically, comparably to how it's done now). We will continue to add new "generic" plug-ins and "syslog" plug-ins until all previous syslog-based plugins have been obsoleted, but the will be a period of overlap. When we're done, the total number of formats supported by Sawmill will be dramatically higher than before, because every one of 20-some syslog formats will work with every of the 20-some generic formats, for 400 new effective supported formats.
  • [by request] Added support for IST logs
  • [by request] Added support for CiscoVoiceRouter (KIWI) logs
  • [by request] Added support for MonitorWare logs
  • [by request] Added support for MonitorWareAlt logs
  • [by request] Added an "allow viewers to rebuild" option. When this option is checked, there will be rebuild/update links in the info bar of the configuration, even if the user isn't administrator, and the user will be able to manually update/rebuild the database by clicking the links. No other administrative operations will be available.
  • [by request] Added ignore_quotes option. This option, which is generally of use only if you're creating a custom format, tells Sawmill to treat quotes (" or ') the same as any other character. Some formats require quotes to be treated specially for optimal processing; some require they they *not* be treated specially; this lets you choose.
  • [by request] Implemented ColdFusion Application (CSV) log format.
  • Improved error handling in CGI mode in the event that the specified "temporary directory" cannot be created, or is a file.
  • [by request] Added update/rebuild links to the info bar in the config menu, to allow for easy updating/rebuilding of database from the statistics (admin only).
  • [by request] Extended support for iMail logs
  • [by request] Added support for Cisco IOS Alt3 logs
  • [by request] Improved W3C support to handle #Fields: lines with a tab after the :. Also, added support for #Start-Date: headers in W3C logs.
  • [by request] Added rekey_collected_entry and reverse_rekey_collected_entry log filter actions. These filters are of interest only to people creating log format plug-ins for strange formats. They allow Sawmill to process log formats where the key field changes as the entry goes through stages, but where there is some value that ties the old and new keys together.
  • [by request] Added a new concatenate2 log filter type, which puts the result in field2 instead of field1.
  • [by request] Added support for Cisco PIX KIWI Alt2 logs
  • [by request] Added support for Cisco SOHO77 logs
  • [by request] Improved the "Data Not Available" error message and page. The error message has been simplified in the event that the viewer is not administrator, and no longer talks about how the data can be made available. When the view is an administrator, the message is similar to before, except that it also includes a "Make Data Available" button at the bottom. Clicking this button adds the necessary xrefs, rebuilds the database, and shows the view again (now with data available).
  • [by request] Added support for Siteminder Netegrity beyond 4.x logs
  • [by request] Added a new "Support email address" field in the Preferences (Miscellaneous tab) which specifies the address where bug reports should be sent. This should be set by administrators of multi-user installations, because the bug reports are usually configuration issues that can be corrected by the Sawmill administrator, and cannot be corrected by us (the software vendor), since we don't have access to the installation. By default, support mail will continue to come to us until this option is changed.
  • [by request] Extended support for Cisco PIX KIWI logs
  • [by request] Added support for GNATBox Kiwi yyyy-mm-dd logs
  • [by request] Improved Squid log format to accept resolved hostnames as well as IPs. Improved export so that the statistics are shown after the export, and the export is done with a .csv file extension and Content-type text/csv. Previously, exports long enough to require progress pages would be left dangling at the progress page after the export completed; now the statistics are shown at the end of the progress cycle. Also, the exported filename was previously the same as the name of the Sawmill executable (e.g. "sawmill" or a derivative), which made it difficult to figure out what format the export was in. Now the file ends with .csv, so the format is clearer (and the browser will be able to open it with an appropriate program, like Excel).
  • [by request] Extended support for Snort logs
  • [by request] Added support for Snort Alternate logs
  • [by request] Added "don't autodetect log format" checkbox to configuration creator, so if the autodetector is getting it wrong, it can be overridden.
  • [by request] Improved "log out" so it takes you back to the configuration login page, if appropriate.
  • [by request] Moved the "single-page summary" to the bottom of the views list, so first-time Sawmill uses won't be discouraged by the long time it takes to generate that view, when they're first clicking through the views.
  • [by request] Added an "extra options" field to Scheduler items, so arbitrary command line options can be sent to Sawmill as part of the scheduled command line task. This allows powerful options like Scheduling different views to be emailed to different people, with different filters, which used to require the use of an external scheduler.
  • [by request] Fixed a bug where errors were not reported properly, especially on Windows (a blank page would appear instead).
  • [by request] Added autodetection of RADIUS format strings, so RADIUS logs with arbitrary fields are now supported.
  • [by request] Added filter numbers to the Log Filters table.

Version 6.4b3, shipped September 05, 2002

Bugs fixed in version 6.4b3:

  • [by request] Fixed a bug where some of the log format plug-ins were corrupt, causing errors when autodetecting log formats.
  • [by request] Fixed a bug that could cause floating point exceptions during command line database builds.

New features in 6.4b3:

  • [by request] Extended support for Netegrity SiteMinder 4.X logs

Version 6.5b3, shipped May 25, 2003

Bugs fixed in version 6.5b3:

  • [by request] Greg fixed a problem with the date getting corrupt in the accept_carryover scenario.
  • [by request] Fixed a bug where with certain log formats (including iMail), when Sawmill accepted the "orphan" log entries that had never been accepted before, it would not run the Log Filters on them, resulting in log entries appearing in the database which violated the Log Filters rules.
  • [by request] Fixed a parsing problem in Unix Sendmail log format
  • [by request] Fixed a bug where statistics filtering did not work properly in some cases if filters were applied simultaneously to two or more fields.
  • [by request] Fixed a bug where "week" links in the Calendar did not work properly.
  • [by request] Fixed a bug where if there was no date/time database field, all log entries would be rejected.

New features in 6.5b3:

  • [by request] Extended Support for EIMSSMTP logformat
  • [by request] Improved "Generate HTML Files" so it updates an existing HTML files set incrementally. Previous versions of Sawmill would generate all files every time. Now, Sawmill breaks the files down by date grouping (year/month/week/day), and only regenerates a group if the number of hits/pageviews/visitors/bytes have changed since the last generation. This gives the same results as the previous version, but can be much faster if the changes are small (or if there are no changes).
  • [by request] Added support for Helix Universal log format.
  • Fixed a bug where the column headers were links even when the page was an offline page.
  • [by request] Added support for "Windows Event Log (dumpevt.exe export)" format.
  • [by request] Added support for "Timestamp (mm dd hh:mm:ss)" syslog format.
  • [by request] Enhanced Cisco VPN Concentrator format to track "visitors" (unique users) and "session" information (including individual user sessions and time connected).
  • [by request] Added support for InterScanViruswall log format
  • [by request] Enhanced W3C and CSV "generic" log format plug-in so it builds the list of database fields from the detected log fields; this allows Sawmill to analyze W3C logs which have log fields it has never seen before.
  • [by request] Added support for Websweeper log format
  • [by request] Extended Kiwi/ISO syslog format to handle a slight variant.
  • [by request] Extended support for SnortStandalone
  • [by request] Added support for MailSweeper log format.
  • [by request] Added support for WatchGuard log format (with arbitrary syslog).
  • [by request] Improved browser identifier to better identify IE, Netscape, Mozilla, Chimera, Phoenix, Opera, Safari, and other major browsers.
  • [by request] Added support for Separ URL Filter Log Format.
  • [by request] Added a command-line documentation blurb which is printed when Sawmill is run with just --help, -help, -?, and a few other variants.
  • [by request] Added support for Unicomp Guinevere log format
  • [by request] Added support for LSMS Admin log format
  • [by request] Added support for Kiwi Cattools Port Stat log format.
  • [by request] Changed web server mode binding behavior so it binds to 0.0.0.0 when no server_hostname is specified. Previous versions would bind to the first server IP, which was sometimes difficult to determine; binding to 0.0.0.0 causes Sawmill to bind to *all* interfaces, so at least http://127.0.0.1:8987/ (the loopback interface) is guaranteed to work. It still prints the first IP of the server, as well as it can compute it, but if it gets it wrong, you can use the actual IP and it will work, without having to reconfigure Sawmill to tell it what that IP is.
  • [by request] Added support for Iplanet Messenger Server 5 log format
  • [by request] Added support for InterScanSecuritySuite log format
  • [by request] Added support for WinProxy Alternate log format
  • [by request] Added support for "Generic CSV" log format, which handles all types of CSV files somewhat intelligently, even if it doesn't know the fields in advance or what they represent.
  • [by request] Modified the gnatbox logformats so that they are syslog & syslog required format combinations
  • [by request] Extended support for MicrosoftExchange2000
  • [by request] Added support for SL4NTDDMMYYYY
  • [by request] Improved SonicWall format to track page views and session information, by building the URL from the proto, dstname, and arg fields.
  • [by request] Extended UnixSendmail log format to support rejects
  • [by request] Improved the Filter Bar display in the statistics to include a dark sidebar and other ways to bring attention to the fact that Filters are active.
  • [by request] Added support for Imail7 log format (syslog required).
  • [by request] Added support for VeritasNetackup log format.

Version 6.4b2, shipped September 04, 2002

Bugs fixed in version 6.4b2:

  • [by request] Disabled numerical and alphabetical sorts for views where it didn't make sense.
  • [by request] Fixed a bug where database updates would fail if 1) they were safe updates, and 2) the configuration name contained a space.
  • [by request] Fixed a bug where safes updates would fail on Windows because it couldn't delete the "writeLock" file.
  • Fixed a bug where the date/time range controls would appear in static pages.

New features in 6.4b2:

  • [by request] Added an "expand all" link to the "paths through the site" toolbar, to fully expand the entire paths tree.
  • [by request] Added support for %{%Y-%m-%d}t and %{%T}t dates and times in Apache custom log format strings.
  • [by request] Added support for Quicktime Streaming Server logs
  • [by request] Added support for CacheFlowRealMedia & CacheFlowerWindowsMedia logs
  • [by request] Added support for LSMTPAccess logs
  • [by request] Changed the way Sawmill handles GUI requests. Previous versions would handle requests by creating a separate thread within the main web server process; Sawmill now creates a separate process for each new request. This has some major advantages. Most importantly, it improves stability by making it impossible for separate tasks to stomp on each others' memory. This was a particularly big problem on Solaris, where there has been a bug which causes crashes when using Sawmill in web server mode; this bug is now fixed. Other lesser bugs (including some unknown ones) may also be fixed by this change. This change also makes other features possible, including the improved progress feature and the web server administration improvements in this version.
  • [by request] Improved progress reporting in CGI mode and in single-threaded mode. Sawmill now uses the full progress page in all cases. Previous versions used a similar version in CGI/single-threaded most with Netscape, but the feature used was not available in other browsers (like IE), so it was forced to fall back on a "stacked lines" progress indicator, where the progress information appeared as a series of tiny lines at the top of the page. That is gone-- Sawmill uses reloading progress in all cases now. It even uses reloading progress pages when it's used from the command line, and the new Web Server Administration task list lets you tap into a running command-line build to see full progress information.
  • [by request] Improved the Web Server Administration page so it shows a table of active tasks. You can click a task to see progress information for that task. This works even if the tasks are CGI-based, Scheduled, or command-line-- any task will show up in the list, and you can get full progress information for any task by clicking it. The Web Server Administration was not available in CGI mode in earlier versions, but now that it has a useful purpose, it has been enabled.
  • [by request] Added support CacheFlow RealMedia logs
  • [by request] Added command-line CSV export of the Sessions (summary) and Overview views.
  • [by request] Added support Watchguard WELF log format
  • [by request] Added expansion of path segment larger than a certain size, automatically in offline pages and manually through an "expand all" link in dynamic stats.

Version 6.5b2, shipped April 06, 2003

Bugs fixed in version 6.5b2:

  • [by request] Fixed bug where country/region/city information would sometimes be garbled.

New features in 6.5b2:

  • [by request] Added support for eManager Spam Filter Log Format.
  • [by request] Added support for SonicWall Alternate Log format
  • [by request] Added support for Kerio Network Monitor Log format
  • [by request] Improved the Postfix log format plug-in to work with arbitrary syslogs.
  • [by request] Improve Lyris MailShield log format to handle many specific types of messages (including spam blocking messages).
  • [by request] Improved Mail Daemon Log Format to handle SMTP lines.
  • [by request] Changed the "Report It" bug reporter to use HTTP instead of SMTP to send bug reports-- this works better when reporting bugs through some firewalls.
  • [by request] Fixed a bug where if a field value contained a comma or semicolon, Filters would not work properly for that value.
  • [by request] Improved hh:mm:ss time parser to accept non-colons between the numbers. This allows Sawmill to handle a Kiwi syslog variant which uses periods.
  • [by request] Extended support for Netscreen log
  • [by request] Improved WUFTP log format to handle a slightly variant
  • [by request] Added support for Miva Common Access log format
  • [by request] Added a new "Rename Configuration" option to the administrative menu, which changes the name of an existing configuration. Unlike the "Save As" option in the Configuration Menu, this deletes the original configuration (after successfully writing the new one), and moves the database over from the old configuration to the new.
  • [by request] added support for RadiusAccounting Log II
  • [by request] Improved DNS lookup so it looks up *all* IP addresses, not just those in the "host" field. Previous version would only look up IP addresses for one field-- the one whose type was "host" in the Log Fields tab. Sawmill now looks at all fields when DNS is on, and resolves all IP addresses.
  • [by request] Extended support of UnixSendmail logs
  • [by request] Added support for msg= bandwidth in WELF logs.
  • [by request] Extended support for Netscreen logs
  • [by request] Added support for SyslogYYYYMMDDHHMMSS
  • [by request] Added support for SmartMaxSMTP & SmartMaxPOP logformats
  • [by request] Added support for Kiwi (mmm/dd dates, hh:hh:ss.mmm UTC times) syslog format.
  • [by request] Added support for NetScreen Syslog Log Format.
  • [by request] Added support for Sophos Mail Monitor for SMTP Log Format.
  • [by request] Added ignore_newlines_after_log_line_regexp option, which makes it possible to create log format plug-ins that automatically combine certain consecutive lines into single lines on-the-fly (useful if a KEY field on one line is needed to parse the next line).
  • [by request] Added support for WinSyslog log format
  • [by request] Added support for Youngzsoft CCProxy log format
  • [by request] Added support for Microsoft Exchange Internet Mail Log formats
  • [by request] Added support for IIS SMTP Log formats
  • [by request] Added a new "Session visitors" view which shows the visitor ids from the session information, along with the number of page views and total time spent for each.
  • [by request] Improved export of "Single sessions" view so it exports the clickstream also.
  • [by request] Improved Interscan Email Viruswall format to report "Forwarded" messages.
  • [by request] Added support for Seconds since Jan 1 1970 timestamp syslog Extended support for Squid syslog required log format Extended support for Webtrends syslog required & syslog log formats
  • [by request] Improved the layout of the Session Filters description (at the left of the Session Filters editor).
  • [by request] Improved table headers so clicking a header name will re-sort the table by the corresponding field.
  • [by request] Extended support of Squid Common Log Format
  • [by request] Added support for iPrism-rt, iPrismMonitor, Webtrends syslog, Webtrends syslog required log formats
  • [by request] Improved Cisco PIX/IOS format to extract IDS error messages better.
  • [by request] Improved instructions for "top countries/regions/cities" view, and added a section in the documentation on using the GeoIP database.
  • [by request] Improved Bind 9 Query log format to handle a slightly variant.
  • [by request] Added support for Firewall1 Next Generation Full logformat
  • [by request] Added support for another (brand currently unknown) syslog format.
  • [by request] Added support for Firewall1 Next Generation General logformat
  • [by request] Added support for AladdinEsafeGateway logs
  • [by request] Extended support for CiscoVPNConcentrator logs
  • [by request] Added support for IIS log with mm/dd/yyyy dates
  • [by request] Improved line parser so it strips off linefeed and newline characters at the end of blank lines; this makes it possible to write parsing filters that identify blank lines (for instance, to accept on blank lines).
  • [by request] Improved iPlanet Error Log Format to handle a slight variant.

Version 6.4b1, shipped August 13, 2002

Bugs fixed in version 6.4b1:

New features in 6.4b1:

  • [by request] Extended support of OpenwaveIntermail Server Logs to capture bandwidth information
  • [by request] Extended support of Postfix Mail Server Logs
  • [by request] Changed language module version numbers to match Sawmill version numbers. For instance, where the language module version for Sawmill 6.3.10 used to be v88, it is now v6.3.10. This makes it easier to find the language module for a particular version, and eliminates certain types of uncommon errors where a pre-release version would not be able to find recent language module variables.
  • [by request] Added support for War FTP logs
  • [by request] Added date range filtering-- the current shown date range now appears at the top of the statistics, in the Filters bar, and you can choose the starting and ending year/month/day to zoom in on a particular date range.
  • [by request] Added a "safe update" feature (on by default) which backs up the database before updating it. This all but eliminates the possibility of an update corrupting the database on an error, but it requires twice the disk space and some additional time for an update.
  • [by request] Added an "If A, then do B followed by C" filter type.
  • [by request] Optimized database updating in the case where the is no new data in the log source. Previous versions would re-consolidate the database and re-write the index, even though there was no change. This version skips that step, resulting in a much faster update in this situation.
  • [by request] Added support for Apache Combined with Server Name after Agent
  • [by request] Added support for Cisco IOS (Unix syslogd) as well as created language variable IOS_PARSING_FILTERS logs
  • [by request] Added support for Cisco CE Common logs
  • [by request] Changed sort order to list directories first in the Browse... window.
  • [by request] Added support for Cisco PIX SL4NT logs
  • [by request] Added a new "paths through a page" view, which shows all paths through a particular page by showing the immediate predecessors and successors in the clickstream.
  • [by request] Added support for a LOGANALYSISINFODIR environment variable which, when set, determines the location and name of the LogAnalysisInfo folder.
  • [by request] Added a number_thousands_separator option, which is a comma (,) by default, and which is used to separate thousands in large numbers.
  • [by request] Extended support for SonicWallKiwi logs
  • [by request] Extended support for Netegrity Siteminder logs
  • [by request] Added support for OpenwaveIntermail logs
  • [by request] Extended support in CiscoIOS log format to support AUDIT_TRAIL information
  • [by request] Added support for SonicWallKiwi (yyyy-mm-dd) log format
  • [by request] Added support for Firebox log format
  • [by request] Added support for Sambar Server log format
  • [by request] Added support for Cisco IDS IOS log format
  • [by request] Added support for Net-Acct log format
  • [by request] Added support for PostOfficeMailServer log format
  • [by request] Added support for IISFTP log format
  • [by request] Added a new option to make graph bars non-clickable (by default, they are clickable). This is useful in cases where there are so many bars that the overheard of making them clickable bogs down the network, Sawmill server, and/or browser.
  • [by request] Added support for Sidewinder log format
  • [by request] Added support for InterscanEmailViruswall log format
  • [by request] Extended support of TinyPersonalFirewall to include a new variety of log
  • [by request] Added support for TomcatAlt log format, also Greg modified Sawmill so that it can collect and carryover
  • [by request] Added support for SquidGuard log
  • [by request] Added an accept_collected_entry_regexp_carryover filter, which works like accept_collected_entry_regexp except that it carries the collected values over rather than resetting them. So a single field value can be collected early on, and later acceptances will all use that value, even if it does not appear again.
  • [by request] Added support for MailerDaemon log
  • [by request] Improved Performance for Cisco PIX KIWI ISO
  • [by request] Added support for Cisco PIX KIWI ISO.
  • [by request] Added support for Whistle Blower Performance Metrics Log.
  • [by request] Added support for Netscape Directory Server log format.
  • [by request] Added "numerical ascending" and "numerical descending" table sorts.
  • [by request] Improved W3C formats to support x-datestamp fields.
  • [by request] Added support for MailMax SE POP log format.
  • [by request] Added support for NetCache NetApp log format.
  • [by request] Added support for MailMax SE SMTP log format.
  • [by request] Added support for Atom Mesge log format.
  • [by request] Added support for RealProxy log format.
  • [by request] Added support for ColdFusion Web Server log format.
  • [by request] Added support for ColdFusion Application log format.
  • [by request] Added support for Microsoft SQL Profiler log format.
  • [by request] Added support for "m/d/yy h:mm" date/times.
  • [by request] Added support for files (like CSV files) which can have line breaks in the middle of a field, as long as the field is quoted.
  • [by request] Added support for WhistleBlower log format.
  • [by request] Added support for Windows Event log format.
  • [by request] Added support for FedEx tracking log format.
  • [by request] Added support for FastHosts log format.
  • [by request] Improved Postfix log format to handle some different types of log entries.
  • [by request] Added support for Network syslog format.
  • [by request] Added support for Merak SMTP Log Format.
  • [by request] Added support for Merak IMAP/POP3 Log Format.
  • [by request] Improved IPTables format to handle a slight variant.
  • [by request] Added support for WebShield SMTP log format.
  • [by request] Added support for Webtrends Extended log format.
  • [by request] Added support for Zyxel/Kiwi log format.
  • [by request] Improved session IDs so they contain the visitor id, for easier identification of the source of the session.
  • [by request] Added a new type of session filter, which lets you zoom in on particular session using its ID.
  • [by request] Improved the Sessions view (now called "Individual Session(s)" so the session IDs are clickable, and apply a session filter to show only that session.
  • [by request] Improved the Sessions view (now called "Individual Session(s)" to show a click-by-click listing of the session, if there is only one session selected.
  • [by request] Added support for Cisco VPN Concentrator (Comma-separated) log format.
  • [by request] Improved iMail log format to track local domains and "ldeliver" messages.
  • [by request] Added support for Hosting.com log format.
  • [by request] Added support for Cisco PIX (NT Syslog, With Hostname) log format.
  • [by request] Added support for NetScreen log format.
  • [by request] Added support for X-Stop log format.
  • [by request] Added support for Microsoft Proxy logs with m/d/yyyy dates.
  • [by request] Added support for ProFTP log format.
  • [by request] Added support for URLScan log format.
  • [by request] Improved error messages to eliminate the ugly traceback information; on a related topic, you can now click Report It to email any error message to our support email address.

Version 6.5b1, shipped February 10, 2003

Bugs fixed in version 6.5b1:

  • [by request] Fixed bug where clicking Back after editing a particular statistics Filter, and then clicking Apply Filters, would re-edit the filter.
  • [by request] Fixed a bug where custom hierarchical fields were grouped incorrectly in some cases.
  • [by request] Fixed bug where Snort format did not work with certain syslogs.
  • [by request] Fixed bug where GNATbox syslog was not always detected properly.
  • [by request] Fixed a bug where Webtrends Extended log format conflicted with Kiwi log format over who got to track the "priority" field, resulting in an error when they both tried to.

New features in 6.5b1:

  • [by request] Added support for Squid Common format.
  • [by request] Improved iPlanet Error format to extract hostname, type, and other message information.
  • [by request] Added support for Cisco IDS Netranger logformat.
  • [by request] Added support for "Kiwi (mm-dd-yy dates, with type and protocol)" syslog format.
  • [by request] Added support for GFI Attachment & Content logs Added support for GFI Spam logs Extended support for IPTraffic log formats
  • [by request] Added support for IPTraffic LAN Statistics, and IP Traffic TCP/UDP Services log formats Extended support for IPTraffic log formats
  • [by request] Added support for decimal seconds on mm/dd/yyyy hh:mm:ss.sss
  • [by request] Added support for Win2KPerformance Monitor
  • [by request] Extended support of Microsft Exchange Log to only count messages that are sent not just queued and to count multiple recipients
  • [by request] Added support for Free Radius Detail Log
  • [by request] Improved default_log_date_year option to it uses the current year by default
  • [by request] Added support for Lucent Brick log format
  • [by request] Added a new log filter type; convert_field_map, which maps specific field values to specific other field values; i.e. you can remap 1->cat, 2->dog, 3->lemur, etc.
  • [by request] Added support for Bind 9 Log Format
  • [by request] Added support for WebSEAL Authorization (XML) Log Format.
  • [by request] Added explicit support for WebSTAR W3C Log Format (it was already supported indirectly through the Generic W3C Web Server Log Format).
  • [by request] extended support of CiscoVPNConcentrator logs formats Unix Syslog
  • [by request] Added support for IAS Alternate Log Format
  • [by request] Added support for mm/dd/yyyy variant of Interscan Proxy Log Format
  • [by request] Improved Interscan Proxy plug-ins to report virus activity
  • [by request] Added support for Filemaker 3 Log Format
  • [by request] Added lock_database_when_in_use option
  • [by request] Added support for iPrism Log.
  • [by request] Added support for NetScreen Traffic Log.
  • [by request] Added support Squid logformat (syslog required).
  • [by request] Extended support Cisco PIX/IOS logformats to collect information from audit lines.
  • [by request] Added support for Sawmill Task Log format.
  • [by request] Added support for ISC DHCP log format.
  • [by request] Improved EIMS SMTP format to track domains/emails hierarchically.
  • [by request] Improved Cisco VPN Concentrator format to work with a wider variety of syslogs.
  • [by request] Added support for ipfw log format.
  • [by request] Added support for Borderware log format.
  • [by request] Added support for Unix Syslog With Year format.
  • [by request] Extended Kiwi dd/mm/yyyy format to handle Kiwi UTC dd/mm/yyyy format too.
  • [by request] Extended support of CommunigatorPro logformat
  • [by request] Added tracking of regions and cities, using the GeoIP database.
  • [by request] Created a SidewiderSyslog and a SidewinderFirewall plugin pair to support a SidewinderFirewall format
  • [by request] Changed Postfix logformat to accept only on sent messages
  • [by request] Extended support of ImageMaker Microtech Media log more variations.
  • [by request] Extended support of Merak SMTP log
  • [by request] Added support for Lyris MailShield log format
  • [by request] Added support for Windows Event (Tab Delimited) log format
  • [by request] Added support for XMail Spam log format
  • [by request] Added support for d/mmm/yyyy as dd/mmm/yyyy
  • [by request] Extended support if IPMon format to work as a syslog_required format
  • [by request] Extended support for a time format without leading zeroes
  • [by request] Added for WAP Log format.
  • [by request] Added support for MicroTech Image Maker Media Log format.
  • [by request] Added support for MicroTech Image Maker Error Log format.

Version 6.2.8b, shipped October 26, 2001

Bugs fixed in version 6.2.8b:

  • [by request] Fixed a bug where the pie chart labels were not appearing correctly.
  • [by request] Fixed a bug where the singlefilter filter option did not work properly when used with a multiple-subview view-- filters from one view could carry over to the next.
  • [by request] Fixed a bug where date graph bars were not clickable.
  • [by request] Fixed a bug where the User menu asked for the Administrative password!

New features in 6.2.8b:

  • [by request] Changed the maximum number of slices in a pie chart so it doesn't show more slices than there are rows in the table.
  • [by request] Shaded the pie chart legend square of the "N other items" row of the table so it matches the shading of the gray/white area of the pie charts.
Version 6.0b4, shipped February 26, 2000

Bugs fixed in version 6.0b4:
  • [by request] Improved "individual hits" view to work better with a variety of database structures.

  • [by request] Fixed bug where logs with an AGENT field (ex. WebSTAR) could cause Sawmill to create "untitled" fields every time the database was built.

  • [by request] Fixed bug in session statistics which could cause a crash when any of the four session-based views were viewed.

  • [by request] Fixed bug with MacOS networking code which could cause long pages to be truncated.

  • [by request] Fixed a couple bugs in MacOS AppleScript support which could cause AppleScript commands to be ignored, or if not ignored, to generate errors.

  • [by request] Improved "individual hits" view to work better with a variety of database structures.

  • [by request] Fixed bug where the maximum number of rows in tables would be reduced to 7 when using Generate HTML Files.

  • [by request] Fixed highly inefficient use of memory during Generate HTML, where Sawmill would consume as much as 1 Meg per file generated. It now consumes almost nothing.

  • [by request] Fixed bug where the "go to filter number" filter action went to the wrong filter.

  • Fixed bug where zero-size pie charts would hang Sawmill. Sawmill just omits them now.

  • Fixed bug where entering a Sawmill Info Directory pathname or a Sawmill Server Directory pathname without trailing / would cause problems if that directory didn't already exist.

  • Fixed bug where Sawmill would print "Content-type: text/html" as part of each progress line in CGI mode using Internet Explorer.

  • Fixed a bug where Sawmill would incorrectly offset date/times by an hour if daylight savings time was active on the system it was running on.

  • Fixed bug where WebSTAR dates were bring processed incorrectly.

  • Fixed bug where Sawmill would save the current page being viewed in the Preferences or the Default Configuration, and would use it as the starting page for new configurations, even though in the case of the Preferences, that page didn't exist in normal configurations.

  • Fixed bug where selecting a log file with a corrupt modification time could cause Sawmill to crash.

  • Fixed bug where the images were not appearing correctly in the "paths through the site" view.

New features in version 6.0b4:
  • [by request] Reworked online documentation slightly; added search feature.

  • [by request] Added four new filters which can be used together to parse log formats where entries are spread across multiple lines and/or include a session identifier. The filters include regular expression parameters which use parenthesized sections to pull out the session IDs and one field. Collections of these filters can be used to parse out multiple fields from the same line, or from multiple lines.

  • [by request] Added new date format: ddmmmyy (ex. 07Jan00)

  • [by request] Added support for new date/time format: "mm/dd/yyyy hh:mm:ss"

  • [by request] Added support for O'Reilly Windows Log Format.

  • [by request] Added Serv-U FTP Log Format.

  • [by request] Added a "remove all limits" link to offline pages with limits.

  • [by request] Change the "top" page of an offline statistics directory to auto-load the top page of statistics.

  • [by request] Added a "date_offset" option which adds or subtracts a specified number of hours to every date in the log file as it reads it. This is useful for adjusting time zones in a log file on the fly.

  • [by request] Improved the statistics display in the case where a single bottom-level item is the limit, and the view matches that item's field (for instance, with a page limit of "index.html" and a view of "page statistics"). Previous versions would attempt to find "subpages" of the page (since that's normally what the view shows), and would generate a confusing message when they failed, claiming no data matched the limit. The new method displays a single-row table containing only the single limit item.

  • Changed default permissions for UNIX files to work correctly in the case that a CGI installation is also being used by a non-root user in web server mode or from the command line.

  • Displayed "-" for percentages in cases where the total was 0.

  • Added a note at the top of average weekdays/hours pages describing how the latest data point is ignored.

Version 6.0b3, shipped February 10, 2000

Bugs fixed in version 6.0b3:
  • [by request] Fixed bug in multithreaded locking which caused database builds to freeze.

New features in version 6.0b3:
  • [by request] Added a warning about the performance hit you get when you click "flatten."

  • [by request] Added a SawmillAccessLog file in SawmillInfo, which logs all hits on Sawmill's built-in web server. This can be useful for determining what who has been accessing Sawmill, and what they have been doing.

  • [by request] Added a Log Out button to the Administrative Menu.

Version 6.0b2, shipped February 9, 2000

Bugs fixed in version 6.0b2:
  • [by request] Fixed bug with old-style IIS format where dates were processed incorrectly.

  • [by request] Fixed bug with log formats that used 2-digit dates, where the dates were interpreted as 1900-offset

  • [by request] Fixed bug where clicking a drive letter in Browse under Windows would just go back to the drive letters list.

  • [by request] Fixed a few bugs which could possibly cause crashes.

  • [by request] Fixed bug where reading from a regular expression log source would terminate early if one of the files was 0-length.

  • [by request] Fixed bug where when using a regular-expression-based log source, Sawmill would not properly close the log files it opened. On some systems, when many log files were being processed, this would lead to a state where Sawmill could no longer open files without an error. The database build would terminate with an error, and nothing further could be done until Sawmill was restarted.

  • [by request] Fixed a bug where the "collapse" value for a database field was ignored.

  • [by request] Fixed bug where agent data was getting scrambled.

  • [by request] Fixed bug where the checkboxes in Edit Limits would not be properly checked when you returned to that page.

  • [by request] Fixed bug where configuration passwords did not work (people could access the configurations in spite of the password settings).

  • [by request] Fixed a bug where clicking the Admin button on a configuration editing page (or a Preferences or Default Config page) did not save the most recent changes.

  • [by request] Fixed a problem with the "Go To Administrative Menu" link at the bottom of statistics pages, where some browsers, under some circumstances, would stay on the same page when that link was clicked.

  • [by request] Fixed bug where scheduled jobs could generate output to stdout on UNIX systems.

  • [by request] Fixed bug where scheduled jobs could cause an additional web server to be started, and an additional period-task thread, possibly causing a crash and certainly causing unwanted error messages and excessive resource usage.

  • Fixed bug where items would sometimes be linked when they shouldn't, ex. referrer http://myhost.com/dir/ would be linked in default configuration even though there was no information available below it.

  • Fixed a few cosmetic errors in the documentation.

  • Fixed bug where various delays were too 1000 shorter than planned on Windows, possibly hurting performance (for instance, instead of delaying 30 seconds between checks of the Scheduler, Sawmill was delaying only 30 milliseconds).

  • Fixed bug where browser numbers were not clickable to zoom in further.

  • Fixed bug where xref icons would appear in computed weekday and hour views, even though they're useless there.

New features in version 6.0b2:
  • [by request] Added four new views: sessions (summary), sessions (paths through the site), entry pages, and exit pages. The summary view is similar to the top section of the previous sessions view (which is now gone). The entry pages and exit pages show the pages visitors entered and exited the site at; that's very useful stuff which was hard to get at with Sawmill 5. The paths through the site shows the paths each visitor took through the site, page by page. It is grouped hierarchically, and can be selectively expanded and collapsed. It's a really neat view, and like the entry and exit pages, it contains information which was not easily coaxed out of Sawmill 5. Unlike the Sawmill 5 sessions view, which did an extensive database lookup and could take a very long time, this will take only a moderately long time to compute the session information, and after it's done it once, it caches it until the next time you rebuild the database. After you've computed the sessions once, it takes only a few seconds to generate a sessions page. You do, however, need to compute separate session information for each combination of limits. Sawmill handles it all automatically, computing the session information when it is needed, and deleting it when it's stale.

  • [by request] Enhanced the regular expression log source option, so it scans the directory you specify hierarchically. This means that it is now possible to combine log files from multiple directories, by putting them in some other directory and scanning it.

  • [by request] Added a "pid" file to the SawmillInfo directory, which on UNIX systems contains the process id of Sawmill.

  • [by request] Improved Windows IP number selection, so Sawmill always shows 127.0.0.1 as an option.

  • [by request] Disabled the "all views" view by default, since it takes so long to generate.

  • [by request] Added "Show Field Limits editor" option.

  • [by request] Changed the "collapse" value of the referrer field to 1. This means that all previously-second-level referrers are now first-level referrers. This makes for a much more meaningful top-level referrer report, with all the main referring sites listed (previously, you had to click on http:// to see them).

  • [by request] Added a default filter to convert "-" referrers to "{no referrer specified}".

  • [by request] Improved Edit Limits interface so that the "no limits" checkbox is ignored if anything else is checked.

Version 6.0b1, shipped December 21, 1999

Bugs fixed in version 6.0b1:
  • [by request] Fixed bug where the "kb %" link in the toolbar actually controlled the "kb" column.

  • [by request] Fixed bug in scheduler which prevented schedules from "firing" in some situations.

  • Fixed bug where entering a filter value with a space in it would cause everything after the space to disappear if you collapsed and expanded the filter.

  • Fixed bug where documentation pages containing '%' characters (ex. the log formats page) appeared incorrectly.

New features in version 6.0b1:
  • [by request] Changed the way Sawmill computes visitors. There is now a "visitor id" log field and a "visitor id" database field. Ideally, this field is unique for each visitor. In a normal log, however, that information is not available, so the hostname is used. In logs where a true visitor id *is* available (through a cookie mechanism or something similar), a filter can be created to set the visitor id log field appropriately. The visitor id database field is cross-reference to everything that needs visitor information (specifically, it is included in the page/datetime/host/referrer group). Sawmill now includes visitor information directly in the database, using the visitor id database field to keep track of unique visitors during log processing. This means that visitor information is no longer "computed"-- it's available directly from the database in the same way that page views and bandwidth information have always been. The performance cost of showing visitor information has been eliminated, so it is shown by default. The hostname field has also been collapsed to just the top two levels by default, since the lower ones are not really needed anymore (they used to be used to compute visitor information).

  • [by request] Added a new Log Source type: "Get log data from a single file on an FTP site." This is just the beginning of the FTP implementation planned for 6.0; among other things, it will include two other log source types, one to get all files in a directory on an FTP site, and another to get all files matching a regular expression in a directory on an FTP site (incidentally, there will also be a new log source type for getting all files in a particular local directory, and an option to make that recursive). The other FTP code will all be built on what's here, though, so it should be pretty easy to get the rest working once this works.

  • [by request] Added a "disable 'all views' view" option and a "disable 'all views' view offline" option to allow the "all views" view to be suppressed.

  • [by request] Added support for Doorstop log format (a variant of WebSTAR format)

  • [by request] Added support for CiscoPix / Redcreek System Message Viewer format.

  • [by request] Improved support for WebSTAR FTP logs.

  • Improved support for WebSTAR, IIS, and Netscape logs so Sawmill identifies log fields even if it doesn't know what they mean.

  • Improved language modules by adding a QUOTEVAR operation which directly quotes the value of its variable parameter. This is used in the filters now so that regular expressions appear correctly in the English filter descriptions, even if they contain language module metacharacters like [ and ].

  • Added "copy_field_regexp" filter statement type, which copies a substring of one field to another field, choosing the substring based on a regular expression match.

  • Improved filter format and menu options to be more intuitive. Functionality is unchanged (except for addition of new "copy" filter), but the menu options are much more verbose, and the various variables have names now which are referred to by the menu options.

  • Changed the host field default to go only two levels deep, and not include bottom-level hostnames. The additional information was necessary in version 5 because it traversed the host field looking for visitor information. This is no longer necessary in version 6 because the dedicated "visitor id" field provides visitor information.

  • Added a "copy field" filter which copies the value of one field to another field.

© 2014 Flowerfire | Copyright | Privacy Policy | License Agreement | Terms of Use | Contact | Feedback | About
Sawmill Software
Sawmill Software
Back to Sawmill Home