Sawmill - Universal log file analysis and reporting

This is the version history for Sawmill 7. The Sawmill 6 version history is here.

Version 7.2.15, shipped May 16, 2008

Bugs fixed in version 7.2.15:

Fixed bug in the Helix Universal Server (Style 5) Log Format where the File Time field was being treated as milliseconds while the Sent Time field was being treated as seconds. According to documentation at real.com, both fields contain times expressed in seconds.
Fixed a bug which could cause a crash (which would appear in the Sawmill GUI as a hang) when autodetecting data on an FTP or HTTP server.
Fixed memory leak which could occur in various circumstances; the specific known circumstance occurred when building a database from a profile with more than 1500 log sources, which caused more than 1GB of memory to be used.
Fixed a bug where the number of visitors could be overstated by 1 in Microsoft Media Server log format.
Fixed a bug in the Critical Path POP3/IMAP plug-in which could cause an error when creating a profile.
Fixed a bug where the "day of year" and "week of year" fields split the day at 23:00, instead of 0:00, on days under daylight savings time.
Fixed a bug subtable Table options were not saved and restored properly, when editing a "table with subtable" report in the report editor.
Fixed a bug in RACF Security log format, which prevented it from importing the final record in a file.
Fixed a bug in RACF Security log format, which prevented it from importing lines where the username contained no spaces.
Fixed a bug IronPort C-Series parsing, where SBRS rejects were not reported.
Fix incorrect reporting of sessions in the Flash Media Server plug-in by only creating session events when x-event eq disconnect and x-category eq session.
Fixed bug in Sidewinder analysis (logged to firewall) which caused incorrect dates when there was a date= field listed.
Fixed a bug where certain filters (especially, ORs of "within" filters) could cause main table scans, when they could have been handled by xrefs. This made some filtered reports slower than they should have been.

New features in 7.2.15:

Enhanced Sawmill.app (on Mac) to detect when there is a running installation of Sawmill already, and give an appropriate error message (rather than hanging while it waits to bind to the port).
Deprecated the "maximum CPU usage percent" option. The option never worked very well, and has done absolutely nothing since Sawmill 7.0.0, so it serves no purpose. Instead, use operating system priorites to minimize the impact of Sawmill's CPU usage on other processes.
Added support for CP Secure Content Security Gateway log format.
Added support for a new version/variant of Aruba Wireless Switch.
Added tracking of "Context" lines in Citrix Netscaler log format.
Added support for Unix Auth log format.
Added support for Unix Cron daemon log format.
Added tracking of VOF quarantine lines in IronPort C-Series logs.
Added reporting of Amavis information in Postfix logs.
In the Kiwi YYYYMMDD Comma Syslog plug-in, added stripping of double quotes from around the syslog message since these can break autodetection. If the message is quoted, the plug-in also now changes doubled double quotes back to single double quotes. Doubling is the way Kiwi escapes them.
Added support for a new FortiGate 100 Firewall format with additional fields to the FortiGate Comma Separated Log Format plug-in.
Added support for Symantec Gateway Security Log Format (via syslog).
Added alias domain reporting to Microsoft Exchange 2000 log format.
Added support for automatic charset conversion of search engines which do not use UTF-8 in their search URLs (specifically, Yandex).
Added reporting of MailScanner lines in Postfix log data.
Added a new plug-in to support the SNARE Epilog Collected Oracle Listener log format. The plug-in was contributed by a Sawmill user.
Expanded the plug-in for the Nortel Meridian 1 Automatic Call Distribution (ACD) log format to include some additional fields from the logs and an additional graph in the Date/Time reports.
Added session analysis to the Flash Media Server plug-in for the purpose of reporting the Maximum Concurrent Connections.
Added support for the Users field and a Unique Users numeric field to the Proxy Plus log format plug-in.
Added support for Tipping Point SMS Log Format.
Added reporting for ARP request and ARP reply lines in Cisco VPN Concentrator.
Added support for AspEmail (Active Server Pages Component for Email) log format.
Fixed a problem with Cisco VPN Concentrator log format, which caused certain "disconnected" lines to be ignored.
Added support for tracking/reporting of the usr field in SonicWall format.
Changed label for the Barracuda Spyware Firewall Log Format plug-in to Barracuda Spyware Firewall / Web Filter Log Format to reflect new product name. Added support for standalone (no syslog header) format. Added support for lines where the action is "sniff" instead of "httpscan". Added Action report.
Made extensive changes to the Anti-Spam SMTP Proxy (ASSP) log format plug-in. Messages, which are described on multiple lines of the log, are now captured in one database entry so reports are more clear and counts are more accurate. These changes apply to log formats for 1.3.3.1, 1.3.3.8 (and in between, presumably, though they have not been tested). Reports for earlier versions of ASSP that have a different log structure are not changed.
Enhanced the JBoss application server plug-in to support a slightly variant.

Version 7.2.14, shipped March 26, 2008

Bugs fixed in version 7.2.14:

Fixed a bug in the IceCast Log Format plug-in where the User Agent field was not being set causing the fields that are derived from it, such as Web Browser, to be empty.
Fixed a date/time parsing bug in Barracuda Spam Firewall, where some lines were reverting to the syslog collected date/time instead of the Barracuda's date/time.
Fixed a bug in the FirePass SSL VPN Log Format caused by an incorrect variable name. The bug would only have been seen if lang_stats.cfg did not have the firepass_ssl_vpn status code mapping section.
Fixed a memory leak which could cause very high memory usage when building a MySQL-based database from a database with many unique values in one or more fields.
Fixed a bug in the Unix Syslog With Year plug-in where the syslog message was being lost.
Fixed a bug which could cause an error in various circumstances (but usually when building a database) on 64-bit Windows, when one of the mapped files in the internal database exceeded 2GB. This is rare, but can happen to the indices if the "main table segment size" option is set to a very high value.
Fixed a bug in the parsing regular expression where the report of multiple Stats or square brackets in the client_info field would cause the entry to be rejected.

New features in 7.2.14:

Changed the IceCast Log Format plug-in to get the duration in seconds from the duration field instead of calculating the duration from the size and an assumed speed. Apparently the duration field was not available at the time the plug-in was first created so a workaround was used.
Enhanced Ironport C-Series plug-in to extract more information about antivirus scanning.
Added support for charset conversion on 64-bit Windows.
Enhanced Tipping Point IPS log format to handle log lines generated by the 2.4.3 firmware revision.
Added support for OpenVPN log format.
Added support for CRYPTO lines in Cisco PIX/IOS/etc. format.
Added a new "Save To Menu" button to the Reports page, to save a filtered report directly to the reports menu.
Added support for a format variation with a date as well as a time to the Windows 2003 DNS Log Format plug-in and increased the flexibility of the autodetect regular expression.
Added support for Tipping Point 2.5.3 log format.
Improved performance of hierarchy builds for MySQL databases. With this change, the time to build the hierarchies for a specific database with 16 million unique IPs dropped from 2:15 hours to 0:40 hours.
Added a new profile option, "Use Overview For Totals." This option controls a recent new feature, which computes the Total rows of report using an Overview report. In recent versions (since 7.2.10), this option has always been turned on; with this version, it is optional, and disabled by default. Turning this option on gives correct totals of "unique" and calculated columns in tables, and correct percentages for unique rows, if they are shown, but can severely hurt performance for some very complex reports, when the "remove parenthesize items" option is turned off for the report. Even under normal circumstances, this option makes two times slower. So as of this version, this option is off by default, and the Totals row is computed by summing the table by default. When this option is off, unique columns will show a dash in the totals row and calculated columns will show a zero.

Version 7.2.13, shipped February 21, 2008

Bugs fixed in version 7.2.13:

Fixed a bug which could prevent scheduled tasks from running.
Restructured the plug-in for the NetScreen Log Format in order to improve performance and fix a bug where a variable that was set if the log line matched the supported format was being accessed whether it matched or not. Also improved performance by omitting the message field where the message consists of key/value pairs that are extracted into other fields.

New features in 7.2.13:

Enhanced "LogSat SpamFilterISP Log Format B500.9" to support a slight variant.
Renamed the field/report message_id to queue_id in the Postfix Log Format plug-in and added a report for actual field actually called message-id in the logs. Improved the efficiency of mapping from the spamd mid field the the postfix message-id field. (There was existing limited support in this plug-in for reporting on spamd along with Postfix where they are logged to the same syslog.)

Version 7.2.12, shipped February 15, 2008

Bugs fixed in version 7.2.12:

Fixed two bugs in the bytes and stream bytes calculations in the Flash Media Server Log Format plug-in where typos caused an error in the results and caused an error message if the log didn't have the c-client-id field.
Fixed a bug in NetCache NetApp 5.5 format, where date/time values would not be parsed correctly for MySQL databases.
Fixed a bug which could cause a checksum error when accessing the web interface, if there were unknown CFV files in the templates folder. This could happen when upgrading from an earlier version of Sawmill.
Fixed a bug with Apache Combined which would show all 0's if hits was not selected.
Fixed a bug with Ironport (mail) format, where multi-RID messages would be reported as a list of RIDs, instead of being reported as each indiviual RID.
Fixed parsing problem in cases where parse_only_with_filters was false, but parsing filters use accept/collect.
Fixed bug where backslashes in wildcard expressions were treated as escapes, instead of being treated as literal backslashes.
Fixed a bug in Cisco NetFlow (flow-export) format, where the total rows were incorrect for some fields, on some platforms.
Improved the efficiency of connection tracking for the Maximum Concurrent Connections report in the Cisco Wide Area Application Services (WAAS) TCP Proxy Log Format plug-in because it was causing performance problems during the database build.

New features in 7.2.12:

Added a mime_types.cfg file in LogAnalysisInfo/miscellaneous, which lists the filename extensions and corresponding MIME types recognized by the built-in web server (previously, this was hard-coded and uncustomizable).
Improved the efficiency of tracking bytes and stream bytes totals in the Flash Media Server Log Format plug-in because the current method caused performance problems during database build. The old method used nodes and the new method uses set_collected_field where collected entries expire if they are not accepted. The trade off is the small risk of skewing results if there are very long connections. The number of log lines after which to expire the collected entries can be adjusted.
Added support for the Mirror Image Flash Media Server Log Format.
Added support for Bomgar Box log format.
Fixed bug where "-v f" output would generate an error if log filters used replace_first() or replace_last().
Added support for FirePass SSL VPN Log Format.
Added support for Cisco IPS log format.
Enhanced SafeSquid plug-in to handle the new Extended format from 4.2.1+.
Added support for Sophos Web Appliance.
Added facility/severity/mnemonics fields to Cisco PIX.
Added a report for Maximum Concurrent Connections to the Cisco Wide Area Application Services (WAAS) TCP Proxy Log Format. This report is based on keeping track of a count open connections for each device.
Added support for McAfee Secure Messaging Gateway (SMG).
Added support for non-AM/PM times in Windows NT4 Event Log Format (save as-CSV).
Added support for Guardix Log Format (IPFW).
Added support for a new variant of IIS SMTP W3C logs.
Enhanced Exim 4 log format support to handle a variant.
Fixed a bug in Syslog NG (no zone) to remove leading space from syslog message.
Greatly improved performance of "NOT" report filters in most cases, when using the internal database. This is particularly important because as of 7.2.11, any report which omits parenthesized items (which is most of them) uses a "not" filter implicitly. This especially affects large databases. In one example, it increased the speed of the "day of week" report from 5 minutes to 12 seconds.
Enhanced dumpel log format to show event code categories and descriptions for common event codes.

Version 7.2.11, shipped November 30, 2007

Bugs fixed in version 7.2.11:

Fixed a bug where the "omit parenthesized items" option did not work for the session users report.
Fixed a bug where log data with repeated $ characters in it could cause a crash, if the "f" option was used for -v for a command-line build.
Fixed a bug which could cause an error when rebuilding database hierarchies with "-a rdh".
Fixed a bug which could cause the error "Expression not supported by field limits (OR across fields)" when using certain advanced filter expressions in the web reporting interface.
Fixed a bug where matches_regular_expression would not set $N variables above $M, if $M was not defined by the expression, e.g. through the use of ()? or ()*.
Fixed a bug in the "beta" IronPort plug-in, which could cause very high memory usage during log processing.
Fixed a bug which would cause incorrect durations to be reported when the "date offset" option was used with Shoutcast W3C.
Fixed bug where DNS lookup would attempt to lookup "..." as though it were an IP address, resulting in DNS errors.
Fixed a bug in "create many profiles" which would cause an error like "Couldn't find node 'clone1' in profiles" if the profiles to be created did not already exist.
In the Flash Media Server Log Format plug-in, corrected calculations for sc_bytes, sc_stream_bytes and cs_stream_bytes, based on the way cs_bytes was calculated. Because the log keeps a running total of these values, the previous accumulated value must be subtracted from the current value for each event to prevent a huge, incorrect total from being shown in reports. Also restored the fix where the filters that do these calculations use c_ip where c_client_id is not available.
Fixed a bug which caused FTP log source error messages in cases where the server split single control response lines into multiple packets (uncommon).
Improved the efficiency of the bytes and stream bytes calculations in the Flash Media Server Log Format plug-in because they were hurting the performance of database builds.
Made autodetection more restrictive in the Apache/NCSA Combined Format With Cookie Last plug-in in order to prevent other log formats from autodetecting as this one.
Fixed bug with flash media server logs which could cause an error during database build, if the stream_duration field was not checked when creating the profile.
Fixed bug which would cause an error with Netscape logs when there was no page field logged.
Fixed a bug where session reports did not work in a profile without a "page" field, when using a MySQL database.
Fixed bug which could cause crashes during long log processing, or during long database updates, involving many files.
Fixed a bug where clicking Browse would cause an error if a CSV filename was in the field.

New features in 7.2.11:

Enhanced the Barracuda Spyware Firewall plug-in to extra domain, category, and username fields, when availiable.
Enhanced Net-Acct to handle a variant.
In Syslog NG, added support for dates in the format "2007-08-23T15:02:28+02:00".
Added support for Windows Event Log (comma or tab delimited, no am/pm, 24h & ddmmyyyy) Log Format.
Enhanced NetCache NetApp support to recognize version 6 logs.
Added support for hMailServer log format.
Improved Filezilla Server format to support single-digit months and days.
Renamed all formats and plug-in files with "beta" in the name because process of identifying stable plug-ins is changing.
Enhanced ASSP log format: Added support for 1.3.3.1 logging (almost complete rewrite); added support for old logging style.
Enhanced NetCache NetApp 5.5+ format to report streaming log data better.
Added support for the Cisco Wide Area Application Services (WAAS) TCP Proxy Log Format.
Moved newly added reports for derived fields to the appropriate groups in the report menu for the Apache Custom Log Format.
Added an option to display bytes using base-10 (1000-based) units, rather than base-2 (1024-based) units.
Added support for a new type of node file, ending with .cfga ("configuration group additions"), which is layered on top of its similarly-named .cfg file, to automatically create a node different from the original CFG file, without requiring editing of the original CFG file.
Added auto-expansion of {==} sections in local log source pathnames.
Changed the name of the log field error to error_message in order to fix a bug in the Apache Error Log Format plug-in that was introduced by a UI change that causes an error when log fields have the same name as Salang functions.

Version 7.2.10, shipped August 04, 2007

Bugs fixed in version 7.2.10:

Fixed a bug in the "beta" plug-in for "Juniper Secure Access SSL VPN Log Format" where user agent information was not being extracted properly.
Added support to the Kiwi Syslog (ISO/Sawmill) plug-in for repeated lines if Unix Syslog (only one format variant so far) is logged to Kiwi Syslog.
Fixed a bug where database deletion (and profile deletion) would fail on Unix or MacOS systems, if the database directory contained a file starting with a period.
Fixed a bug which would cause a crash during log processing, when processing gzip files which were corrupt in certain ways (valid gzip files cause no problems).
Fixed a bug where {= =} or $ sections were not handled properly in the output directory option of the Scheduler, while generating HTML reports.
Fixed a bug which would cause an error when clicking an individual session in the Individual Sessions report, when using a MySQL database, with any log format where the session visitor ID field is not called "hostname".
Fixed a bug which could cause an "empty node error" when processing Symantec AntiVirus Corporate Edition logs, if the encrypted time field was less than 12 characters long.
Fixed a bug in the BETA IronPort plug-in, where if the log was not generated through syslog, and contained a timestamp header, all entries would be discarded.
Fixed a bug in the BETA IronPort plug-in, where if there was no "log" tag in the data, all entries would be rejected.
Fixed bugs in the Watchguard XML Log Format plug-in: Fixed bug in filter add_dstname_arg where wrong field name was used and url was not found. Changed filter to not concatenate dstname and url (arg) if either is empty. Mapped field names rcvd_bytes and sent_bytes to recv and sent since at least one format variation has these instead.
Fixed a bug which caused an error when using { or } characters in wildcard expressions.
Added BETA support for Visonys Airlock log format.
Fixed a bug where database builds could repeat, in CGI mode, building over and over, if Talkback was turned on.
Fixed the "beta" Barracuda spam firewall plug-in to track logging devices.
Fixed a bug with Sawmill's encoding of MIME emails (HTML reports) which could cause Amavis (and possibly other spam filters) to flag Sawmill's report emails as spam.
Fixed a bug where the filters for Remove Database Data were not logged correctly to TaskLog.
Fixed a bug which would cause an error ("Unable to delete file seendata") when doing a multiprocessor build, if the dataset was very small (less than the size of thread_data_block_size, which is 1MB by default).
Fixed a bug where the session duration did not match the play duration, in Microsoft Media Server analysis, if there was a custom log filter on the cs_uri_stem field.
Fixed a bug in Microsoft Media Server plug-in, where session durations would be reported incorrectly if a date offset was specified in the profile.
Fixed a bug which caused an error when building a MySQL database, if the profile contained a field called connection_id.
Fixed a bug which caused the TaskLog entry from a database build or update to report the bytes processed in the last log source, rather than the full bytes processed.
Fixed a bug where profiles would not be removed properly from non-administrative users, if the username contained unusual characters, and command-line authentication was used.
Fixed a bug which causes an error "Expression not supported by field limits (OR across fields)" when using complex filters on the Overview.
Fixed a bug which could cause duplicate rows in report tables, for very large datasets.
Fixed a bug which were command line execution would fail quietly, without doing anything, if there was no valid license installed.
Fixed a bug in the beta NetScreen format where a key value pair on a line with no logging category could be placed in the category field.
Fixed/enhanced the beta Postfix format to reduce memory usage, and improve performance with large datasets.
Fixed bug in the beta IIS SMTP W3C Log Format where some data could carry over from earlier connections for the same Client IP.
Fixed a bug in the IronPort Log Format (BETA) where aborted entries were not being accepted.
Fixed problems with rekeying and duration tracking in "CT Mod 10 Nortel Contivity Log Format". (Note that there is an improved version of this plug-in called "Nortel Contivity Log Format (BETA)".)
Fixed a bug which would cause an error in a report, if you removed the column in the Report Editor which was the "sort by" column.
Fixed a bug which could cause errors when exporting CSV reports from the web interface, in CGI mode.
Changed name of plug-in "Microsoft ISA WebProxy Log Format (W3C)" to "Microsoft ISA Server Log Format (W3C)" to reflect correct product name (ISA Server replaced ISA Proxy). Fixed bug in and simplified autodetection. Added new possible W3C fields to groups to organize reports menu.
In the Quicktime/Darwin Streaming Server Log Format, changed type of x_duration and all fields with pkt in the name (packet fields) to float to handle large values.
Fixed a bug where Windows error messages containing "\" would be displayed without the "\", or Windows error messages containing "\t" would be displayed with "__TAB__" in place of "\t".
Fixed a bug where the Log Detail showed 12-hour times instead of 24-hour times, when using a MySQL database.
Fixed a performance problem where processes waiting to access files locked by other processes would use CPU, instead of waiting quietly.
Fixed a bug which could cause crashes during report generation.
Fixed a bug which would cause authentication failures on Windows, if the password contained a & character.
Fixed a bug where the number of session users was miscounted in Session Overview report, resulting in one fewer one-time users being reported than there actually were, or one fewer repeat users being reported than there actually were.
Fixed a bug where values formatted with "duration" format (the long duration format) would end in commas if there were 0 seconds.
Fixed a bug in IronPort where log entries would be rejected if the "log file" field contained a space.
Fixed a bug where static report could be generated while a database was being updated, resulting in erroneous reports.
Fixed a bug with Blue Coat W3C, which caused a bogus report to be created when certain fields were missing, causing an error when creating a new report.

New features in 7.2.10:

Added report group for "Security" related reports to menu to beta "SonicWall or 3COM Firewall Log Format".
Added beta support for version 2.9.8 to the DansGuardian 2.9 Log Format.
Added beta support for Sun ONE Directory Server 5.2. It is greatly enhanced from the Netscape Directory Server Log Format, but should continue to work with Netscape Directory Server 5.1.
Enhanced IIS SMTP W3C Log Format: Added support for a new log format variant; Added operation and server_response fields and connect/disconnect counts.
Added "BETA" support for the Foundry Networks Log Format. This plug-in is based on the Foundry Networks BigIron plug-in and maintains support for BigIron while adding support for ServerIronXL.
Added "BETA" support for the Merak SMTP Log Format. Support is added for a format where the date is taken from the log file name and is not found in the log. Backward compatibility is maintained with the version supported by the existing Merak SMTP plug-in.
Enhanced praudit "BETA" plug-in to handle a single digit day in the date and simplified autodetection.
Enhanced the "beta" plug-in for "Juniper Secure Access SSL VPN Log Format": Improved session tracking by identifying more events which could be considered the end of a session. This will result in a more accurate "Maximum Concurrent Sessions" number.
Enhanced the plug-in for the Aventail Web Access Log Format to allow syslog and to strip layered syslog entries in the case where Unix Syslog logs to Kiwi Syslog. (Non-syslog logs are still supported by selecting "no syslog header" as the syslog type.)
Enhanced Watchguard XML plug-in to handle a few new fields.
Added support for ichain format.
Added support for a slight variant of Cisco VPN Concentrator.
Added "beta" support for the ipop3d Mail Daemon Log Format (BETA).
Added support for some types of "crashinfo" events for Cisco PIX/ASA/Router/Switch Log Format (BETA).
Improved FreeRADIUS to support all-capital month names.
Enhanced support for the Firebox x1000 format (among possible others) in the Watchguard Log Format plug-in. More types of TCP flags and lines with multiple TCP flags are now supported. A file with flags on all lines will now autodetect. The field "parameter" is now called "flags" to reflect its actual use.
Added support for version 5.2 to the Aladdin eSafe Sessions Log Format v5 plug-in and split the field "File Name\Mail Subject" field based on value of File Type, that is, if there is a file type, assume it is a file name, otherwise it is a mail subject.
Enhanced the Zyxel Firewall WELF Log Format to support newlines in the "msg" field. Without this support, information, such as Anti-Virus info, that followed the "msg" field was lost.
Increased number of lines examined during format auto-detection to 100 in "Oracle Listener Log Format".
Enhanced Symantec AntiVirus Corporate Edition plug-in to rewrite several additional fields to human-readable values.
Added enhanced error detection and reporting during Send Bug Report, so errors contacting the SMTP server to send a bug report, or other errors during the process, are reported in the web browser page when the bug reportis submitted.
Enhanced the "Novell iChain Extended (W3C) Web Server Log Format" plug-in.
Enhanced the Amavis log format plug-in.
Enhanced the Apache Combined (syslog required) log format plug-in to handle a slight variant.
Added parsing of Anti-Spam and Anti-Virus log lines to the NetScreen Log Format (BETA).
Enhanced Filezilla Server format to handle a new variant.
Enhanced IIS SMTP W3C format to include bytes transferred.
Changed field name "hits" to more accurate "events" in "Oracle Listener Log Format".
Added beta support for the IBM Tivoli NetView Log Format.
Added support for SurfControl "URL BLOCKED" entries to the beta NetScreen Log Format.
Added support for a new variant of EIMS SMTP (24 hour) Log Format.
Enhanced the "beta" postfix plug-in to handle a slight variant.
Improved Limelight plug-in; added better field labels.
Enhanced the "beta" IIS SMTP W3C Log Format to support another format variant. It now collects the server response from server response lines or from the sc-status field, whichever is available. It also now collects client-to-server bytes and server-to-client bytes from DATA and BDAT operations.
Enhanced the Juniper Secure Access SSL VPN Log Format (BETA) plug-in to allow users to configure the Host Checker rule and policy names for which passes and failures are counted.
Added "beta" support for the BroadWeb NetKeeper Log Format.
Added url field and associated derived fields and log filters to the Juniper Secure Access SSL VPN Log Format.
Added support for a log format variant that has no "Incoming client version" lines to "CT Mod 10 Nortel Contivity Log Format". (Note that there is an improved version of this plug-in called "Nortel Contivity Log Format (BETA)".)
Added support for a log format variant that has no time stamp to "Nortel Contivity Log Format (BETA)". (Note that there is an earlier version of this plug-in called "CT Mod 10 Nortel Contivity Log Format".)
Enhanced the Juniper Secure Access SSL VPN Log Format (BETA) plug-in to allow Host Checker rule or policy pass events to be tracked that are not explicitly in the log. This feature relies on user configuration of LogAnalysisInfo/rewrite_rules/host_checker.cfg. Also customized Host Checker reports.
Added expansion of Salang expressions in report headers and footers.
Added support for %I and %O LogFormat directives for Apache Custom format.
Added suport to the IronPort Log Format (BETA) for a format variant from Async OS version 4.6 which does not label lines with "Info:" and and "Warning:", at least with syslog. (Note that 4.7 was already supported.) Added rekeying of entries to avoid losing syslog info. Added handling of rewritten MIDs to pick up more antivirus info.
Added support to the Microsoft Media Server Log Format for a format variant where the field cs-uri-stem has been renamed (more accurately) cs-url.
Added expansion of Salang expressions in the "-of" command line option. For example, -of "/reports/report_{= replace_all(substr(epoc_to_date_time(now()), 0, 11), '/', '_') =}" will generate a file name with today's date at the end of it.
Added support to the Juniper/Netscreen Secure Access Log Format (BETA) for TCPPkt lines.
Enhanced Clavister Firewall support to handle differently-ordered fields.
Added the unique total to the sub-total and total rows of reports for unique numeric columns. The percent column, if it is visible, for unique numeric columns, will use this total instead of the sum of the values in the column. (Percents in the rows will not add up to 100% if there is overlap among the unique values, but the totals row will show 100% because the total will be the total of all unique values for that column.)
Enhanced the "beta" postfix plug-in to handle a new variant (2.4).
Added support for the GeoIP Organization database (with separate purchase and download of the database).
Added support for the GeoIP ISP database (with separate purchase and download of the database).
Added support for the GeoIP Domain database (with separate purchase and download of the database).
Fixed a bug where, in a report filter, an OR of empty "matches" expressions (e.g. '(page matches "zzz*") or (page matches "yyy*")' where there were no pages matching either) would select everything, instead of selecting nothing as it should.
Enhanced Barracuda Spam Firewall plug-in to extract much more information; also fixed some errors with extraction.
Added a server session timeout preference; web access to Sawmill times out automatically if the session is inactive longer than the specified value.
Enhanced memory management of "beta" IronPort plug-in, to prevent excessive memory usage on very large datasets.
Worked around a bug in Microsoft Media Server where x-duration could overflow, resulting in very large durations.
Optimized authentication command line so it is called only once per authenticated session (instead of once per page).
Enhanced Gene6FTP support to add support for DELE, RETO and REFR lines.
Added support for Wowza Media Server log format.
Fixed a bug with Blue Coat W3C format, where the time-taken field was reported in seconds, rather than milliseconds.

Version 7.2.9, shipped February 09, 2007

Bugs fixed in version 7.2.9:

Fixed a bug which could cause a crash when rendering tables with subtables, if there were more than two non-aggregating (string) columns.
Fixed a bug where the referrer field was not handled properly in Apache Custom data; the referrer field was not given the correct 1-to-3 hierarchy, and the "search engine" and "search phrases" reports were not created.
Fixed a bug in the Blue Coat W3C Log Format (ELFF) preventing correct parsing of logs with date and time fields instead of a localtime field in the W3C header.
Fixed a bug in the Blue Coat W3C Log Format (ELFF) where a GMT offset in the localtime field would be treated as the next field.
Fixed a bug which could cause an error if a Report Filter specified in the report editor contained a literal $.
Fixed a bug where scheduled tasks would compute the previous day based on the current time in GMT, rather than the current time, sometimes resulting in a "yesterday" report containing data from two days ago.
Fixed bug where month names were incorrectly displaying in non-English installations, if the month name was more than three bytes long.
Fixed a bug where action emails did not include To: or From: headers.
Fixed a bug where month names were not translated in graphs, in non-English installations.
Fixed a bug where capitalized month names were not correctly recognized in Free Radius logs.
Fixed a bug with the "beta" plug-in for Interscan Messaging Security Suite, which caused some fields to not be tracked in some variants of the format.
Fixed a bug with the "beta" plug-in for Postfix, where an Empty node name error would occur when building a database.
Fixed a bug which would cause an error on Solaris, when converting the charset of the log date, or of CSV export data.
Fixed a bug which could cause unreasonably high memory usage while generating reports, sometimes resulting in an out-of-memory error "while expanding fstring buffer".
Fixed a bug which would cause Sawmill to process Windows dump evt logs while consuming high disk and CPU resources, and seeing extremely slow processing times.
Fixed a bug where command-line options containing plusses (+) were not handled properly, resulting in command-line parsing errors.
Fixed a bug in the beta plug-in for Cisco PIX/ASA/Router/Switch Log Format where the "Destination service" was not being set.
Fixed a bug which would cause an error "Unknown variable 'lang_admin.action_emails.actions.remove_database_data'" when sending an action email for a remove_database_data operation.
Fixed a bug which could cause errors on Windows, when using a command line argument containing spaces, and ending with a slash. Among other things, this would cause an error when generating all reports to an HTML folder when the folder name contained a space.
Fixed a bug where in some cases, three-digit negative integers like -123 would be displayed as -,123.
Fixed a bug where log entries using EPOC format (seconds since 1970) were rejected as corrupt, if the date/time was older than 08/Sep/2001 18:46:40.

New features in 7.2.9:

Added "BETA" support for the Datagram Syslog Format.
The Cisco PIX/ASA/Router/Switch Log Format now handles negative connection numbers in "Built" and "Teardown" events.
Added "BETA" support for the Metavante CEB Failed Logins Log Format.
Changed the Juniper Secure Access SSL VPN Log Format to be a "Syslog Required" format. The standalone version is still supported by selecting the "No Syslog Header" syslog format.
Enhanced Microsoft Media Server log format, to track successful events (called "clips") as well as all events (called "events"), so averages shows averages over successful events.
Enhanced Interscan Messaging Security Suite log format to handle a slight variant.
Enhanced Snare format to extract a different format of "action" field, to handle Windows pathnames with drive letters in field value, and to handle slight variations in spacing between fields.
Added support for LRS VPSX Accounting Log Format.
Added support for a new variant of qmail-scanner logs, with dd/mm/yyyy format dates.
Enhanced ISA CSV format to track all fields, using proper names, and to track all numerical fields, and to group reports.
Added support for Limelight Flash format.
Fixed a bug where emails send by Sawmill on Windows would not display the correct time when viewed in some mail clients (especially Outlook Express).
Enhanced IronPort plug-in to handle a variant.
Enhanced Ironmail Spam format to extract IP information from the RBL record.
Added "BETA" support for the OpenBSD Packet Filter (tcpdump -neqttr) Firewall Log Format
Added support for IronPort logging to a syslog server, in the "beta" IronPort plug-in.
Enhanced the "beta" plug-in for "Juniper Secure Access SSL VPN Log Format": Added detailed reporting of Host Checker policy failures and numeric fields to count unique users failing each Host Checker rule. Added Sessions reporting to capture such information as maximum concurrent users.
Extended the existing MDaemon 8 log format plug-in to support a slightly different variant of the log format.
Added support for TACACS Accounting log format.
Enhanced Zywall plug-in to group reports, and to include a new Security group.
Improved the tracking of connections in Cisco PIX/ASA/Router/Switch Log Format (BETA).
Added support for additional types and formats of "Deny" events for Cisco PIX/ASA/Router/Switch Log Format (BETA).
Enhanced the Firewall-1 NG format to support many more fields, to group reports, and to track many numerical fields.
Added a beta plug-in for the Sidewinder Firewall Log Format which supports the Sidewinder 6.1.003 format in addition to already supported versions.
Enhanced MAILsweeper plug-in to support a new variant.

Version 7.2.8, shipped November 06, 2006

Bugs fixed in version 7.2.8:

Fixed a bug where date/time information could not be displayed as a pie chart; it would always appear as a bar or line graph, even if a pie chart was requested.
Fixed a bug where numerical fields of type float, and aggregation operator "sum", would be truncated to about 2 billion (2GB), or it would be wrapped to smaller and possibly negative numbers.
Fixed a bug where non-derived database fields with no corresponding log fields in Netgear Security Log Format caused errors during report generation.
Improved the "beta" plug-in for the Backup Exec Log Format (XML). The plug-in now handles CDATAs, multi-line tags and extremely large numbers of backed-up files.
Fixed a bug where the text of the Use Sawmill button in the windows Sawmill.exe program was not internationalizable.
Fixed a bug which would give a "no such directory" error after connecting to a password-protected share through the File Browser, if the share name contained a $ (the share was correctly connected, but was not correctly displayed on the next page).
Fixed a problem where Windows sometimes tried to start Sawmill in C:\, when running as a service; this caused it to fail. SawmillService.exe now explicitly changes to the installation directory before running SawmillCL.exe.
Fixed a bug where 24-bit screen depths were not reported in the Screen Depths report.

New features in 7.2.8:

Added "BETA" support for Separ URL Filter Log Format.
Added an improved "beta" plug-in for Helix Universal Proxy log format.
Added "BETA" support for Netgear FVL328 Log Format (logging to syslog).
Added "BETA" support for Web Sense Log Format.
Added improved "BETA" plug-ins for Argosoft Mail Server Log Format to support a format variation and dd-mm-yyyy dates.
Added support for version 200A to "beta" plug-in for FortiGate Log Format.
Added "BETA" support for Aventail Web Access Log Format.
Added "BETA" support for IBM Tivoli Access Manager WebSEAL Log Format.
Enhanced Sidewinder Firewall and Sidewinder Syslog formats to track more fields, and to group reports.
Added "BETA" support for SiteMinder Policy Server Log Format.

Version 7.2.7, shipped October 06, 2006

Bugs fixed in version 7.2.7:

Fixed a bug where times in log data with AM/PM specified would be misreported; noon-1pm would be reported as midnight-1am, and midnight-1am would be reported as noon-1pm.
Fixed a bug in handling of dd/mm/yy formats in log data, which could cause a crash if mm was greater than 12.
Fixed a bug in the send_email() function where the message had to end with \r\n.
Worked around a problem where some firewalls introduced an additional space in HTTP headers, causing problems for Sawmill when Sawmill was running in web server mode and was accessed through the firewall. Problems included repeated reloads of report pages and other pages in the web interface; pages would finish loading, and would then reload, repeatedly. Sawmill now removes this additional space automatically, which works around this type of firewall issue. This is known to be an issue in ISA 2004, and is probably an issue with some other firewalls.
Fixed a bug in the FAQ, present only in Sawmill 7.2.6, which caused the error "Unknown variable 'docs.faq.db.duplicate_profile.question'" when clicking FAQ.
Fixed a bug where log filter descriptions containing variables would not be expanded; the variables would appear literally.
Fixed a bug which could cause a crash during log processing, if using a plug-in which used "rekey" functionality, and which rekeyed a key to itself (uncommon, but could happen in imail format).
Fixed a bug which could causes crashes on some Linux systems, when building the xref tables for databases with "max" or "min" numerical fields.
Fixed a bug which would cause an error if all reports were deleted in the Report Editor.
Fixed a bug where recentdays:N and similar old-style filters, did not work from the command line, and failed with an error about not being able to find node "type".
Fixed bug in IceCast format which could cause negative or very large total_duration_96kpbs.
Fixed bug in Blue Coat W3C format which could cause data to be ignored, if a dataset switched in midstream from using date and time fields, to using a localtime field.
Fixed a bug in Exim 4 "beta" format which could cause the error "Unknown variable 'v.name_value_pairs' in expression while processing" while building a database.
Fixed bug in "Symantec Security Gateways Log Format (SGS 2.0/3.0 & SEF 8.0) (BETA)" log format; mapped key "Target" to url field to get url reporting.
Fixed a bug where if a log source contained a UTF-16 file, followed by an 8-bit file, the 8-bit file would fail to parse.
Fixed a bug in the "beta" Snare plug-in which caused lines to be rejected if they ended with a space.
Fixed a bug which could cause single lines of log data to be split into multiple lines during processing, if the line was more than 50kiB.
Fixed a bug where '+' characters were not properly converted to spaces, when displaying search phrases.
Fixed a bug where SMTP connections from Sawmill did not send an appropriate QUIT message, resulting in errors logged in the SMTP server.
Fixed a bug where "action" emails did not contain full information about the task performed.
Fixed several bugs which could cause error message parameters to be omitted, resulting in error messages with words or other values missing.
Fixed a bug in the WebNibbler log format plug-in which caused search engines and search phrases to be blank in the reports.
Fixed a bug which could cause a crash on 64-bit Windows, when displaying empty table reports.
Fixed a bug which could cause an error when generating a report, if a hard-coded report filter, or report element filter, contained an expression which could not be delivered by a database xref table, and required a full table scan; the error was, "Expression not supported by field limits."
Fixed a bug which could cause an error about "v.icid" not existing, when building a database using the "beta" IronPort plug-in.
Fixed bug which could cause a hang while building a database with an FTP log source, if one of the log files was not readable on the FTP server.
Fixed a bug which could cause underreporting of large duration numbers in HTML reports, when they exceeded about 38 years, when using certain duration display formats ("duration_compact" and "duration"); numbers above the limit were truncated to values between 0 and 38 years.
Fixed a bug where "max" or "min" numerical fields were not displayed properly in tables, in some cases (they appeared as huge negative numbers).

New features in 7.2.7:

Improved the "beta" Cisco PIX plug-in: added extraction of user name from "Accessed URL", "Built" and "Teardown" lines, and added support for system where outgoing users are authenticated with TACACS.
Improved the "beta" Symantec Gateway Security plug-in, adding support for two different formats for repeated lines ("Message Count = N" and "message repeated N times"), and adding / as allowable character in a service name, and adding support for two more format variations, plus some simplification of regular expressions and flow.
Added "beta" support for Juniper Netscreen Secure Access log format.
Added "beta" support for SafeSquid Combined log format.
Added "beta" support for SonicWall TZ 170 log format.
Improved Squid (syslog required) log format to add tracking of geographic location.
Added support for TFS MailReport Extended Log Format.
Enhanced Zone Alarm support, to report on source descriptions which are not in IP:port format.
Added support for Nortel SSL VPN Log Format.
Added support for WinRoute Connection Log Format.
Added support for WinRoute Web Log Format.
Added support for Symantec Mail Security Syslog header format.
Enhanced IronPort beta log format to report SBRS information better.
Improved Windows console output of SawmillCL.exe, so messages and text displayed to console appear in the proper OEM code page on non-latin systems. This allows, e.g., Russian systems to see error messages correctly in the Command Prompt windows, instead of seeing garbled messages (due to an attempt to display UTF-8 error messages in the OEM code page).
Enhanced InterScan Viruswall Log Format to track HTTP message, and another format of SMTP messages.
Added support for CWAT log format.
Enhanced support for Steel Belted Radius, with a new "beta" plug-in.
Enhanced SonicWall 5 format to handle yyyy/mm/dd dates.
Added support for Syslog NG (tab-separated) format.
Enhanced "beta" Cisco PIX to track "Received ARP request collision" lines, and "Denied SSH session" lines.
Improved support for Nagios log format.
Enhanced "Cisco VPN Concentrator (Comma separated - MMDDYYYY)" log format; added support for disconnect lines without a "Session type"
Added support for "Fiserv Financial Easy Lender - Unsuccessful Login Audit" log format.
Added support for "Easy Lender - Login Audit - Comma Separated" log format.
Added support for "Apache/NCSA Combined Log Format with Syslog (BETA)" log format.
Enhanced "Juniper/Netscreen Secure Access Log Format (BETA)"; expanded actions collected (upload, download, etc), added new numeric fields, made it work with non-syslog, and added collection of user details (break long user string into name, realm, role).
Added support for "Intersafe HTTP Content Filter Log Format (BETA)" log format.
Added support for "Squid Common Log Format - Syslog Required" log format.
Fixed bug in "Netgear Security Log Format"; removed non-derived database fields with no corresponding log fields which were causing report errors.
Enhanced "Backup Exec Log Format (BETA)"; added server and name fields from header.
Added support for "Kernun DNS Proxy Log Format".
Added support for "Kernun HTTP Proxy Log Format".
Added support for "Kernun Proxy Log Format".
Added support for "Kernun SMTP Proxy Log Format".
Enhanced the "beta" Postfix plug-in to report spam information, when present.
Enhanced the "beta" IMSS plug-in to handle a variant in virus.log, and to report recipients better in log.log.
Added "beta" support for IBM HTTP Server log format.
Added "beta" support for Piolink Network Loadbalance log format.
Fixed a bug which would cause an error when building a MySQL database, if the profile contained a database field called "length".
Improved date/time graphing to properly graph hours, minutes, and seconds on extended date/time intervals.
Added "beta" support for Datagram Syslog Agent log format.
Enhanced the "beta" Fortigate plug-in to handle a new variant (OS 3.0).
Added support for Sharetech Firewall log format.
Fixed a bug which resulted in %-sequences not being converted in search phrases.
Added "beta" support for the latest Border Manager log format (an earlier non-beta plug-in, which is still included, also supports the older format).
Enhanced Fortinet format to track all numerical fields.
Improved "remove database data" for MySQL databases, by adding an OPTIMIZE TABLE step after removing the data from the main table, to compact the size of the table on disk.
Added a new "beta" log format plug-in for IIS SMTP W3C, which handles any combination of W3C fields being present, and tracks queued and delivered messages and bytes.

Version 7.2.6, shipped June 09, 2006

Bugs fixed in version 7.2.6:

Fixed a bug in cisco_vpnconcentrator format where sent bytes was reported as both sent and received bytes.
Fixed a bug in Unix Syslog plug-in, where if the first line of data seen by a database update was a "last line repeated" line, it would cause an error about "unknown node v.last_log_line".
Fixed a bug which affected most firewall and proxy plug-ins, which caused page views and file types to be calculated incorrectly (file type would be empty, and page views would be the number of total events).
Fixed a bug where the Paths Through A Page report could become mangled if pages contained the sequence %00, or other strange sequences.
Fixed a bug where if a report was generated from the "main table" of the database (i.e., if it could not be generated from any xref table), then zooming to that report would show an empty table.
Fixed a bug where "minimum" fields were not computed properly (they were only computed if they were less than 0).
Fixed a bug in the Session Overview report which could cause undereporting of number of repeat sessions when the session visitor ID field is hierarchical.
Fixed a bug which would cause autodetection to hang on Alpha/Linux.
Fixed a bug in the Quicktime Streaming Server plug-in, which would cause an error when displaying session reports.
Fixed a bug which could cause a crash when displaying reports, if certain types of filters were uses simultaneously, with the internal database. This bug could theoretically also affect other aspects of Sawmill, and under some circumstances, could cause corruption of the cross-reference tables during builds.
Fixed a bug where "bottom-level items" checkbox was missing from the report editor, when editing a subtable report.
Fixed a problem where a report element could be saved with no numerical fields, which would break the report and the report editor.

New features in 7.2.6:

Added support for an "expression" field in database fields, which specified a Salang expression to evaluate to compute the value of that field in reports. This provides almost unlimited flexibility for customizing and computing the values of table cells.
Added a new built-in Salang function, unique(), which returns the list of unique field values (e.g., the list of IPs, for a visitors field) for a particular filter expression.
Enhanced "BETA" Juniper SSL format to track TCPPkt data.
Added filter_initialization_syslog and filter_finalization_syslog plug-in options, to run initialization and finalization code for the syslog portion of the plug-in
Added maximum concurrent sessions to Sessions Overview.
Improved Microsoft Media Format plug-in to treat each event as a session: a login and a logout. This allows reports of maximum concurrent connections, on a per-file or per-directory basis.
Added detection of Firewall-1 Binary Log Format. Sawmill cannot process this log format directly because it is a binary format, but it now generates a useful error message about that, describing how to convert it to a format which Sawmill does support (using "fw log" or "fw logexport" or "fw log ftn export", or by exporting it from the Log Viewer).
Added support for the latest variant of SpamFilter ISP log format.
Added "BETA" support for Cisco NetFlow (flow-export) log format.
Added "BETA" support for RAIDiator Error log format.
Fixed a problem which could cause and "itemnum=0" error when filtering on a non-existent value in a field, using a report filter.
Improved Oracle Failed Logins format to handle a variant with missing fields.
Added support for %Y/%m/%d and %H:%M:%S format, in Apache Custom format strings.
Enhanced the Apache Custom plug-in to track {URI}, {Content-Length}, and source IP fields.
Enhanced the "beta" Symantec Gateways Security plug-in to support a new variant.
Added support for Metavante log format.
Improved SQL Profiler log format to handle dashes in dates, and to parse multi-line fields better.
Improved Unix Syslog plug-in to handle IPv6 addresses.
Added support for Aladdin Esafe Sessions logs, version 5.
Added support for Sonic Wall TZ 170 log format.
Improved Shoutcast 1.8+ format to track countries/regions/cities.
Improved ISC DHCP format to handle another variant, and to handle some extra fields.
Enhanced SHOUTcast W3C format to track unique client IPs, to categorize reports, and to report session information, including concurrent sessions.
Enhanced Microsoft Exchange 2000 log format to report number of unique recipients.
Improved date extraction in Watchguard log format when no syslog header exists.
Added support for Bind Update log format.
Added support for Event Reporter v6 log format.
Added support for MDaemon 8 log format.
Added a MacOS 10.4 distribution for Intel Macs.
Changed the "recent linux" distributions, both 32-bit and 64-bit, to build on CentOS 4, ensuring compatibility with Red Hat Enterprise Linux 4. Previous versions were built on Fedora Core 4, which is less compatible.

Version 7.2.5, shipped May 08, 2006

Bugs fixed in version 7.2.5:

Fixed a bug where profile deletion did not work from the web interface.
Fixed a bug in Antispam SMTP Proxy log format, where entries single-digit month days were rejected.
Fixed bug where sessions were not logged out by a page value of "(logout)", if that value happened to be the very last unique page encountered in the log data.
Fixed a bug which would cause an error when searching the documentation, when Sawmill was running in CGI mode and was configured to use a "temporary directory" to serve static files.
Fixed "wildcard" and "regular expression" report filters to be case-insensitive for case-insensitive fields.
Fixed a bug which could cause a crash when updating using "skip previously seen files" from a log source containing tens of thousands of files.
Fixed a bug which could cause a slow updates when updating using "skip previously seen files" from a log source containing tens of thousands of files.
Fixed a bug which would cause an error when generating the Log Detail report, if it contained more than 61 non-numerical columns, when using a MySQL database.
Fixed a bug which could cause a crash while generating reports from a MySQL database, if one or more of the numerical column names conflicted with MySQL keywords.
Fixed a bug in the Sawmill service which caused it to fail to start on some Windows systems. This could theoretically affect any Windows system, but apparently does not affect most, and was only actually seen on some (not all) Windows 2003 Enterprise systems.
Unix syslog; extraction of the year portion of the date from file name changed to use a more restrictive regular expression.
Fixed a problem where license keys would be considered invalid if they were entered with capital letters.
Fixed a bug where numerical database fields which aggregated using the "maximum" or "minimum" operators were not aggregated properly when using a MySQL database (they were summed instead).

New features in 7.2.5:

Added "beta" support for Flash Media Player log format.
Enhanced Ironmail Spam log format to handle a variant.
Enhanced "beta" Cisco plug-in to handle a few additional lines, and to report the "list" field.
Enhanced Barracuda Spam Firewall to report BLOCKED lines.
Enhanced Microsoft Exchange 2000/2003 log format to report sender and recipient domains, and to group reports.
Added "beta" support for nmap log format.
Enhanced the "beta" Symantec Gateway Security plug-in to handle integer month names.
Added "beta" support for PeopleSoft AppServer Log Format.
Improved Kiwi mm/dd/yyyy format to allow logging_device and syslog_priority to be in either order.

Version 7.2.4, shipped April 07, 2006

Bugs fixed in version 7.2.4:

Fixed a bug where sorting by an "average" field would actually sort on the sum of the field values, and where an "average" graph would actually graph the sum.
Fixed a bug where if you zoomed on a date/time value, then zoomed on a session user, it would result in an error, "Couldn't find node 'xyz' in language.english.lang_stats.months_short".
Fixed a bug which would cause drive mapping to fail from the File Browser in Windows.
Fixed a bug which would cause new data to be discarded at the end of a CSV file, if the database was built from a partial CSV file, then later updated from a longer version of the same file. In that case, Sawmill would not add the new data.
Enhanced dumpel format to report the "strings" field.
Fixed a bug which could cause a crash if LogAnalysisInfo was relocated using LogAnalysisInfoDirLoc, and the new location was a very long pathname.
Fixed a bug where the "previously processed filenames" list was not updated on database update, so Sawmill was falling back on slower checksum-based log data skipping, when it should have been doing faster filename-based skipping.
Fixed a bug in Apache Custom format which could cause an error about "unknown field 'referrer'" when building a database from a log file which does not contain a referrer field.
Improved/fixed detection of Windows 2003 Server, Windows .NET Server, and Vista in user agent fields.
Fixed a bug in charset conversion, where if the input log data was being converted to a different charset than its native charset, a few characters (up to one every 10,000 bytes) could be dropped from the data. This could affect the skipping algorithm used to determine what data was already in the database, causing an update to re-add data which was already there.
Fixed a bug which would cause an error a about "file in use by another process" when viewing Single-Page Summary on Windows, if a filter set was selected which discarded all session events, or if there were no session events in the log data.
Fixed a bug in Blue Coat W3C support which would cause an error when viewing the Single-page summary, when processing data with "date" and "time" fields separate.
Fixed a problem with the user-agent parser which would incorrectly categorize some Windows 2003 systems as Windows Vista.
Fixed a bug which could cause incorrect parsing of date/time values in W3C data which had both a time-taken field and a time field.
Fixed a bug in the tracking of log lines processed, which could result in them lines processing being overreported by as much as 2x, for some log formats (those which uses accept/collect to parse). This was a cosmetic bug, affecting the progress display; the numbers in report were not affected.
Fixed a bug where the service did not start properly on x64 Windows.
Fixed a memory leak which could cause high memory usage (or out-of-memory errors) while generating reports.
Fixed a memory leak which could cause high memory usage (or out-of-memory errors) while building a database, especially when using a log format plug-in with complex parsing filters.
Fixed a bug which would cause an error about "item is its own superitem" when doing a MySQL database build, if the referrer field (or another URL field) ended in a question mark (?), and the log format plug-in did not list a question mark as the hierarchy divider for that field.
Fixed a bug where the Cisco Voice Router plug-in did not handle one-digit days properly.

New features in 7.2.4:

Enhanced "beta" Cisco PIX support to handle access-list lines, and to handle hit-cnt lines.
Added support for wildcard expressions in "session contains" filters.
Added support for Aventail Client/Server Access log format.
Added "beta" support for Instagate Syslog format.
Added "beta" support for RACF Security log format.
Enhanced RACF Security log format to extra username, intent, and allowed fields.
Added "beta" support for SmartFilter (Bess) Log Format.
Added -sb (sort_by) and -sd (sort_direction) options which can be used on the command line, or in the Extra Options of the Scheduler, to temporarily override the sort_by order and sort_direction.
Enhanced EIMS 24-hour format with a new "beta" plug-in which tracks a new log line format.
Enhanced Microsoft Media Player format to add play duration per visitor, and play duration per clip.
Improved the File Browser to jump straight to a share or drive after it is authenticated or mapped.
Added support for field named "date-time" in W3C headers, with EPOC format.
Enhanced the Cisco PIX "beta" plug-in to handle PIX, ASA, Router, and Switch messages in a single plug-in.
Added "beta" support for Kiwi Syslog (Logged to Access MDB, then exported tab-separated).
Enhanced Symantec SGS format to handle German month names.
Added "beta" support for log4j Log Format.
Added a a line showing the number of files matched, to the "Show Matching Files" window.
Enhanced IceCast version to track total and average durations (based on 96kbps stream), and to track 15+ minute sessions. This can be used to report FC Cume and other radio metrics.
Added support for a new variant of Free Radius Detail log format.
Added a support for specifying the ending row of a CSV export, using -er from the command line.
Added a new -tr command line option, which when true, includes a Total row in CSV export, similar to the one that appears in HTML reports.
Added a new -of command line option, which when specified, causes CSV export output to be generated to the specified file, rather than to standard output.
Added a new -eol command line option, which when specified, overides the default end-of-line character used in CSV output.
Added "beta" support for EventReporter log format.
Enhanced the "beta" Mail Enable plug-in to show POP3 events.
Added "beta" support for iptables config log format.
Added "beta" support for Netscreen Neoteris Web Client Export log format.
Added "beta" support for Barracuda Spyware Firewall log format.
Enhanced "beta" sendmail plug-in to track queued and delivered messages (and bytes) separately.
Enhanced RADIUS Accounting log format to track maximum connections.

Version 7.2.3, shipped March 06, 2006

Bugs fixed in version 7.2.3:

Fixed bug where month names appeared in English in Individual Sessions, and in the date range display at the top of reports, regardless of the selected language.
Fixed bug which could cause an error "Unknown variable 'lang_admin.log_filters.simplify_referrer_label' in expression" when viewing Log Filters.
Enhanced the "beta" Mail Enable plug-in to handle two-digit years.
Fixed a bug in Coradiant Object v2 format which would cause an error "Unknown variable 'cs_referrer'" when building the database.
Fixed a bug in the "beta" Cisco PIX plug-in which could cause the error "Syntax error: Expected variable, subexpression, or identifier -- found <" when building a database.

New features in 7.2.3:

Enhanced Exim 4 log format to report Antibody information, if present.
Added verification of LogAnalysisInfo version, to ensure that a Sawmill binary is not used with a mismatched LogAnalysisInfo directory.

Version 7.2.2, shipped March 03, 2006

Bugs fixed in version 7.2.2:

Fixed a bug where DNS lookups were not cached, resulting in very slow performance when using DNS lookup.
Fixed a bug in "beta" IMSS log format, where some "Received by" lines were ignored.
Fixed a bug which would cause an error during log processing, if the build was done from the command line with the "-v f" option, and a log filter used starts_with() or ends_with(), and one of the values used by the function contained a dollar sign.
Fixed a bug which could cause a crash when running Sawmill on a SPARC processor, during the cross-reference table build step of database builds, if there was a floating point numerical field in the database.
Fixed ISA/CSV format to handle one-digit hours.
Fixed a bug in ISA W3C which would cause an error when processing log data which did not have a cs-uri.
Fixed a bug where if an FTP password contained a plus sign (+), the Create Profile wizard would fail with an "Unknown command line options" error.
Fixed a bug where the "logo" line was generated to the console for each new process spawned, instead of just once.
Fixed a bug which could cause a crash during databases builds, especially with very large datasets.
Fixed a bug where the LogAnalysisInfoDirLoc file contents was ignored, causing an error if LogAnalysisInfo had been relocated by putting its pathname in that file.
Fixed a bug which could cause a crash if a web server click took too long to process.
Increased the timeout for web server clicks to 60 seconds (from 10), so documentation searches (and other long processes) taking more than 10 seconds will not time out.
Fixed a bug which could cause an error about "node type not found in ... framed_portocol" when analyzing IAS log data.
Added support for "last message was repeated N times" lines in UNIX syslog. The UNIX syslog plug-in now properly identifies these lines, and uses them to replicate the previous line N times, so the correct number of events appear in reports.
Fixed a bug where if a database was corrupt in a certain way (missing itemnums table), it could not be rebuilt from the web interface, because clicking Rebuild Database would give an error.
Fixed a bug where CSV exported files did not download properly when using Sawmill in CGI mode.
Fixed a bug which could cause a crash when viewing reports, if the database was corrupt in a certain way (in particular, if the "items" table did not exist in the database, or was not fully built).

New features in 7.2.2:

Added support for DansGuardian 2.9 log format.
Added "beta" support for Mailman Post Log Format.
Added "beta" support for Watchguard XML log format.
Added "beta" support for Windows Firewall log format.
Added an improved "beta" Amavis log format plug-in.
Added "beta" support for Cisco As5300 Log Format.
Added Polish translation of Sawmill.
Added "beta" support for McAfee WebShield XML format.
Added support for \t and %% in Apache LogFormat directives.
Added "beta" support for XWall log format.
Added "beta" support for Snare for AIX log format.
Added a new "No Syslog" plug-in which can be used to report on log data which sometimes has a syslog header, but does not in this case, using the standard "device" plug-in for that format.
Enhanced the "beta" Snare plug-in to extract timestamp information from the Snare data, if present.
Added "beta" support for Internet Security Systems Network Sensors log format.
Enhanced Domino Access log format support to track processing time, cookies, and translated URL.
Added "beta" support for Juniper Secure Access SSL VPN Log Format.
Enhanced Argsoft Mail Server Log Format to handle a slight variant with AM/PM times, slashes instead of dashes in dates, and some slightly different spacing from the older format.
Added "beta" support for Sourcefile IDS log format.
Added "beta" support for AutoAdmin log format.
Enhanced Symantec Security Gateways Log Format to support a slight date variant.
Enhanced Symantec Antivirus plug-in to handle a slight variant.
Added "beta" support for Annex Term Server log format.
Added a new "beta" version of IAS CSV log format. The new version categorizes reports for a much nicer report menu, and tracks many additional database fields, including octets and packets.
Added support for direct serving of static HTML files through Sawmill's built-in web server. This makes it possible to generate HTML report to LogAnalysisInfo/WebServerRoot, and serve them from there directly.
Added "beta" support for Flex/JRun Log Format.
Added "beta" support for Netscreen Web Client Export Log Format.
Added "beta" support for Kerio Mailserver Mail Log Format.
Added "beta" support for Bintec VPN 25 or XL Log Format.
Added support for Novell Border Manager Log Format logs with a W3C header.
Added "beta" support for Backup Exec Log Format.
Added a new "beta" version of Argosoft Mail Server Log Format, which tracks much more information, including messages queued vs. delivered, multiple recipients, and connections rejected.
Changed log processing order to process the newest file (based on modification date) first. This helps with a common problem with IIS logs (for instance) where the oldest log has less complete headers than the newest log (because fields have been turned on); profile creation in this case needs to be done based on the newest log, not the oldest one, to get best results. This change in order ensures that the profile creation will be based on the newest log data.
Added "beta" support for MPS log format.
Added "beta" support for TippingPoint IPS Log Format.
Added a "Printer Friendly" icon to the report toolbar, for generating a version of a report formatted for printing.
Enhanced the "beta" Cisco PIX plug-in to autodetect ASA data, and to support "side" fields containing spaces.
Added detection of binary Watchguard log format. Sawmill can't process that format because it's a binary file, but it reports what it is, and describes how to export it.
Enhanced Barracuda Spam Firewall log format plug-in to track senders and recipient data separately, and to track quarantined, spam block, virus block, and tagged messages.
Enhanced "auto" date format to support any mix of upper/lower case in month names.
Enhanced the "Network Shares" button (formerly "Map Drives"), in the file browser, to support access to password-protected shares on Windows, without mapping them as drive letters.
Added additional information to the TaskLog line generated by database builds and updates, including total bytes processed, time elapsed, entries per second, and bytes per second.
Added support for per-profile languages (through statistics.miscellaneous.language), and per-user languages (through a language value in the user .cfg file).
Added "beta" support for Cisco Switch/Router Log Format.
Enhanced Cisco Voice Router format to track duration, bandwidth, and much more.
Added "beta" support for openldap Log Format.
Added "beta" support for Barrier Group Log Format.
Added "beta" support for Nortel Networks Instant Internet Log Format.
Added "beta" support for Performance Monitor Log Format.
Added "beta" support for Cisco WLAN Controller Log Format.

Version 7.2.1, shipped February 02, 2006

Bugs fixed in version 7.2.1:

Fixed a bug which DNS lookup which could cause crashes on some platforms, including 64-bit Windows.
Fixed a bug where "average" database fields reported the sum of the field values instead of the average.
Fixed a bug where duration_milliseconds and duration_microseconds fields omitted zeros after the decimal point, when they were displayed in HTML reports.
Fixed a bug where if the log time format was "auto", then times of the format "12:nn PM" would be normalized as "24:nn", resulting in an error in the "hour of day" report.
Fixed a bug which could cause a "floating exception" on Tru64 UNIX during database builds.
Fixed a bug which could cause a crash while generating a table report if: 1) the profile used a MySQL database, 2) the profile included a "unique" field like the "visitors" field fo web log analysis, and 3) the table being generated did not include that field in any column.
Fixed a bug with the NetScreen "beta" plug-in, where src/dest IPs were not reported for attacks.
Fixed a bug which could cause an error "no node 0 found in log_source" when clicking Show Matching Files, if the first log source had been deleted.
Fixed a bug where if scheduled tasks overlapped, the later one would sometimes not be run.
Fixed a bug where multiprocessor builds would fail on Windows if a custom database directory was specified, and if that directory name contained a space.
Fixed a bug in the handling, in Sawmill's HTTP server, of the If-Modified-Since and If-None-Matches headers; this should improve performance for caching browsers and proxy servers.
Fixed a bug in PIX Firewall Syslog Server Format which caused some log data to fail to parse.
Fixed a bug which could cause a crash when generating a report table, when using a MySQL database, if the table contained both unique and non-unique rows, and if at least one of the rows had all zeros for the non-unique columns.
Fixed a bug with NetApp, where entries could be rejected due to date/time corruption, though the date/time values were not actually corrupt.
Fixed a bug where the Apache LogFormat directive parser did not recognize special fields, like User-Agent, unless they were capitalized just the way it expected.
Fixed a bug which occurred when using the internal database, where if a database field was case-insensitive, and a value occured with different cases in different places in the log data, zooming on that value would show only some of the items below it.
Fixed a bug which could cause some files not to load while using the web interface, resulting in sporadic cases where certain frames or files did not appear.
Fixed a bug which could cause some portions of pages to load (e.g., CSS or JS files) when accessing Sawmill through the web browser interface.

New features in 7.2.1:

Added "BETA" support for Lancom Router Log Format.
Added support for Sophos Antispam Message Log Format.
Added an Italian language translation of the reports and the Admin interface.
Added a "BETA" Sendmail plug-in which parses logs data faster, and tracks more fields.
Added "BETA" support for msieser SMTP log format.
Enhanced praudit "BETA" plug-in to handle -l format.
Added an Active Tasks section in the Admin page, showing information about active tasks, including time elapsed and progress information.
Added a Task Log section to the Admin page, showing the contents of the TaskLog file.
Enhanced praudit "BETA" plug-in to handle Snare/Solaris logs.
Enhanced the "beta" Cisco PIX plug-in to handle a different type of Deny line.
Enhanced the "dumpevt" Windows Event Log plug-in to handle a different data format "d/m/yyyy".
Added a limited Scheduler in Lite tier. This version of the Scheduler can update databases, rebuild databases, and send email, but does not have the "extra options" field of the full Scheduler, and cannot do other tasks.
Greatly enhanced support for Zyxel Firewall WELF Log Format; added tracking of all numerical fields, and some fields which were not tracked before. Added support for a variant with a leading date stamp in the syslog message.
Added "beta" support for Scanmail for Exchange log format.
Added an option to change the MySQL socket file pathname.
Changed internal filenames to always be less than 32 characters long, to work in environments which do not allow long filenames.
Enhanced "beta" fortigate plug-in to handle additional fields, including URL and username, which are present in some variants.
Added support for user agent fields which use underbars instead of spaces, for example Windows Media Server.

Version 7.2, shipped December 19, 2005

Bugs fixed in version 7.2:

Fixed a bug which caused MySQL builds to immediately fail when the MySQL server was configured to use a port that was not then the default port, 3306, regardless of if the user specified host:port in the GUI field.
Fixed a bug which could cause an error when editing Log Filters in a profile analyzing Snare log data.
Fixed a bug which could cause an error when building a database in a profile analyzing Interscan Web Security Suite log data.
Fixed a bug which where the number of visitors was always shown as 1, when analyzing Quicktime Streaming Server logs.
Fixed a bug which could cause an error (about an item's number being the same as its parent) when processing log data with a MySQL database, if the log data contained field values ending with spaces, and other identical values which did not end with spaces.
Fixed a bug where zooming on a value would show 0 events, if the value contained a backslash ('\') followed by an 'n'.
Eliminated potential problems with corruption of IPNumberCache by eliminating IPNumberCache. This file, which used to live in LogAnalysisInfo, kept a cache of all previously looked-up DNS addresses. But it didn't honor the DNS TTL option, and could become corrupt during multiprocessor builds in some cases. Plus, it didn't improve performance much. So it's gone now. The memory cache, which definitely does improve performance, is still there, but is discarded after each build.
Improved Create Profile wizard to show progress during autodetection, solving a problem where very large compressed files on FTP sites could cause the browser to time out while autodetecting.
Fixed a bug where multiprocessor MySQL builds could fail due to conflict between the threads in access of itemnums tables, resulting in errors like "insert into fielditemnum set field = 'xxxx' (Duplicate entry 'xxxx' for key 2)".
Fixed a bug where multiprocessor MySQL builds could fail at the end with an error like "table fieldsubitem1 does not exist".
Fixed several issues with SQL queries which caused errors when using Sawmill with MySQL Cluster, with a default table type of ndbcluster.
Fixed problems with x-timestamp fields in W3C and fields with square brackets, where the brackets would be treated as quotes, resulting incorrect field values.
Fixed a bug in ProxyPlus log format which could cause an error about the authenticated_user field not being defined.
Fixed a problem where the NetCache NetApp plug-in reported full URL values, resulting in extreme memory usage for large datasets.
Fixed a bug where "unique" values (e.g., visitors, in web log analysis), could be underreported in some table reports, when complex report filters were used. The values would be reported as values typically between 10,000 and 15,000, even if the correct values were much higher.
Fixed a bug which caused an error when running a binary of Sawmill built from encrypted source on Itanium HP-UX.
Fixed a bug where when running Sawmill with the -scheduler option (e.g., using the Sawmill Scheduler in CGI mode), which caused the last task to terminate before completion.
Fixed a bug which could cause a MySQL build error if a database field was named "count" in a profile.
Fixed bug which could cause a progress prediction error when displaying a multi-element report with disabled report elements.
Added a "beta" log format plug-in for Apache Extended. The new version tracks several new numerical fields separately from normal hits and page views (spiders, worms, errors, broken links, and screen info hits); and it includes a new broken links report which shows broken links with referrer URLs.
Added a new discard_expired_entries option to fix a bug where some log format plug-ins, including Postfix, would never discard old collected log entries, resulting in a gradually growing memory usage which could, given enough log data, exceed the available system RAM.
Fixed a memory leak with certain log format plug-ins (those which use "rekey" functionality), and certain datasets (those with duplicate keys) which could cause gradually growing memory usage during log processing.
Improved performance of subitem table merges during MySQL multiprocessor builds. This can be a very long process for extremely complex fields; in one case, the previous algorithm took 150 minutes for a particular dataset, and the new one takes 40 minutes.
Fixed a bug in passlogd log format which could cause all log entries to be ignored.
Fixed a memory management inefficiency which could cause extreme memory allocation for a short time while computing a report with complex filters.
Fixed a bug where the cross-reference tables would not be up to date with the final few lines of data in the main table, resulting in slightly low numbers. This could happen in particular in multiprocessor builds using the internal database.
Fixed a bug which would cause a useless (redundant) profiles.cfg file to be written to LogAnalysisInfo when deleting a profile.
Fixed a bug where the expand_path_greater_than value did not expand at all when it was set to 0.
Fixed a bug where the values "max" and "min" for the aggregation_method parameter did not work properly; it was expecting "maximum" and "minimum", contrary to docs, and furthermore, it was accepting the parameter to be called aggregation_operator. aggregation_method is the correct parameter name, and "max" and "min" are the correct values, and they now work (aggregation_operator and "maximum" and "minimum" will also work, for compatibility, but may be deprecated at some point).
Fixed a bug which could cause an error when building a MySQL database, if the log data contained a field value with a leading divider, for a right-to-left hierarchical field, e.g., the value @yahoo.com in a hierarchical email field.
Added "beta" support for du log format.
Fixed Symantec Antivirus plug-in to better differentiate different line layouts.
Fixed a bug where expressions in custom report headers/footers were not expanded on HTML export.
Fixed a bug where deleting a profile created a "profiles" file in the LogAnalysisInfo directory.
Fixed a bug in the left menu where the active menu item was not indicated at startup
Fixed a bug where custom report and report element headers/footers became deleted upon editing a report via the GUI.
Improved the users form so that a single administrator cannot be deleted.
Improved the admin licensing page and added license validation.
Improved setup/login for first time installation, added a wizard like setup.
Improved the trial switch so that the trial mode can be changed from the admin GUI.
Added About section which shows the version number.
Added support page to admin interface.
Fixed a bug where if there was an N-profile license installed, and also an unlimited-profile license, it would not allow more than N profiles to be used.
Fixed a bug which could cause an error on profile creation (and possibly other times), if a profile was being rewritten just as it was being read by another process. This could cause a variety of errors, including "node database not found" or "unterminated quote" or "unexpected end of configuration".
Fixed a bug where commas (thousand dividers) were incorrectly inserted in negative numbers, resulting in numbers like -,123.45 instead of -123.45.
Fixed a bug where visitor numbers could be corrupted during a database merge, for instance, during a database update or a multi-processor build. The resulting database would show 1 visitor for some table rows, where it should have been several thousand.
Fixed a bug (introduced after 7.1.14, so affecting only some recent pre-release builds) which could cause the hour_of_day field to be added to all xref groups, instead of the date_time field, in case where there was no date_time log field.
Fixed a bug where rows with all zero values were included in MySQL-based report tables.
Fixed a bug in Snort processing where events beginning with parentheses were not displayed in the Events table.
Fixed a bug where Sawmill's HTTP server did not notice remote socket closes in some cases, resulting in some portions of pages failing to load.
Fixed a bug where if there were two report elements with subtables in a single report, the second one could contain the data of the first, in addition to its own data.

New features in 7.2:

Added support for UTF-16 encoded 8-bit log data.
Improved Create Profile wizard to load profile list at autodetect time, which eliminates a long delay at the beginning of profile creation.
Added a new profile option, database.options.mysql_engine, which controls the engine used for itemnum tables when using a MySQL database. When this is not specified, MyISAM is used. This option must be set to ndbcluster when using MySQL Cluster.
Added a new "beta" version of the Fortigate plug-in which extracts and reports much more information.
Added "beta" log format support for Firepass.
Added "beta" log format support for TACAS+ Accounting log format.
Added "beta" log format support for NetScreen log format. The new version tracks much more information.
Added "beta" log format support for Nortel Contivity. The new version tracks much more information, including usernames for HTTP events.
Added "beta" log format support for IIS SMTP Comma-Separated log format.
Added "beta" log format for Cisco PIX. This is much faster to process log data than the production PIX/IOS plug-in, and adds reporting of services, and adds reporting of various two-field tables (e.g., destination IP by source IP).
Added "beta" log format for MailEnable W3C format. The new version tracks messages in much more detail, tracks messages sent and received, tracks bandwidth in both directions, tracks errors, and more.
Added a "simplify URL" log filter to all proxy/firewall formats, to chop off the "pathname" portion of the URL. This filter was present in many proxy/firewall log format plug-ins, to keep the database from getting overly complex, but had not been added to many others, including MS Proxy formats. Without this filter, the memory usage exceeded 1GB for one dataset; with the filter, memory usage was under 100MB.
Added a "simply URL" log filter to MS Proxy formats, to chop off the "pathname" portion of the URL. This filter is present in most proxy log format plug-ins, to keep the database from getting overly complex, but had not been added to the MS Proxy plug-ins. Without this filter, the memory usage exceeded 1GB for one dataset; with the filter, memory usage was under 100MB.
Added support for per-report-element and per-report headers and footers.
Added "beta" support for Symantec Antivirus log format
Added a new improved "beta" plug-in for Postfix. The new version breaks from/to fields hierarchically, and breaks traffic into independently tracked numerical fields: messages delivered, messages processed, messages blocked, messages expired, messages delivered, messages bounced, bytes delivered, bytes processed, bytes blocked, bytes expired, bytes delivered, and bytes bounced.
Enhanced Single-page Summary so it is computed (when the profile is created) by cloning all other reports. Previously, it was created separately, so some report would not be present, and some customized report settings would not apply to the single-page summary. Now, all report customization done in the log format plug-in applies automatically to the version of the report in the single-page summary
Added a new improved "beta" plug-in for Snare. The new one extracts much more information, and supports a wider range of messages.
Added "beta" support for Ascenlink Log Format.
Added a new "-a rp" option to recreate a profile from the command line. This is useful for log format plug-in authoring, where the profile often must be recreated many times until the plug-in works.
Greatly expanded and improve the Custom Log Format documentation.
Added "beta" support for msieser HTTP Log Format.
Added "beta" support for Nessus Log Format.
Added "beta" support for an enhanced Kasper Skylabs Mailserver Log Format.
Added a new "-a pl" option for the command line, which processes the data in the log source, dumping the accepted entries to the standard output stream in comma-separated format, without building or modifying the database.
Added "beta" support for Java Administration MBEAN Log Format.
Fixed/improved Merak log format to handle "Client session" in the middle of some lines, and to track only events which involved messages actually being sent.
Added "beta" support for Trend Micro Control Manager Log Format.
Improved robustness by changing progress prediction errors to warnings. Progress prediction errors occur when the order of steps that are predicted for a task does not match the actual steps which occur. This occurs due to bugs, but it is difficult to predict in every case what steps will occur during a task, so there have been many bugs of this sort. This change works around this sort of bug by displaying a wanting message and patching the progress prediction to match the actual steps being taken by the task. This may mean that the steps in the progress page will change in some cases, but this sort of issue will no longer be a fatal issue, terminating the database build or report.
Enhanced "auto" time for to support times in the format "h:mm".
Improved the "create profile wizard" when using a remote log source, so instead of downloading the data (very slow for large compressed data by FTP) twice, once for autodetection and once to set up the fields, it caches the first download to make the second operation fast.
Added "beta" support for Symantec System Console log format.
Added support for associating numerical fields with database fields in log format plug-ins, so only appropriate numerical fields are added to the reports, and only appropriate numerical fields are added to the xref groups, by default.
Greatly enhanced the level of report customization available in the report_groups section of log format plug-ins, allowing for virtually all custom report options to be specified there.
Added support for mapping drives in Windows, directly from the File Browser window.
Added "beta" support for Netscreen SSL Gateway log format.
Added a "Report It" link to error messages, for reporting errors to Flowerfire.
Added an improved "beta" log format plug-in for Communigate Pro.
Added context-sensitive help links to Log Filters, Scheduler, Report Editor, and File Browser.
Greatly enhanced the Create Profile Wizard to include more context-sensitive help, show progress during autodetection, and improve performance. Changed page flow and page layout in the wizard to reduce the chance of error, especially when using Lite. Removed several unnecessarily options. Added an option to go straight to the reports after creating a profile. Improved wizard to load plug-in list at autodetect time, which eliminates a long delay at the beginning of profile creation.
Added a button to rebuild and update the database from the reports.
Added support for a new command line option, -er (ending_row) which overrides the ending_row option of the report being generated. This is useful for doing a command-line 1000-row export, for instance, of a report which defaults to show only 20 rows.
Added a new "beta" version of Interscan Web Security Suite, which tracks all log format types and includes much more advanced reporting.
Added a new "log.filter_finalization option to the profile. This is an expression which is run at the end of log filtering, and can be used to finish anything that needs finishing (e.g., to accept entries, or to write something to disk).
Eliminated the need for the "temporary directory" and "temporary URL" when running Sawmill in CGI mode. This significantly simplifies CGI mode installation.
Added support for file locking in Salang (the internal language). This is useful when creating log filters which must share a single read/write resource, like a map file, and need to synchronize so they don't both write it at the same time.
Added support for Last-Modified, Etag, If-Modified-Since, and If-None-Matches headers in Sawmill's HTTP server, to allow for caching of static files (for faster browsing).
Added an "Advanced Filter Expression" option in the report Filters window, for entering advanced Boolean filters, like "(recipient_address within 'someone@somewhere.com') or (sender_address within 'someone@somewhere.com')", which cannot be constructed using the other filter options.
Fixed a bug with command-line authentication which could cause an error "Can't find node command_line_authenticated_user in users" when command-line authentication was used.

Version 7.1.14, shipped September 08, 2005

Bugs fixed in version 7.1.14:

Fixed bug where memory usage could grow without bounds while building with build_indices_during_log_processing turned on.
Fixed a bug where "Show matching files" in the "New profile wizard" caused an error for FTP log sources with empty username or password.
Fixed bug which would cause an error in the "create profile" wizard if FTP passwords contained a # or a double quote.
Fixed a bug where "automatically update if older than" could compute the age of the database incorrectly, resulting in an unending series of updates when viewing reports.
Fixed conversion of if_a_then_b_c log filters, and goto_filter_number log filter actions, in Sawmill 6 configurations; converting them was causing an error during conversion.
Fixed a bug which could cause an error about query syntax when using certain versions of MySQL, due to the use of varchar(1000) in the query, and indices of length 1000, when some versions of MySQL do not allow more than 255.
Corrected extraction of date field in WebSEAL CDAS Log Format plug-in, incorrect date extraction caused all records to be ignored during rebuild or update of database.
Fixed a bug where the Lock files in the IPC directory were not properly deleted, leading to large numbers of them after long periods of usage.
Fixed a bug where errors during database building would not be reported in the web interface, if they occurred very early in processing (e.g. incorrect username for MySQL database).
Fixed a bug where the Browse window (the File Browser) had problems with directory names containing non-English characters, and would not process log data contained in them.
Fixed a bug in generate_all_report_files which always required that an Overview report exists in the left menu. A disabled Overview report caused a page loading error and a hang in the reports menu.
Fixed a bug in generate_all_report_files where Sawmill generated reports of a menu group although the menu group was disabled.
Fixed a bug in File Manager where foreign characters where not correctly interpreted.
Fixed a bug which cause cause an error in the Create Profile wizard when creating a profile with extremely long pathname information, or a huge number of numerical fields.
Fixed a bug in reports menu editor where single quotes in report menu names caused the reports menu editor to freeze.
Fixed a bug where if there were multiple simultaneous browsers accessing reports, with a MySQL database, it could cause an error about the table "sumstats" not existing, or already existing.
Fixed a bug which could cause a SQL syntax error if the database contained a field called "user", when using certain versions of MySQL.
Fixed a security issue (cross-scripting vulnerability). By accessing Sawmill with a carefully constructed URL, it is possible to execute arbitrary JavaScript code the web browser system. This makes it possible to hijack another user's browser's session by convincing them to click an apparently reasonable link to Sawmill. The latest version now prevents this type of malicious activity.
Fixed a memory leak which occurred when there were a very large number of very long lines in the log data. Memory usage in that case could be tens or hundreds of times more than it should have been, during database build. It would eventually stabilize at a high value, but that value could be 1GB or more.
Fixed Quicktime/Darwin Stream Server log format so it tracks full filenames by default.
Fixed a memory allocation issue which could cause Sawmill to use extremely large amounts of memory while building the hierarchy tables for a MySQL database (which typically occurs after the log processing, and before the xref table builds). Memory would eventually be deallocated, but it was growing unnecessarily large during that stage. It is now cleaned up regularly through that stage to keep memory usage in check.
Fixed a bug where the Logout link did not work properly for non-administrators; the user would be logged out, but could not then log in again from the resulting page.
Fixed a bug in the v6-to-v7 configuration converter, where log fields with type "server response" were not converted properly, resulting in an error on Config.
Fixed a bug in Syslog (yyyymmdd hhmmss) Log Format, which could cause lines of log data not to be recognized by the device parsing plug-in (this was verified with iMail 7, but could happen with other formats too).
Fixed a bug in Trial Login where it was not possible to change between the Enterprise and Professional feature set.

New features in 7.1.14:

Added a -db command line option to to a date breakdown, e.g. to generate a report of months in a single year.
Improved performance of "main table" based reports using the internal database, by keeping indices on disk instead of reading them into memory.
Added output format directive duration_hmmss, for displaying duration information in H:M:S format. rather than with year and days shown.
Added a new generate_pdf_friendly option for generating HTML export output which converts well to PDF format using Adobe Acrobat. Without this option, the HTML export displays well in the browser, and uses JavaScript and other browser features, but these extra features do not play well with Acrobat. With this option, it does not look as good in a web browser, but works very well for PDF.
Modified the report style sheets to achieve better results in emailed reports, when generating PDF friendly files or when printing a report.
Improved Snare support to support literal tab characters, in addition to the <009> variant (which is what the old plug-in supported).
Added a -zv option which specifies the "zoom value" for command-line and Scheduler exports, i.e. the item we're zoomed into. This is needed in cases like showing data by region for a particular country in the Countries/Regions/Cities report.
Improved autodetection to report an error if there is no data in the specified log source (rather than just reporting that no format matched).

Version 7.1.13, shipped July 29, 2005

Bugs fixed in version 7.1.13:

Corrected extraction of missing data in fields the CISCO Voice Router plug-in.
Corrected missing 'host' type field which caused the Domain Description report to report an error, and fail to generate for Urlscan log format plug-in.
Corrected missing extration of "TO:" and "FROM:" in the CommunigatePro log format plug-in.
Fixed a bug where progress reports would fail with a "can't find node step in volatile.progress" error, after the task has been running for more than an hour.
Fixed a bug where session event times were all 00:00:00 when using MySQL on Windows.
Fixed a bug where progress reporting could "stick" or fail in the web interface, with an error about "step" or "STEP" not being found.
Fixed a bug where pages in a session after the first page were omitted when remove_reloads was true and there was no session page field.
Fixed an issue which occured when using an FTP log source to download a single file from a server which did not allow directory listing. Sawmill would fail when the directory listing failed, but it didn't really need the listing if there was just one file; it now allows this failure and downloads the file anyway.
Fixed a bug with Proxy Pro Gate Keeper log format which would cause an error about "default not found".
Fixed a bug where if a generic W3C log had a c_ip field (rather than a cs_ip field), it would not track session information properly.
Fixed bug which could cause an "superitem is 1, and subitem is 1" errro when building or updating a MySQL database.
Fixed bug which could an error when using an FTP log source with a local (non-absolute) directory, e.g. "logs/*". In this case, it would attempt to change to the "logs" directory before each file, which would fail after the first one, because there is local "logs" directory when it's already in logs. It now does the directory change only once, at the beginning of the session.
Fixed bug where database build could fail if both build_indice _in_threads and build_indices_during_log_processing were true for a multiprocessor build.

Version 7.1.12, shipped July 13, 2005

Bugs fixed in version 7.1.12:

Fixed a bug where session reports were wrong if there was no "page" field in the log data (very uncommon).
Fixed a bug which caused an error when processing Windows Media log files without a referrer field.
Fixed a bug where subprocesses would not exit in some cases when clicking Cancel during a multiprocessor build.
Fixed a bug where charset conversion did not work on Windows (only a problem in 7.1.11), failing with an error "iconv() not available."
Fixed bug where database updates with MySQL did not rebuild the cross-reference tables, resulting in out-of-data information in some reports.
Fixed a bug which could cause incorrect results when performing a MySQL query with multiple filters on the same field.
Fixed a bug which could cause incorrect results when performing a negated ("NOT") MySQL query.
Worked around a problem with MySQL and trailing whitespace, by automatically removing trailing whitespace from field values. The problem could cause hangs during builds if a field value was a single space, and incorrect reporting of values with trailing whitespace.
Fixed bug where "remove database data" operations with MySQL did not rebuild the cross-reference tables, resulting in out-of-data information in some reports.

Version 7.1.11, shipped July 07, 2005

Bugs fixed in version 7.1.11:

Fixed a bug where action emails would fail with a bug "can't find node network in command_line".
Fixed Bug where MySQL failed to update data base when sql protected keywords in among the log fields
Fixed a bug where invisible columns in multi-column tables were not properly omitted.
Fixed a bug which could cause an empty table when using a date range filter together with a "year" zoom (and other similar situations involving simultaneous filtering on lower-level items like days or files, and higher-level items like years or directories).
Fixed a bug where the Log Filter editor window did not close properly after clicking Save and Close, when using Safari.
Fixed a bug where if a profile was using the "internal" database, and was switched to a MySQL database, it could cause an error when displaying the Build Database page.
Fixed bug where multithreaded builds in MySQL failed to create hierarchy tables
Fixed a performance issue which made index builds several times slower than they should have been, especially on Windows.
Fixed a bug which could cause an "unknown variable v.syslog_message" error when processes corrupt syslog data.
Fixed a memory leak which could cause large MySQL database builds to use arbitrary amounts of RAM (roughly 50MB to 100MB for each million lines of data processed).
Fixed a bug where one user canceled the report generation of another user when two or more users accessed the same profile.
Fixed a bug which could cause incorrect durations to be listed in the Session Pages and Session Users reports, when using the "internal" database.
Fixed a problem with processing some variants of Netscreen log data.

New features in 7.1.11:

Added "page header" and "page footer" options (including the file variants) to the web interface in Config->Manage Reports->General Display/Output.

Version 7.1.10, shipped June 17, 2005

Bugs fixed in version 7.1.10:

Fixed a bug which caused an empty table when zooming or filtering on a bottom-level item for a particular field, while viewing the report for that field, with a MySQL database.
Fixed a bug which could cause items to be omitted from tables when using MySQL, if the item existed in the log data in both uppercase and lowercase variants.
Fixed bug in Mysql where nonexistent filter gave an error now it gives zero result.

Version 7.1.9, shipped June 10, 2005

Bugs fixed in version 7.1.9:

Fixed a bug which could cause zero results when using a regexp report filter containing backslashes or certain other special characters, with a MySQL database.
Fixed a bug which could cause zero results when using a regexp report filter containing backslashes or certain other special characters, with a MySQL database.

Version 7.1.8, shipped June 10, 2005

Bugs fixed in version 7.1.8:

Fixed a bug where the Overview could show 0's when using a MySQL database and regular expression report filters.
Fixed bug which could cause an error when generating the log detail report with regular expression report filters with a MySQL database.
Fixed bug where zooming on a day, or using a date range, would result in an empty Years/Months/Days report.
Fixed bug which could cause a crash when viewing session reports, if there was no session page field in the database.
Fixed a bug which could cause very high memory usage with some log formats, including "Interscan Messaging Security Suite Log Format".
Fixed the page_