 |
ALL PLUG-INS
Sawmill has plug-ins to support the following log formats: |
 |
This is the version history for Sawmill 8.
The Sawmill 7 version history is here
and the Sawmill 6 version history is here.
Version 8.0.9, shipped June 6, 2009
Bugs fixed in version 8.0.9:
[532124] Filtered reports show higher numbers than unfiltered reports, when using "is NOT" filters. [532124] The description of report filters sometimes contains a "not" even when it isn't a "not" filter, if there is also another filter which is a "not" filter. [554819] Multiprocessor database builds create zombie processes on UNIX systems, which last for the duration of the build; on systems with very few processes available, this can cause errors during build, including, "Error spawning process for multi-threaded SSQL query split." [584458] The sort_by (-sb) and sort_direction (-sd) command line options for "-a ect" are not documented. [586149] Profiles with duplicate labels are allowed by the Create Profile Wizard [590678] Log detail report is very slow, when using with a large internal database and no filters. [591432] The update.pl script in the Extras folder does not copy the system.cfg file, so installations updated with that script rerun the installation process. [591629] Scheduled tasks with a single profile, and multiple actions, run all actions simultaneously, instead of running them in sequence. [592187] Database indices, and unique-tracking lists, can become corrupted on 32-bit systems, when processing large datasets (typically, any dataset which would require an index > 2GB, though it could happen as low as 600MB). This can give various errors about "array caches" or "ranges." [592295] When using file-by-file distributed parsing, and "skip previously seen files on update," the list of previously-seen files is not recorded properly, resulting in some files being re-processed on update. [595619] Imported v7 profiles show integers in the sum row of "start time" or "end time" columns, instead of showing "-". [596194] When filtered, the Sessions Overview sometimes shows a negative number for "sessions for one-time users", and shows a few other incorrect values. [596597] Database fields with large aggregated values (more than about 10 billion) incorrectly show very small values (< 10) in the Overview. [596753] The current_log_pathname() function returns the wrong pathname for some log lines, when using multiprocessor database builds. [597095] Removing database data with a date range in a MS SQL database gives an error, "Invalid column name."
[597455] Database indices can become corrupt on update, when using the internal database. This can cause errors when displaying filtered reports, or errors during future database updates. Possible error messages include "Unable to allocate N bytes of memory (Reallocating array cache)", "Internal: list N in IntegerLists X ends with -M, which is the start of a range", and possible others. [597949] In real-time profiles, reports hang at 33% during the initial database build, if the build is using parsing servers (multiple processors), or if the file-by-file option is turned on. [597953] Date filter expressions containing capital letters give an error, "Date filter not valid." [598038] Log format plug-ins which use "collected listed", where field values are quoted (e.g., Fortigate), ignore the values immediately after quoted fields. [601338] Cross-reference tables of imported Sawmill 7 profiles do not contain numerical session fields, resulting in slow report generation. [601365] Regular expression or wildcard filters on the "hour of day" or "day of week" field are displayed incorrectly in the yellow filter description at the top of reports, as "corrupt date/time". [601810] Database updates applied to databases imported from Sawmill 7, or to databases using non-internal database servers, do not propagate the new data to the xref tables in some cases, resulting in reports which do not have the latest data.
New features in 8.0.9:
[569464] Added support for a new format variant of the NcFTP Xfer log. For now, the new fields at the end of each line are being ignored. [584392] Added support for overriding the port of SFTP, by using "hostname:port" in the hostname field.
Version 8.0.8, shipped May 20, 2009
Bugs fixed in version 8.0.8:
[527528] The "Lookup pages" link in "Paths through a page" gives an error, "Syntax error: Unknown variable 'xyz' in expression," with certain datasets. [538929] The return address option for Action emails in the profile is not editable from the web interface. [543954] Database fields with identical labels (but different internal names) cannot be distinguished in the web UI. [548862] Building a database from Quicktime Streaming Server log data which has no cs-uri-stem field gives an error, "Can't find tableAlias=main_table, fieldName=cs_uri_stem in table main_table". [549277] Information about the extra MaxMind databases (Organization, ISP, and Domain) is missing from the documentation. [550333] Salang truncates fractional floating point numbers to 6 decimal points, when converting them to strings, and also unnecessarily converts parameters to strings and back when calling subroutines. This could cause graphs percentages to incorrectly show 0%. [550413] PDF report generation from the Scheduler on Windows gives an error if the pathname contains "\r" [552621] CSV files sometimes parse incorrectly when using multiprocessor parsing--headers are not distributed properly to all processes, resulting in some log lines being ignored. [553307] ODBC log sources do not support TINYINT fields. [554929] The Help link in reports replaces the current report with the documentation (instead of just opening a window with the documentation in it).
[555013] The Advanced licensing tier is still mentioned in the trial mode page, and the trial mode menu. [555297] Charset conversion does not occur on CSV export from the command line [555602] The "remove data" action does not complain when the -df option is used; it does not support the -df option, so it should throw an error message if -df is attempted. [558333] The Calendar report fails or crashes, if there is no data (no accepted entries) in the database. [560172] The Create Profile Wizard hangs at the database page (won't click past it), if the log format has only one numerical field. [560264] Report generation can hang when using the internal database, if a report contains more than one column displaying the same "unique" field. [560324] Verbose filter (-v f) output causes an error ("unknown value") when used with replace_last(). [560565] Database date range is reported incorrectly when all xref tables are disabled. [560818] Creating a profile using Cisco NetFlow (flow-export) format gives an error: Unknown configuration group "location" in node "profiles.*.log.fields" [560832] Emails sent by Sawmill from Windows systems have an incorrect SMTP Date header (off by one hour). [561105] The .= operator does not work on log fields. [563263] Several report filter examples involving date ranges, in the documentation, contain syntax or usage errors. [564442] Customizations to spiders.cfg are overwritten when upgrading [565897] The date_filter option "1day" selects the most recent day from now, rather than selecting the most recent day of the log data. [569311] Building a database from Firewall-1 NG (text export) Log Format gives an error, "Unknown variable 'elapsed' in expression," if the exported data does not contain an "elapsed" field. [569542] Import of Sawmill 7 database fails with the error "Syntax error in line ... of file ...; no '=' in itemnum record" when one of the items in the database contains a literal line break character. [569615] The CFG newsletter documentation refers to "1" when it should refer to "$1". [573522] The DSN parameter is not saved by the Config section of the profile. [576315] The options in "Admin > Preferences > Support & Action Email" show no "i" icon documentation. [580158] The report footer is not displayed in non-dynamic reports; headers and footers are not expanded in non-dynamic reports.
New features in 8.0.8:
[531179] Added tracking of the "action type" field in Barracuda Spyware Firewall / Web Filter Log Format. [532973] Created a plug-in to support the Palo Alto Networks Firewall Traffic Log Format. [550936] Improved the performance of database building for large Postfix datasets, and probably other large datasets, by defaulting to keep itemnums in memory, and to use rand_hash as the itemnums hashing function when none is specified. [554267] Added documentation of the custom actions (-a command line options) in LogAnalysisInfo/actions, so the -a section of the option documentation. [555124] Improved progress for index builds, to show the number of indices built, and the total number to build. [565553] Added support to the Firewall-1 (fw log -ftn export) Log Format plug-in for log variant with no date in the log file, but log files containing a date in this format _yyyy-mm-dd_, as in this example: o_2008-01-28_000000.log. [570348] Improved performance of filtered reports using indices on the main table (which is generally any filtered report other than those using date/time filtering). [574676] Improved performance of loading large normalization tables--this improves the performance of hierarchy table builds, and probably some reports also, especially when fields have a huge number of unique values. [574677] Improved progress display during build of hierarchy tables. [576873] Added a new custom action create_user (-a cu) for creating or updating users from the command line. [579290] Improved performance of database updates with external SQL database server, by enhancing itemnum upload to upload only new itemnums on database update, instead of re-uploading the whole list for every update.
Version 8.0.7, shipped April 17, 2009
Bugs fixed in version 8.0.7:
[520725] Common Access Log profiles with large datasets use a huge amount of memory on database builds. [528835] The parser adds "{default}" to the end of any log field value which ends in the hierarchy divider specified by its log field, even if the field is not hierarchical. I.e., {default} or "(default page)" sometimes appears unexpectedly in reports of non-hierarchical, non-page fields.
[529667] Some strings in the web interface are in English, even when non-English translations are used. [533349] Ironport C Series reports do not show recipients when message rewriting occurs in the log data. [534986] Session ID reports appear as integers in the Individual Sessions report, on profiles where the session ID is computed by log filters, instead of appearing as the computed value. [535446] The IPC folder of LogAnalysisInfo collects many files with names HTTPRequest*, which linger for hours; these could be cleaned up much sooner. [535576] Building a database for a MS SQL profile will give an error like, "drop the index 'main_table.mt_field_name', because it does not exist in the system catalog," if another profile, or a profile using different prefix/suffix, has used that same database to build a Sawmill database in the past. [536085] Firewall-1 NG (text export) Log Format format does not support hh:mm:ss format for the "elapsed" field. [536108] Database builds started from Config -> Database Info immediately display a "build completed" page, before the build is actually completed, when a profile is "real time." [536525] PDF report generation sometimes crashes on x64 Windows. [538883] Filters created in the Filters window using "is NOT item name" appear as "is item name" in the filter description of the report. [539107] During a multiprocessor database build, the IPC folder of LogAnalysisInfo collects several files with names like ParsingServerPort_*.done, which linger for days; these could be cleaned up much sooner.
[539683] PDF export fails from the scheduler when using a drive letter in the pathname, with an error like "Can't create directory c: (File exists)". [539696] Session duration shows 0 for profiles without a session ID field, and with "maximum session duration" set to 0. [539814] Individual Sessions report of imported Sawmill 7 profile gives error, "Unable to read file LogAnalysisInfo/Databases/mms_pub_imported/main/Tables/ssession_idsubitem/header.cfg". [540871] Deleting a database field does not delete it from all cross-references tables. [541655] Schedules configured to "update all profiles" run all updates simultaneously, rather than sequentially. [541975] The "Creating Many Profile in a Batch" topic is missing from the FAQ. [543436] Some command line operations give the rather cryptic error message "Couldn't find node licenses in" when there is no license installed. [543560] When using Firefox, page formatting is messed up if a report is generated, the server is stopped, the Admin page is accessed, the server is started, and the Admin page is accessed again. [543657] Profiles with a field named "level" give an error "invalid identifier" when used with Oracle. [543851] An internal string management issue could cause crashes during database builds or updates, when using a SQL database server. [544036] The "Session ID" profile option is not customizable in the web UI. [544295] Database updates can give an error, "Unable to allocate X bytes of memory (allocating preconversion buffer)" where X is a huge number. [545134] Running the Windows installer to upgrade overwrites the preferences.cfg and default_profile.cfg files from the previous installation. [546690] The label of the report element filter field has a typo: "epxression" [548666] Log data with pseudo-W3C headers starting with "# Date\tTime" (like some Exchange 2000 logs) does not parse, resulting in empty reports. [550414] Exporting PDF tables containing certain data can give an error, "Unknown HTML entity"
[553375] Improved performance of building indices, for the internal database, for large datasets. [553713] Memory usage can be very high on profiles using the internal database, with very complex fields (fields with many unique values).
New features in 8.0.7:
[538958] Added a -pp ("path page") command-line option for specifying the focus page for command-line export of the Paths Through A Page report. [539823] Added better detection of mobile web browsers. [541059] Enhanced Firewall-1 (fw log -ftn export) Log Format to handle logs with lines starting with dates; added some new fields. [545835] Added support for Linksys VPN Router log format [546006] Added the option to disable cross-reference tables individually, for better control over database build and report performance. [548920] Enhanced support for IPtraf logs, to support a variant with single-digit days. [550396] Improved performance of report generation in most cases. This is partly due to an increase in the default value of the "maximum paging caching buffer memory usage", and partly due to other optimizations. Profiles will benefit most when setting "maximum paging caching buffer memory usage" to the new default value of 64MB. In one example (a large pivot table), report generation speed increased from 13 minutes to 2 minutes; memory usage increased by less than 2x. [553685] Improved progress reporting for cross-reference builds and index builds, to show more granular progress while building each xref table or index (especially when using the internal database).
Version 8.0.6, shipped March 19, 2009
Bugs fixed in version 8.0.6:
[532899] Corrected typo that was breaking sessioning (sessions_id -> session_id) and restored session_id to database.fields. Renamed the field clips to successful_accesses. Removed c_rate from numeric fields because it is a value relative to 1 which means the standard rate. c_rate is now the Visitor Systems->Client Rate report instead of the Client Rate column. [533310] Date filter strings of the form last1d, last1m, last3m, etc (with single-letter units) generate an error, "The date filter is not in valid date filter syntax." [534897] The Paths Through A Page report uses much more memory than it should (several GB).
When using a prefix or suffix with MS SQL or Oracle or MySQL, the
expiration query fails with a "no such table" error.
The "file by file" option for log processing is not editable in the web UI.
Database build performance is slow in some cases due to splitting
processing across N+1 threads (when N is the number of processors or cores)
instead of the more efficient N.
Log parsing uses arbitrarily large amounts of memory when reading log data
from corrupt log data or log data with extremely long lines.
Command-line authentication login fails with error,
"No Permission
You don't have grants to view this page or profile. Please contact your system administrator for more details."
Profiles using filter_finalization give an error on database build,
"Internal: sourceFileNode=NULL during ConfigNode::ParseInfixStatements()"
Database build generates an error,
"Error in writing ODBC table main_table: ODBC error: rec1:
SQLstate: 22003; msg=[Microsoft][ODBC SQL Server Driver]Numeric value out of range;"
when importing data into
MS SQL which has negative integers in it.
Browsing to a UNC pathname like \\pub\pub\logs, on Windows, shows
nothing in the right panel of the File Browser.
Viewing reports sometimes gives an error,
"Attempt to get node number 0 from node v.temp_x_ticks."
Browse button gives the directory above the entered pathname,
if the pathname ends with a slash.
Global page headers and footers do not appear in reports.
The "Automatically update database when older than"
option is not saved properly in the UI.
Database builds could sometimes give an error, "Unable to read contents of
directory LogAnalysisInfo/TemporaryFiles/DeleteMe_...
(No such file or directory)"
Tasks scheduled to run at the same time run in sequence instead.
Temporary files are not cleaned up quickly enough, resulting in large numbers
of DeleteMe files in the TemporaryFiles directory.
Memory usage is very high when zooming to large reports.
Date/time graphs with one-minute granularity show no data.
"Default date filter" (global per-profile date range filter) has no effect.
Imported Sawmill 7 profiles sometimes give error,
"Couldn't find node 0 in language.english.lang_stats.weekdays_short."
Removing all cross-reference groups gives an error,
"Couldn't find node xrefs in v.fp."
The Paths Through A Page report takes a very long time, and
a huge amount of disk space.
The date filter expression "last1month" (and similar ones)
give an error about incorrect date range format.
A database removal operation on a MS SQL database gives an error,
"Unable to Execute ODBC Query='delete from main_table x using main_table x where 1=1'; diagnostics=ODBC error: rec1: SQLstate: 37000; msg=[Microsoft][ODBC SQL Server Driver][SQL Server]Line 1: Incorrect syntax near 'x'.;"
Months report of imported Sawmill 7 profile, where the date/time report
had been edited in Sawmill 7, gives an error,
"Unable to read file LogAnalysisInfo\Databases\\main\Tables\rows_...o\header.cfg (No such file or directory)"
Microsoft Media Server plug-in gives wrong results for the c-rate field.
Generating a report with a customer date range filter string gives an error
"Couldn't find node in."
Reports give an error, "Couldn't find node modification_time in v.lang_stats_file_info," when English language is not installed.
Reports show wrong numbers for fields containing single field values
greater than 2 billion, when using multiprocessor database builds
on a 32-bit system.
The "Process subdirectories" checkbox erroneously states that it is for
"local folders only."
ODBC log sources do not support LONGVARCHAR (a.k.a. TEXT) or BIGINT fields.
Microsoft Media Server profiles show 0 for session duration.
W3C log data (like IIS log data) is very slow to parse,
when using multiprocessor database building.
Date filter strings of the format "last1d", "last1m", and others using
single-letter units, give an error, "The date filter is invalid."
The Profile menu in an action in the Scheduler is not alphabetized.
The "automatically update when older than" option sometimes has no effect.
The "configure" script does not complain when there is no C++ compiler,
when building Sawmill from encrypted source, which causes the compilation
to fail later when it attempts to compile the C++ files.
New features in 8.0.6:
[486806] Added new plug-in to support the Palo Alto Networks Firewall Traffic log format.
A new option has been added for doing LOAD DATA imports into local MySQL
database servers, which is faster than the default LOAD DATA LOCAL INFILE,
and works on servers where local infile is disabled, but requires the MySQL
server to be on the Sawmill server.
A new variant of Unix Daemon Messages log format is now supported.
The name of the Sawmill executable has been changed to just "sawmill"
on non-Windows platforms, instead of having the version number in the name.
Sawmill now determines the amount of physical RAM on the system,
and when splitting queries across multiple threads with "auto,"
it ensures that each thread has at least 2GB of RAM.
A new option, "maximum read block size," controls how much memory
can be allocated to holding a single line of log data.
Version 8.0.5, shipped February 26, 2009
Bugs fixed in version 8.0.5:
Fixed a bug which could cause an error when searching the documentation
in Windows.
Fixed a bug which could cause an error on builds with MS SQL databases, if there was
a database field called "rule."
Fixed a bug which could cause database corruption if the main table of the database
was more than 4GB, with 32-bit versions of Sawmill. This generally happened
with datasets more than about 30 million lines. It could cause a variety of symptoms,
including out-of-memory errors while building indices, crashes during reporting,
and incorrect numbers in the reports.
Fixed a bug which could cause an error "Parsing server returned itemnums list for unknown database field 'fieldname'"
when building two databases simultaneously on Windows.
Fixed a bug which could cause an error "Unable to read file ... sessions_join"
when no valid log entries were found in the dataset.
Moved the MySQL DLL to the Sawmill installation directory, to reduce conflicts with PHP and other programs
which install different versions of the MySQL DLL.
Added support for MySQL on the x86 Linux ES4 platform, where is was disabled.
Restored the "create many profiles" script (now in LogAnalysisInfo/util), which was missing earlier
versions of Sawmill 8.
Fixed a bug where multiprocessor log processing would fail with a parsing server error,
if the "Server hostname" was not empty in Preferences.
Fixed a bug where lines of log data could be split incorrectly,
for log formats where quoted values can span multiple lines,
if the lines spanned the boundary between log reading blocks.
This could result in slightly low numbers.
Fixed a bug in Shoutcast 1.8 processing, which could cause a regular expression error on parsing.
Fixed a bug which could cause incorrect sorting of rows in tables on 32-bit systems,
if they contained large floating-point numbers (more than about 4 billion).
Fixed a bug which caused the graph tick marks and labels to be in the wrong locations, when rendered by Firefox.
Fixed a bug which could cause an error if all log filters were deleted.
Fixed a bug where the Create Profile Wizard incorrectly validated the pathname
when regular expressions were used, causing a "no such pathname" error in some cases
where the pathname was correct.
Added the "maximum paging caching buffer memory usage" option to the web UI.
Removed some database optimization options which no longer have any effect in Sawmill 8.
New features in 8.0.5:
Enhanced support for Smarter Mail log format to handle dates containing dots.
Enhanced support for Wowza log format to handle large negative x-duration values (due to logging or server bugs).
Enhanced support for Limelight log format, to handle spaces in #Fields lines, in addition to tabs.
Added the "integer bits" option for database fields to the web GUI.
Improved handling of v7-to-v8 database conversion in the case where the database directory
was overridden--it now prompts for a new directory for the Sawmill 8 database directory, so it won't
overwrite the Sawmill 7 one.
Version 8.0.4, shipped February 13, 2009
Bugs fixed in version 8.0.4:
Fixed a bug which could cause progress reporting to halt partway through long database builds,
on 32-bit Windows.
Fixed a bug in the Intermapper Event Log Format plug-in that caused an error if Events was not selected as a numeric field during profile creation.
Fixed a bug where reporting would hang in CGI mode, if a "temporary directory" was set.
Added "always include bottom-level items" to the database fields editor.
Fixed a bug which could cause an error when processing Blue Coat W3C logs, if the log data contained
both a cs-uri-path and a cs-uri-stem field.
Fixed an error, "Attempt to join main_table to filtertmp_5652_1 on 's.session_id = filtertmp_5652_1.itemnum', but there must be one column from each of the two tables in the ON condition, and there doesn't seem to be", which could occur when zooming in on a particular session in DIndividual Sessions.
Fixed a bug which would cause an error if a template was edited in an Advanced installation,
though template editing is permitted by Advanced licensing.
Fixed a bug which could cause an overflow when reporting processing time in ISA 2004 log data.
Fixed/improved cleanup of temporary files in LogAnalysisInfo.
Fixed a bug where PDF generation would fail if the report contained a pie chart with a legend.
Fixed a bug where "-a rdi" (rebuilding indices from the command line) gave an error when used with the internal database.
Corrected references to SawmillCL.exe, which is now called Sawmill.exe (in version 8).
Fixed a bug where the Cities report showed States instead, when reporting on W3C log data (like QTSS).
Fixed a bug where the Logout link did not work on the first click, in Firefox.
Fixed a bug where the Support Email Address in Config -> Miscellaneous -> Support & Action Email, did not save.
Fixed a bug where the Config -> Reports page would not display when logged in as a Manager.
Fixed a bug which could cause crashes when using the page tagging server on Windows.
Fixed a bug which could cause 0s in reports, when clicking single date item while displaying a
date report at a different level (e.g., clicking a month, while displaying the Days report).
Fixed a bug which could cause an error when there was a database field named "count".
Fixed a bug where long values could overlap the columns to the right of them, in HTML tables.
Fixed a bug where database updates reported themselves as "Rebuild Database" in the Database Info page.
Fixed a bug where the command-line progress display did not show the number of lines processed.
Fixed a bug where the Sessions Overview report did not change when filters were applied.
Fixed a bug which could cause some log data not to parse, if the "sessions visitor ID" field was empty.
Added the End User License Agreement file to the distribution directory (it was in the installer, but not the installation).
Fixed a bug which could cause an overflow when displaying time-taken for Blue Coat W3C logs,
on a 32-bit system.
Fixed a bug which would cause an error when filtering a report, if there was a SQL table prefix or suffix specified
in the profile.
Fixed a bug which would cause 0 reports when using a regular expression filter on a non-itemnum field (like hour of day).
Fixed a bug which could cause an error, "Unknown variable 'lang_stats.field_labels.time_stamp' in expression",
when reporting on Common Access log data.
Fixed a problem with the formatting of FAQ pages.
Fixed a bug in iMail parsing which caused an error:
'Unknown configuration group "from" in node "profiles.qmail.log.fields".'
New features in 8.0.4:
Modified the Intermapper Event Log Format plug-in to find the year in the log file name it contains the pattern LogYYYYMMDD.
Added an option to RBAC to disable password changing.
Extended Sidewinder Firewall reporting to show countries, regions, and cities for each IP.
Added suport for Squid format logging to Unix Syslog, with a Squid timestamp.
Added support for a new variant of Mirapoint SMTP log format.
Improved the performance of some internal database operations used for indices and uniques tracking.
Improved the performance of some internal database queries involving the sessions table.
Version 8.0.3, shipped January 23, 2009
Bugs fixed in version 8.0.3:
Fixed a bug which could cause database builds to terminate unexpectedly. This occurred with ASA log data, but
could theoretically happen with any log data.
Fixed an issue with the copyright in the Czech translation.
Fixed/enhanced the TaskLog file to suppress the execute_sql_query, which were numerous and mostly useless.
Added documentation for the "Remove Reloads From Sessions" option.
Corrected the MacOS Install.txt file to remove incorrect upgrade instructions.
Fixed a bug which would cause an error when zooming on a session user, and zooming to the Session Pages report.
Fixed a bug which could cause an error, "Error in get_progress_state.cfv" when displaying the Overview.
This bug was caused by a failure during the database build, so profiles showing this behavior will need
to have their databases rebuilt.
Fixed a bug which could cause database builds to hang, when built from Config -> Database Info,
if an error occurred while the database was being deleted.
Fixed a bug with the v7-to-v8 profile converter, which would cause errors when viewing reports,
if the v7 profile contained references to database fields which were not translated in Sawmill 8.
Fixed a bug which could cause an error during database build, if the database directory
was on a different drive or partition from the LogAnalysisInfo (installation) folder.
Fixed a bug where the current_log_pathname() function did not work when using parsing servers
(multithreaded database builds).
Fixed a bug which would cause an error, "Syntax error: Expected ) in expression; found '" when
creating a profile from BlueCoat W3C data.
Fixed a bug which could cause an error when relocating the database directory.
Fixed a bug which could cause sporadic (though harmless) crashes of the web server.
Fixed a bug which could cause an error, "Unknown configuration group 'cs_uri_stem_bottom_level_items'" when viewing
reports for a Microsoft Media Server profile.
New features in 8.0.3:
Improved the Windows installer to omit the minor "rcN" from the version number,
to simplify the Sawmill version number display, and make it clear that these
is a production releases (successful release candidates), not pre-production releases (unsuccessful release candidates).
Improved documentation of "session events" and "sessions."
Added section to documentation, "Adjusting date/time values for Daylight Savings Time".
Enhanced the Limelight plug-in to handle arbitrary W3C headers, allowing variable field layout.
Enhanced the Squid plug-in to handle "action" field values containing spaces (like "TCP MISS").
Version 8.0.2, shipped January 16, 2009
Bugs fixed in version 8.0.2:
Fixed a bug where the "automatically update if older than" option was not saved in Config.
Fixed a bug which caused an error when creating an ISA profile: 'Unknown configuration group "filterinfo" in node "profiles.isa_logs.log.fields".'
Fixed a bug where some hierarchical reports would show no results when drilled into.
Fixed a bug where direct login by URL would sometimes give an error.
Fixed a bug which would give the error 'column "" does not exist' when analyzing Bluecoat log data.
Fixed a bug which could cause an error when using a literal backslash in a custom report expression.
Fixed a bug which could cause an error while analyzing Nortel ACD log data:
'Unknown configuration group "avg_agent_time_busy" in node "profiles.test.log.fields.'
Fixed a bug which could cause an error when clicking Scheduler after importing v7 profiles
with field names containing numbers.
Fixed a bug where full-month filters on the Overview gave zero results.
Fixed errors compiling the encrypted source on SuSe 11, and other recent operating systems.
Fixed a bug where the progress display for one profile could show in another profile's Reports display,
if there were two simultaneous profiles in use.
Fixed a bug where x86 Windows was reported as x64, and vice versa, in automated bug reports.
Fixed several places where images were broken in CGI mode.
Fixed a bug where very long field values could generate an error, when using ODBC.
Fixed a bug which could cause an error when analyzing Sidewinder log data with Microsoft SQL Server.
Fixed a bug where Log Detail report was empty when using ODBC databases.
Added support for not splitting queries at all, in the web interface.
Fixed a bug where some URLs used HTTP, even when running Sawmill in CGI mode under an HTTPS server.
Fixed a bug where parsing servers could linger after the process that spawned them was gone,
in the event of a parsing error.
Fixed a bug which caused an error, 'Internal Error: Empty node name,' when setting a session field to "(None)".
Fixed a bug which caused an error when generating the Top Malware report in Ironport S-Series logs,
'Couldn't find node display_format_type in sessions_cache.xyz.profiles.profilename.extended_profile_dat.'
Fixed a bug which could cause an error like, 'Unable to read file LogAnalysisInfo\Databases\profile\main\Tables\rows_1_6248o\header.cfg'
on location reports, in imported v7 profiles.
Fixed a bug which could cause sporadic crashes of the web server process.
Added the update_xrefs_on_update option to the web interface.
Fixed a bug which caused the Help link to sometimes have no effect.
Fixed a bug where pivot tables would sometimes show NULL rows.
Fixed a bug which could cause errors when using a SQL prefix or suffix.
Fixed a bug which could cause an error, "Unknown variable 'lang_stats.field_labels.ms_ras_client_name' in expression."
Fixed a bug which could cause an error when analyzing Sidewinder logs with MS SQL.
Fixed a bug where Config/Reports did not prompt to save changes.
Converted some hard-coded strings to language module strings, for internationalization.
Fixed a bug which would cause the error, "Couldn't find node next_pages in v.query_result.data" when
viewing a Session Paths report with no data.
Fixed a bug which prevented error messages from being viewed when not logged in.
Fixed several broken images in the documentation, in CGI mode.
Fixed a bug where non-root-admin users could not see the version number of Sawmill in the web UI.
Fixed a bug which could cause the error, "Unable to read...\main\Tables\sessions_join\header.cfg (Day of week)."
Added the missing language module variable lang_admin.log_filters.simplify_playerid_label.
Fixed a bug which would cause error when rebuilding a database, if it was stored a different
drive or partition from the LogAnalysisInfo folder.
Fixed a bug which would cause database build errors when the log data contained extremely large or infinite numbers.
Restored the missing FAQ about emailed report and Outlook 2003.
Fixed the FAQ about resetting the admin password to describe the new method.
Fixed a bug where the SPARC Solaris 9 distribution did not have the necessary libraries included.
Fixed a bug where Config -> Log Source did not support multiple log sources in Professional tier.
Fixed a bug where once a trial license expired, it would not accept another license key.
Restore the Support page from Sawmill 7, to Sawmill 8.
Restored the HTML comment describing the filters, from Sawmill 7, to Sawmill 8.
New features in 8.0.2:
Updated the Czech, German, and Polish translations.
Enhanced Radware DefensePro plug-in to handle ip:port variation.
Enhanced Flash Media Server plug-in to support analysis when there is no cs_uri_stem field.
Implemented a custom action, reset_root_admin, for resetting the root administrator password from the command line.
Added more documentation about real-time log importing.
Version 8.0.1, shipped December 24, 2008
Bugs fixed in version 8.0.1:
Fixed a bug where once a trial license expired, Sawmill would not accept a new license.
Fixed a bug where a Sawmill 7 license would be called "invalid" instead of being reported as a valid older license, no longer valid for Sawmill 8.
Fixed the upgrade instructions in the README of some platforms, which were still describing the Sawmill 7 upgrade method.
Fixed a bug where Sawmill 8 would not install its service, if Sawmill 7 was already installed, and would uninstall the Sawmil 7 service when Sawmill 8 was uninstalled.
Fixed a bug in the Create Profile wizard, which would cause an error if the Pathname was not literally a valid existing pathname,
even if it contained wildcards which should have matched valid pathnames. This almost always caused wildcard or regular expression log sources to fail on profile creation.
Fixed a bug where wildcard and regular expression report filters did not work (generated an error when attempted) with Microsoft SQL Server profiles.
New features in 8.0.1:
[604421] Enhanced documentation search to show a match when any word in the phrase matches, instead of requiring the whole phrase to match. [610256] Added a new custom action, convert_version_7_user, to convert users from a Sawmill 7 installation using the command line.
Added a separate chapter to the documentation about real-time importing.
Added a Support link to the Admin page.
Added the Salang expression of the current filter to the HTML of the report, as an HTML comment (useful for creating command-line or scheduled filters, or for debugging).
Chopped off the "rcN" part of the version number in the web interface, to make it look better.
Switched the default port of the web server to 8988. This makes it much simpler to run Sawmill 7 and Sawmill 8 together on the same system.
Enhanced support for ISA W3C format, to handle a significant variant (2007)
Added support for Unicode with Microsoft SQL Server as back-end database, so non-ASCII log data can be imported
and stored in MS SQL, and queried from outside Sawmill.
Version 8.0.0, shipped December 12, 2008
Bugs fixed in version 8.0.0:
New features in 8.0.0:
Changed the GUI concept from html frames to single pages.
Added report fields for more flexibility and fine tuning of report elements and table data.
Added a simplified date_filter syntax (i.e: 2m, last2m, etc.) for date/time filtering.
Added a new caching system which caches various report components and database data independently.
Added RBAC (Role Based Access Control)
Added support for sequential actions per schedule in scheduler.
Added a "Run Now" button in the Scheduler, to run any task immediately.
Added log fields editor
Added database fields editor
Added session fields editor
Added report fields editor
Added new field wizard (which allows to create a log field, database field and report field at once)
Added URL support to view reports by URL definition by defining: profile name, report name, date filter, filter expression, filter comment
Added support to view any hierarchical depth of a hierarchical database field as non-hierarchical report. This allows i.e.: to view a months report or a region or city report. (These reports have been added by default)
Added the calendar as optional report.
Added support to dynamically create a pivot table within the reports GUI
Added support for different sort field and sort direction of the drill down field.
Added support to drill down data on a table with multiple string fields.
Added a new date picker which combines single date, date range and relative date selection.
Changed the zoom concept in that zoom automatically adds the zoomed item to filters.
Added support to zoom to multiple items at once.
Added support to save filter items as filter group
Improved the filters editor.
Added support to email a report within the reports GUI
Added support to define the row numbers and aggregation rows when exporting a table within the reports GUI
Added min and max aggregation rows to tables.
Added a row_visibility_expression per report element. This expression allows to show/hide table rows by an expression, i.e. show only rows where page_views > 300 and page_views < 1500.
Added support for a default date filter per profile.
Added support for a date filter per report or per report element.
Improved the Customize Report Element form/options.
Added table column info support.
Added table row selection support (to mark a row in yellow color).
Added support for 3D pie charts
Added support for antialiased PNG graphs
Added support for use of MS SQL or Oracle databases as back-end databases (where processed log data is stored, and reports generated from).
Added support for use of MS SQL, Oracle, or MySQL database as log sources
Implemented multiple scalability improvements, to allow Sawmill to process log data with less memory usage. These largely lift the restrictions of log processing on 32-bit systems, so any amount of log data can be processed without exceeding the address space of a 32-bit system.
Added real-time reporting. Reports can be generated while data is being imported, and will be up-to-the-moment, based on the latest imported data. Data can be streamed continually into the database, without any reporting downtime.
Implemented major SQL performance improvements, especially in the building of cross-reference tables, and in the performance of database update.
Enhanced the internal database to support SQL syntax, for universal SQL querying of any Sawmill database.
Added support for reading log data from a SFTP server
Added directory recursion on (S)FTP servers, so a log source can point to a directory, and all subdirectories will be processed.
Added filtering of reports on numerical fields; e.g., show all events where bytes > 1000
Implemented multiprocessor splitting of report queries: report calculations are split across multiple processors for better performance.
Enhanced multiprocessor and multi-system log processing. Log process is now done more efficiently on a single system, and does not use the disk as much; and it is also possible to split log processing across multiple "parsing servers" on the same network.
Added database import and export.
Added support for user-created actions (-a options), with fully customizable parameters and behavior.
Version 8.0.0b8, shipped December 06, 2008
Bugs fixed in version 8.0.0b8:
Fixed a bug which would cause an error when creating a BlueCoat W3C profile.
Fixed a bug where "start time" and "end time" were integers in imported MySQL profiles, rather than bring formatted timestamps.
Fixed a bug where W3C log data (and other log data with header lines) would not be imported properly
(some lines would be dropped) when using multiprocessor log parsing.
Fixed a bug where the "index" attribute of database fields (whether to index the database field) was not editable in the web interface.
Fixed a bug where the database name was required in Config -> Database -> Server, when it should have been optional.
Fixed a bug where the web interface did not have an option to *not* use distributed parsing.
Changed xref tables to be hierarchical by default, which results in faster report generation, especially for large datasets,
but somewhat slower xref builds on import.
Improved performance of the "session pages" report, and other reports which use indexed joins internally.
Fixed a bug where the "use overview for totals" option had no effect.
Fixed a bug where the "use overview for totals" option was not editable from the web interface.
New features in 8.0.0b8:
Version 8.0.0b7, shipped November 30, 2008
Bugs fixed in version 8.0.0b7:
Fixed a bug where the Reset/Collapse All button did not display properly in the Session Paths report.
Added the "User Overview For Totals" option to the Config interface.
Added editing of the Index option to the Database Fields editor (to specify which database fields should be indexed).
Fixed multiple bugs in reporting, which caused error when generating reports with a SQL database table prefix or suffix.
Fixed a bug where Windows pathnames were shown with slashes doubled, in error messages.
Fixed a bug where entering a directory pathname ending with a slash, then clicking Browse, would browse the parent of the directory, rather than the directory itself.
Fixed a bug where the Report Menu would not be displayed, when viewing a report with no elements.
Fixed a bug with Internet Explorer 7, where the Date Picker did not display properly.
Fixed a bug which caused an error when viewing reports for profiles with a database fields called "from" (or other SQL keywords), e.g., Postfix.
Improved performance of Squid log processing.
Fixed a bug where the expiration date always appeared as Feb 4, 2008, in the Licensing page.
Added links to documentation examples, from the Report Filters editor.
Fixed a bug where Config -> Database -> Server required a database name to be entered, even though it should default to the profile name when empty.
Fixed a bug where the Date picker showed a red "undefined" as the first day of the week.
Fixed a bug where the ODBC form required a username to be entered, even of the DSN had one embedded.
Removed "log processing threads" from the web interface; it is deprecated.
Fixed a bug where some windows did not have title bars, in CGI mode.
Fixed a bug where CGI mode would fail when running queries, if they were large enough to be split across multiple processors.
New features in 8.0.0b7:
Version 8.0.0b6, shipped November 29, 2008
Bugs fixed in version 8.0.0b6:
Fixed a bug where the Windows install included an unnecessary DLL in the installation directory.
Fixed a bug where the Calendar report was not enabled on imported v7 profiles.
Fixed a bug which could cause an "Unknown hash function" error when building some types of profiles,
including Microsoft Media Server.
Fixed a bug which could cause progress reporting errors, if there was error during a database build or report generation.
Fixed a bug which could cause a crash during autodetection, if there was an error in autodection (instead of reporting the error).
Fixed a bug which could cause random results in the Overview, if the filter set contained no rows, and the database
type was Microsoft SQL Server.
Fixed a bug where Microsoft SQL Server databases were not updated properly during a database update; in particular,
the itemnum (normalization) tables were not updated.
Fixed a bug which could cause an error when zooming on session fields, and zooming to session reports.
Fixed a bug where MySQL databases were not imported successfully from Sawmill 7 profiles.
Improved autodetection of Instagate log format, so it doesn't detect Apache Combined as Instagate.
New features in 8.0.0b6:
Version 8.0.0b5, shipped November 27, 2008
Bugs fixed in version 8.0.0b5:
Fixed a bug where relative date filters (e.g., "last 4 months") could cause an error.
Fixed a bug which could cau |
