 |
ALL PLUG-INS
Sawmill has plug-ins to support the following log formats: |
 |
This is the version history for Sawmill 8.
The Sawmill 7 version history is here
and the Sawmill 6 version history is here.
Version 8.1.5, shipped June 2, 2009
Bugs fixed in version 8.1.5:
[797770] Update a database after removing data can cause an error like, "Short read in SingleWindowCachingBuffer file LogAnalysisInfo/TemporaryFiles/Task24630/ris_main_table_x_loadorder_le_1014599626/data.tbl;
tried to read 1 bytes from position 1014599626, but only got 0 bytes." [798195] Creating a profile using the WarFTPalt plug-in causes a Sawmill alert, 'Unknown configuration group "destination_ip" in node "profiles.profilename.log.fields"' [798479] In the Session Paths report the pages are not in chronological order. [806410] Bug fixed where percentages were wrong when "Use overview for totals" was set for reports, and "Show total value" was set for columns of type Unique. [808447] If Users/Roles (RBAC) has "view only" changes made to permissions, it now no longer displays a "confirm message" asking the user to save changes before leaving the page. The toolbar message is also improved to be "Your grants for this page are limited to view and changes cannot be saved." [814810] When an incorrect password is entered for an SFTP log source, the error message is "unknown keyboard-interactive response: 1". [816641] Fixed bug which could cause 0's in the Overview, after removing data, and updating with new data. This could also cause an error about reading past the end of a temporary file. [818357] Using an FTP log source to certain versions of Pure-FTPd server can result in an error like, "Unexpected response from FTP server: 150 4.977 seconds (measured here), 0.50 Mbytes per second." [823230] Bug fixed where Internet Explorer 8 shows skewed calendar months. [839414] Filtering on a single second gives an error like "No date applied. The date
filter '7apr1998_165306'" is invalid.'" [842123] Export of a database from the command line (with -a ed) generates empty itemnum tables. [843395] Command-line export (-a ed) of a MySQL database gives an error, "No database selected" [844142] Building an Oracle database with some versions of the Oracle Client driver gives an error, "Internal Error: Attempt to get SQL_INTEGER value from a SQL_FLOAT"
New features in 8.1.5:
[712684] Slightly enhanced the Admin menu so clicking Profiles reloads the profiles list, even when already displaying the profiles list. [800148] Added support for Microsoft Exchange Server log data, logged through a syslog server. [814809] Extended the command-line authentication protocol so the script can print *ROLE*:N to indicate that the user logging in is part of role N (N is the internal role node name). [815421] Added sorting of the Log Detail report. [816854] Added a "Automatically direct to reports upon login if user accesses only one profile" to the Users editor. [821749] Added support for an additional UNIX-syslog style header (after the first syslog header) in ISC DHCP log format.
Version 8.1.4, shipped March 30, 2010
Bugs fixed in version 8.1.4:
[767223] When using multiple log sources, the progress displays count of bytes processed restarts at zero for each log source, instead of showing the total cumulative bytes processed. [770870] The built-in Salang function convert_charset() allocates memory equal to the string to be converted, and does not deallocate it. This can result in high memory usage over long builds, if log filters use this function. [785338] After doing a "remove database data" operation, the Overview hangs for a long time with no progress display. [785650] The week_of_year field uses 0 for the first week of the year, instead of 1 as documented. [790229] The "config" file of a database is not updated when xrefs are rebuilt using the "-a rcrt" command-line option. [791207] When using the command-line authentication script, the script is called for every click, rather than just once per login session. [793905] After creating a report using "Generic W3C Web Server" as the format, reports give an error, "Unknown variable 'volatile.new_profile_name' in expression" [794485] On MacOS 10.6, the "run at startup" option does not start Sawmill successfully. [795215] Multiprocessor builds of profiles using the global date regular expressions option, can incorrectly reject some log entries. [796244] Fixed problem where files with Kiwi ISO syslog headers autodetect as "Kiwi (mm-dd-yyyy dates)" as well as "Kiwi (ISO/Sawmill)". The ISO files have the year first (yyyy-mm-dd), so selecting the wrong syslog completely prevents the logs from parsing. Files with the year last in the date are detected as both "Kiwi (mm-dd-yyyy dates)" and "Kiwi (mm-dd-yyyy dates)" so that the correct date can be selected. Now files with the year first are only detected as "Kiwi (ISO/Sawmill)". [798584] The SHOUTcast W3C Log Format plug-in did not parse correctly after encountering a second W3C field header in the same file. This bug was introduced in Sawmill 8.1.3 in a change to the plug-in that worked around for a logging problem in SHOUTcast 1.9.8. [798843] Log Detail gives an error, if the number of rows in the filter set is less than the number of rows displayed in the table, like: "Unable to read 392 bytes from LogAnalysisInfo\Databases\profile\main\Tables\_select_result_1344_
0\data.tbl.moved; got only 0 bytes". [799951] The built-in Salang function collect_listed_fields() does not work properly when the divider or separate parameters are not constants. [804591] When using command line authentication, logging in as root administrator causes an error like: Unknown configuration group "user_grants" in node "sessions_cache.479a3ff1bcb1ea9ba79e4f2113018196.session_info" (02705354).
[806853] Profile patterns entered in the Scheduler were are treated as patterns unless they are preceded by "pattern:". [806929] The Windows installer does not properly install the Microsoft 2008
redistributable packages, which are required for Sawmill to start on
some older versions of Windows.
New features in 8.1.4:
[686796] Support for a new Windows Event Log variant has been added. It is comma separated and has a m/d/yyyy date format. The fields are Level, Date, Time, Source, Event ID, Task Category and Message. The plug-in was tested using files from Vista logs with 24 hour times and Windows Server 2008 logs with AM/PM times. [781872] Support has been added to the FortiGate Traffic Log Format plug-in for a format variation that is comma instead of space separated. [799782] Added support for a format variant of the Astaro SMTP Proxy Log Format which has a single space after between the To email address and the "Reason". Other variants have a field there whose value is ignored. The result of not allowing this was rejection of most To lines (=> lines).
Version 8.1.3, shipped February 12, 2010
Bugs fixed in version 8.1.3:
[742760] When the language in the Preferences does not match the language for the current user, reports can contain a mix of languages. [744621] Database build, using an Oracle database, with a database field whose internal name exceeds 30 characters, fails with an error "ORA-00972: identifier is too long" [752760] If all lines of log data are rejected during log processing, the resulting database, which correctly shows zeros for all values, incorrectly shows "01/Jan/1970, 1 day (entire date range)" as the date range in the Reports page. [752760] The date range at the top of Reports shows 1970, when there are no entries in the database. [758276] When the Log Detail is filtered, then paged, the second page data matches that of the first page. [759965] Building a database with more than 64 main table indices (e.g., a default profile with more than 64 non-aggregating field) gives an error: Too many keys specified; max 64 keys allowed. [763994] Apache Custom format strings containing literal \" values do not parse properly. [765835] The Concurrent Session number is computed from the unfiltered session data--report filters have no effect on it. [767731] The "+" operator does not match properly in regular expression log sources when using Show Matching Files in Config->Log Sources. [767843] The Windows installer does not install the VC++ 2008 redistributable libraries, which can cause errors when trying to start Sawmill, if the libraries are not already installed. [772469] SFTP log sources give an error "Unknown SSH server prompt 'Password: ' -- only 'Password:' is supported" on certain SFTP servers (specifically, SUSE Linux Enterprise Server 10). [775640] Building a database with Microsoft SQL Server, in a profile with a session analysis, where the name of the session page field is "file" or some other reserved SQL keyword, gives an error like, "Incorrect syntax near the keyword 'file'." [776369] When password expiration is configured in Preferences, an error occurs when the password expires: "Couldn't find node login_plugins in". [776937] The Session Paths report can cycle back on itself, making sessions infinitely deep, when using the web UI.
New features in 8.1.3:
[704687] Merged changes from Sawmill 7 that were never replicated in Sawmill 8. This adds support for a new format variant and extracts some additional fields. [717071] Added support for an all caps string instead of a hex number after the url in the Barracuda Spyware Firewall / Web Filter Log Format plug-in. This field is ignored in all cases because it's purpose is unknown. [718244] Reporting of pass throughs (BP events) was added to the Cisco Wide Area Application Services (WAAS) TCP Proxy log format plug-in for WAAS 4.1.x. Counts of connection starts, pass throughs and two types of connection ends were added to the numeric fields. [748047] Implemented support in the SHOUTcast W3C Log Format plug-in for a logging problem in SHOUTcast 1.9.8 which causes numeric values Play Duration, Server-to-client bytes and Average Bandwidth. The W3C log format specifies that if there is no value in a field, it must be replaced with a dash, but SHOUTcast logs sometimes have a completely empty cs-user-agent (player) field which caused the remaining fields to be shifted and the value of sc-bytes to end up in the x-duration field. The modified plug-in compensates for this problem. This is an important concern for Soundexchange RIAA reporting.
Version 8.1.2, shipped December 10, 2009
Bugs fixed in version 8.1.2:
[629538] When using MYSQL 5.1, database data removal fails with an error, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'as x
using main_table as x where (x.date_time < 'YYYY-MM-DD HH:MM:SS')' at line 1" [634747] Changed the way play duration is distinguished from non-play duration in the Wowza Media Server Pro plug-in. This fixes a bug in 8.1.1 where the stream duration is incorrect when there are pauses or seeks. Also restored full tracking of x_duration in the stream_duration field and added the play_duration and pause_duration numeric fields. Added check for no publishing start time because we it is possible to have no c_client_id values on unpublish events. [676495] Reports show incorrect results, when zooming on multiple days in the Days report, and simultaneously zooming on multiple items in some other report. [680285] Profile creation fails, when using Aventail Client Server log format, with an error, "Could not find main_report_field_name in log format report description 'connect_tunnel_sessions_overview'".
[686802] When using the Microsoft ODBC for Oracle driver, Sawmill gets an error on database build: Internal Error: Attempt to get SQL_DOUBLE value from a SQL_DECIMAL ODBC table column: 0. Note: with this bug fix, this error no longer occurs; however, other errors may occur because Sawmill requires an ODBC 3 driver, and the Microsoft ODBC Driver for Oracle does not support ODBC 3 (it supports only ODBC 2). Therefore, do not use this driver with Sawmill; instead use the Oracle Client driver (provided by Oracle), or another third-party ODBC driver for Oracle. [686819] Building a database in a profile with session information, using an Oracle back-end database with the Oracle Client driver, gives an error, "Internal Error: currentBufferColumn=8 in UploadChunk() for sessions_update." [686901] If two reports are run simultaneously against a real-time database, the second report can "collide" with the resuming database build, causing various errors. [688118] Database xref tables, in the internal database, are much larger on disk that they need to be, for some datasets. [691995] Internal database indices skip a row when being updated. Among other possible effects, this can cause one day of data to be missing from the reports, and the Date Picker. [692073] The order of log sources in Config -> Log Source is not permanently saved when they are manually reordered. [693811] The name of the plug-in the Generic W3C Log Format has changed to Generic W3C Web Server Log Format to indicate the basic kinds of fields it expects to find. The way fields are created has been changed overcome a bug where the creation of some derived fields with functions caused an error because the database fields did not exist. [699372] A Microsoft SQL Server log source give the error "Invalid log source. Numeric valid out of range" when one of the selected columns is a BIGINT, and one of the values is larger than about 2 billion. [703189] Use of "create_many_profiles" gives an error: "Syntax error: Unknown operator in expression" due to a syntax error at the keyword "string". [707419] HTTPS does not work on Windows (it starts an HTTP server instead). [707422] Database updates using an Oracle database server give an error, "ORA-00001: unique constraint (STAT.LOADORDER) violated" [710643] Double-quoted W3C fields values which with with a literal double-quote (escaped as two double quotes), e.g. appearing literally in the log data as """"hello""" to indicate a value of a double-quoted value hello, are treated as an empty value. [713843] Importing the database of a Sawmill 7 profile using MySQL, and changing the database name during import, gives an error, "Table '.locationitemnum' doesn't exist at ../src/run.cpp:2768" [717967] Database updates can crash during the index build/merge step, for very large datasets. [725411] The Overview report in the Single-page Summary sometimes shows incorrect average bandwidth values. [737620] ISA CSV log data is very slow to process, due to a problem normalizing the highly unique filter_info field. [741574] With IronPort C-Series logs, relayed SBRS events are reported as "rejected." [743313] Filtered date/time reports can display graphs from previous cached filtered versions of the report, instead of the correct graph.
New features in 8.1.2:
[573418] Added basic support for keyboard-interactive sshd authentication; this allows SFTP access to default FreeBSD systems, and others using keyboard-interactive with a "Password:" prompt. [731045] Added support for Windows 7 detection in user-agent string.
Version 8.1.1, shipped September 8, 2009
Bugs fixed in version 8.1.1:
[659410] Creating a profile with Mirapoint SMTP Log Format gives an error, 'Unknown configuration group "date_time" in node "profiles..log.fields"'.
[662875] When using Professional trial mode, Save Report To Menu gives an error, "Checksum does not match for file 'templates/statistics/save_as_new_report/get_report_dat.cfv'. Your licensing does not permit editing of template files, but it looks like that file has been edited. Please upgrade to Enterprise licensing if you need to edit templates, or remove the edits from that file to continue with your current licensing." [662959] Updating a database when the database has not yet been build gives an error, "Can't find tableAlias=main_table, fieldName=bottomleveldate in table main_table_join_sessions_join". [665590] Use of an ODBC log source on a Microsoft SQL Server with a BIGINT field, results in an error, "Invalid log source Unknown ODBC type; 5"
[665606] Cross-reference builds done with the internal database, with large datasets, sometimes give an error like, "Internal: list 99 in IntegerLists .... ends with -3412465, which is the start of a range" [671050] In some circumstances, dates which have data are not clickable in the Date Picker. [671133] If a scheduled task is running when a second scheduled task is scheduled to run, the second task does not start, in some cases.
[671684] A session contains filter, using wildcards, on a page which does not exist, gives an error, "Internal Error: Empty node name." [671949] The global_date_filename_regular_expression has no effect in multiprocessor database builds. [679482] Fixed a bug which could cause an error on database build or update, similar to "#### Internal: listitem=338931424, which is larger than the array cache size (308152)."
[680155] Database builds may crash at the end of log reading, if DNS lookup is turned on, and parsing servers are being used for multiprocessor building. [680181] Numeric comparisons in report filters give incorrect results. [680671] Running a multiprocessor "split" build (using Sawmill to split data into multiple per-profile log datasets) can result in an error "mismatched brackets" in the info.cfg file. [681790] Removing data from a database without session information gives an error, "Can't delete file LogAnalysisInfo/Databases//main/session_table_info.cfg". [857851] Dates are incorrectly offset multiple times in the database, when using a MSSQL database and a profile which uses accept/collect to parse log data, and a non-zero date offset value.
New features in 8.1.1:
[606985] Added a plug-in to report on OpenFire IM logs. These are XML logs and the lines can be extremely long. Testing was done by introducing newlines between closing and opening packet tags. [661926] Added new support for AIX CPU Utilization log format. [667394] Added support for Communigate Pro 5.2 to the Communigate Pro Log Format plug-in. Improved tracking of multiple recipients. Eliminated counting a message twice when ACCOUNT/delivered and DEQUEUER/LOCAL/delivered are both in the log. Added basic support for DEQUEUER/LIST/relayed. Eliminated underused operation field and added an action field just for delivery type. Added Queue ID field.
Version 8.1.0, shipped September 8, 2009
Bugs fixed in version 8.1.0:
[567010] Switching between "single color" and "numerical field color" in Config/Report Options/Graph Color Schemes sometimes has no immediate effect in the report output; if there is a cached report with the old setting, it is still used. [591276] Many of the Database Tuning options have no effect, and should be removed from the web UI. [597332] Reports show no rows when using a comparison filter (e.g., >=, <=, <, >) on a field which is the same as the main field of the table, but when the table is not showing bottom-level items. In particular, this happens when using a date range filter on the Years or Months report.
[603582] A version 7 profile put in the version 8 profiles directory (instead of being imported properly) gives a cryptic error: 'Unknown configuration group "server" in node "profiles.my_v7_profile_1.database.options"' instead of a better error like "you can't put a v7 profile in your v8 profiles directory." [605703] Zyxel Firewall Welf profiles are missing some of their Security reports. [613325] Parsing regular expressions extract log fields only as far as the first non-matching ()? section, if one or more of the ()? sections does not match. [613884] Running "remove database data" for an internal-database profile, from the Scheduler, gives an error, "Error parsing SSQL query; expected ')'; found ''.
[614397] Shoutcast W3C profiles can give an error: Unknown configuration group "session_event_type" in node "profiles..log.fields"
[617076] If the Log Detail report has been customized to include a non-timestamp date/time field, it will generate an error when exported to CSV, e.g. "Attempt to read beyond end of LogAnalysisInfo\Databases\\main\items\date_time\offsets_by_num
(fileSize=4000); attempted to read from 669806112 to 669806116."
[619197] The Overview report shows incorrect numbers (approximately double the correct values), when filtered with a NOT filter on a single value. [619738] Internal database index builds are extremely slow for certain very large datasets. [621923] Exported Log Detail contains one less row than the number shown in the UI table. [622939] Documentation search gives an error, 'Unknown configuration group "options" in node "database"', in installations which have been upgraded from pre-8.0.9 installations.
[623023] The encrypted source compilation fails with an error about unknown symbols, when ODBC is enabled, and ODBC driver manager headers and libraries are installed. [623070] Sawmill 7 profiles with session information give the error, "Unknown database field "ssession_entrances" in cross-reference group," on database build or report generation, when converted to Sawmill 8 profiles. [623118] Unique numerical fields (like "visitors") show 0 or a sum in the Total row by default, instead of the intended "-". [629646] In CGI mode on Windows, when using a temporary directory/URL, CGI export fails with an error: Unknown fileref file 'csv_export\username\profile\reportname.csv'
[630805] Fixed a bug which could cause database corruption when using a database created and updated with Sawmill 8.0.8 or earlier, and updating it with Sawmill 8.0.9. Possible error messages include, "Attempt to read beyond end of LogAnalysisInfo\Databases\\main\Tables\xref\sets\\lists.dat (fileSize=); attempted to read from to " and "Unable to allocate 4294967292 bytes of memory ()" (or similar large number).
[633524] Cisco Netflow profiles give an error, "Can't find tableAlias=, fieldName=start_time" when the database is built. [635081] SFTP support is not enabled in the binary distribution for x86 Linux (ES/AS 5). [636792] The field_length parameter to database fields, which specifies the maximum length of the database field value in characters, does not appear in the web interface. [637463] If a database update finds no new lines in the log data, future updates will not update their cross-reference tables correctly. This causes the newest data to be missing from the date range, Calendar, Database Info, unfiltered Overview, and other unfiltered top-level reports. [638943] Creating a profile using the border_manager or iisweb_with_syslog plug-in gives an error, "Unknown variable 'util.parse_w3c' in expression". [649962] If the profile name is too long when using an Oracle database, the database build fails with an error about the length of an index name. [649978] Regular expression report filters on an Oracle profile give an error about "regexp binary." [650092] Oracle Database builds use hundreds of cursors, and do not clean them up properly, which can cause the build to fail with a "too many cursors" error. [656549] Columns do not line up properly in emailed table-with-subtable reports--numerical columns are one column to the left of their header. [660518] Removing database data, then viewing a report which requires an index, sometimes gives an error like, "Internal: found leftrownum 1303 in index LogAnalysisInfo/Databases/ae/main/Tables/main_table/indices/mt_hit_type, but the table only has 1301 rows" [663015] A "remove database data" operation on an internal database does not properly track what data is new when updating xref tables, resulting in some data being missing from top-level reports.
New features in 8.1.0:
[580203] Added an option to duplicate a profile from the web interface. [580204] Added an option to rename a profile from the web interface. [604421] Enhanced documentation search to show a match when any word in the phrase matches, instead of requiring the whole phrase to match. [610256] Added a new custom action, convert_version_7_user, to convert users from a Sawmill 7 installation using the command line. [637041] Added support for HTTPS in Sawmill's built-in web server. [637044] Added a network API, for running custom actions by accessing the Sawmill server using a specially formatted URL. [637549] Added new support, or enhanced existing support, for the following log
formats: AIX messages, Citrix NetScaler, Clearswift MIMEsweeper, Juniper
Netscreen, Kingdon Firewall, Marshal8e6 Content Appliance, Open WebMail,
Watchguard Firebox XTM.
[643222] Added a plug-in to support the Pure FTP (Syslog) Log Format. The syslog format is the typical Unix daemon format and is different enough from the standalone version of the format that a separate plug-in is required. [655308] Added support for two types of events from Citrix Netscaler 9.0 to the Citrix Netscaler Log Format plug-in. Also increased the number of types of 8.x events that are supported.
Version 8.0.9, shipped June 23, 2009
Bugs fixed in version 8.0.9:
[532124] Filtered reports show higher numbers than unfiltered reports, when using "is NOT" filters. [532124] The description of report filters sometimes contains a "not" even when it isn't a "not" filter, if there is also another filter which is a "not" filter. [545749] Indices exceeding 4GB are truncated on 32-bit platforms, resulting in error like "Internal: itemnum=7081689, which is larger than the array cache size (7081664)" when generating reports from large datasets.
[554819] Multiprocessor database builds create zombie processes on UNIX systems, which last for the duration of the build; on systems with very few processes available, this can cause errors during build, including, "Error spawning process for multi-threaded SSQL query split." [584458] The sort_by (-sb) and sort_direction (-sd) command line options for "-a ect" are not documented. [586149] Profiles with duplicate labels are allowed by the Create Profile Wizard [590678] Log detail report is very slow, when using with a large internal database and no filters. [591432] The update.pl script in the Extras folder does not copy the system.cfg file, so installations updated with that script rerun the installation process. [591629] Scheduled tasks with a single profile, and multiple actions, run all actions simultaneously, instead of running them in sequence. [592187] Database indices, and unique-tracking lists, can become corrupted on 32-bit systems, when processing large datasets (typically, any dataset which would require an index > 2GB, though it could happen as low as 600MB). This can give various errors about "array caches" or "ranges." [592295] When using file-by-file distributed parsing, and "skip previously seen files on update," the list of previously-seen files is not recorded properly, resulting in some files being re-processed on update. [595619] Imported v7 profiles show integers in the sum row of "start time" or "end time" columns, instead of showing "-". [596194] When filtered, the Sessions Overview sometimes shows a negative number for "sessions for one-time users", and shows a few other incorrect values. [596597] Database fields with large aggregated values (more than about 10 billion) incorrectly show very small values (< 10) in the Overview. [596753] The current_log_pathname() function returns the wrong pathname for some log lines, when using multiprocessor database builds. [597095] Removing database data with a date range in a MS SQL database gives an error, "Invalid column name."
[597455] Database indices can become corrupt on update, when using the internal database. This can cause errors when displaying filtered reports, or errors during future database updates. Possible error messages include "Unable to allocate N bytes of memory (Reallocating array cache)", "Internal: list N in IntegerLists X ends with -M, which is the start of a range", and possible others. [597949] In real-time profiles, reports hang at 33% during the initial database build, if the build is using parsing servers (multiple processors), or if the file-by-file option is turned on. [597953] Date filter expressions containing capital letters give an error, "Date filter not valid." [598038] Log format plug-ins which use "collected listed", where field values are quoted (e.g., Fortigate), ignore the values immediately after quoted fields. [601338] Cross-reference tables of imported Sawmill 7 profiles do not contain numerical session fields, resulting in slow report generation. [601365] Regular expression or wildcard filters on the "hour of day" or "day of week" field are displayed incorrectly in the yellow filter description at the top of reports, as "corrupt date/time". [601810] Database updates applied to databases imported from Sawmill 7, or to databases using non-internal database servers, do not propagate the new data to the xref tables in some cases, resulting in reports which do not have the latest data.
New features in 8.0.9:
[569464] Added support for a new format variant of the NcFTP Xfer log. For now, the new fields at the end of each line are being ignored. [584392] Added support for overriding the port of SFTP, by using "hostname:port" in the hostname field.
Version 8.0.8, shipped May 20, 2009
Bugs fixed in version 8.0.8:
[527528] The "Lookup pages" link in "Paths through a page" gives an error, "Syntax error: Unknown variable 'xyz' in expression," with certain datasets. [538929] The return address option for Action emails in the profile is not editable from the web interface. [543954] Database fields with identical labels (but different internal names) cannot be distinguished in the web UI. [548862] Building a database from Quicktime Streaming Server log data which has no cs-uri-stem field gives an error, "Can't find tableAlias=main_table, fieldName=cs_uri_stem in table main_table". [549277] Information about the extra MaxMind databases (Organization, ISP, and Domain) is missing from the documentation. [550333] Salang truncates fractional floating point numbers to 6 decimal points, when converting them to strings, and also unnecessarily converts parameters to strings and back when calling subroutines. This could cause graphs percentages to incorrectly show 0%. [550413] PDF report generation from the Scheduler on Windows gives an error if the pathname contains "\r" [552621] CSV files sometimes parse incorrectly when using multiprocessor parsing--headers are not distributed properly to all processes, resulting in some log lines being ignored. [553307] ODBC log sources do not support TINYINT fields. [554929] The Help link in reports replaces the current report with the documentation (instead of just opening a window with the documentation in it).
[555013] The Advanced licensing tier is still mentioned in the trial mode page, and the trial mode menu. [555297] Charset conversion does not occur on CSV export from the command line [555602] The "remove data" action does not complain when the -df option is used; it does not support the -df option, so it should throw an error message if -df is attempted. [558333] The Calendar report fails or crashes, if there is no data (no accepted entries) in the database. [560172] The Create Profile Wizard hangs at the database page (won't click past it), if the log format has only one numerical field. [560264] Report generation can hang when using the internal database, if a report contains more than one column displaying the same "unique" field. [560324] Verbose filter (-v f) output causes an error ("unknown value") when used with replace_last(). [560565] Database date range is reported incorrectly when all xref tables are disabled. [560818] Creating a profile using Cisco NetFlow (flow-export) format gives an error: Unknown configuration group "location" in node "profiles.*.log.fields" [560832] Emails sent by Sawmill from Windows systems have an incorrect SMTP Date header (off by one hour). [561105] The .= operator does not work on log fields. [563263] Several report filter examples involving date ranges, in the documentation, contain syntax or usage errors. [564442] Customizations to spiders.cfg are overwritten when upgrading [565897] The date_filter option "1day" selects the most recent day from now, rather than selecting the most recent day of the log data. [569311] Building a database from Firewall-1 NG (text export) Log Format gives an error, "Unknown variable 'elapsed' in expression," if the exported data does not contain an "elapsed" field. [569542] Import of Sawmill 7 database fails with the error "Syntax error in line ... of file ...; no '=' in itemnum record" when one of the items in the database contains a literal line break character. [569615] The CFG newsletter documentation refers to "1" when it should refer to "$1". [573522] The DSN parameter is not saved by the Config section of the profile. [576315] The options in "Admin > Preferences > Support & Action Email" show no "i" icon documentation. [580158] The report footer is not displayed in non-dynamic reports; headers and footers are not expanded in non-dynamic reports.
New features in 8.0.8:
[531179] Added tracking of the "action type" field in Barracuda Spyware Firewall / Web Filter Log Format. [532973] Created a plug-in to support the Palo Alto Networks Firewall Traffic Log Format. [550936] Improved the performance of database building for large Postfix datasets, and probably other large datasets, by defaulting to keep itemnums in memory, and to use rand_hash as the itemnums hashing function when none is specified. [554267] Added documentation of the custom actions (-a command line options) in LogAnalysisInfo/actions, so the -a section of the option documentation. [555124] Improved progress for index builds, to show the number of indices built, and the total number to build. [565553] Added support to the Firewall-1 (fw log -ftn export) Log Format plug-in for log variant with no date in the log file, but log files containing a date in this format _yyyy-mm-dd_, as in this example: o_2008-01-28_000000.log. [570348] Improved performance of filtered reports using indices on the main table (which is generally any filtered report other than those using date/time filtering). [574676] Improved performance of loading large normalization tables--this improves the performance of hierarchy table builds, and probably some reports also, especially when fields have a huge number of unique values. [574677] Improved progress display during build of hierarchy tables. [576873] Added a new custom action create_user (-a cu) for creating or updating users from the command line. [579290] Improved performance of database updates with external SQL database server, by enhancing itemnum upload to upload only new itemnums on database update, instead of re-uploading the whole list for every update.
Version 8.0.7, shipped April 17, 2009
Bugs fixed in version 8.0.7:
[520725] Common Access Log profiles with large datasets use a huge amount of memory on database builds. [528835] The parser adds "{default}" to the end of any log field value which ends in the hierarchy divider specified by its log field, even if the field is not hierarchical. I.e., {default} or "(default page)" sometimes appears unexpectedly in reports of non-hierarchical, non-page fields.
[529667] Some strings in the web interface are in English, even when non-English translations are used. [533349] Ironport C Series reports do not show recipients when message rewriting occurs in the log data. [534986] Session ID reports appear as integers in the Individual Sessions report, on profiles where the session ID is computed by log filters, instead of appearing as the computed value. [535446] The IPC folder of LogAnalysisInfo collects many files with names HTTPRequest*, which linger for hours; these could be cleaned up much sooner. [535576] Building a database for a MS SQL profile will give an error like, "drop the index 'main_table.mt_field_name', because it does not exist in the system catalog," if another profile, or a profile using different prefix/suffix, has used that same database to build a Sawmill database in the past. [536085] Firewall-1 NG (text export) Log Format format does not support hh:mm:ss format for the "elapsed" field. [536108] Database builds started from Config -> Database Info immediately display a "build completed" page, before the build is actually completed, when a profile is "real time." [536525] PDF report generation sometimes crashes on x64 Windows. [538883] Filters created in the Filters window using "is NOT item name" appear as "is item name" in the filter description of the report. [539107] During a multiprocessor database build, the IPC folder of LogAnalysisInfo collects several files with names like ParsingServerPort_*.done, which linger for days; these could be cleaned up much sooner.
[539683] PDF export fails from the scheduler when using a drive letter in the pathname, with an error like "Can't create directory c: (File exists)". [539696] Session duration shows 0 for profiles without a session ID field, and with "maximum session duration" set to 0. [539814] Individual Sessions report of imported Sawmill 7 profile gives error, "Unable to read file LogAnalysisInfo/Databases/mms_pub_imported/main/Tables/ssession_idsubitem/header.cfg". [540871] Deleting a database field does not delete it from all cross-references tables. [541655] Schedules configured to "update all profiles" run all updates simultaneously, rather than sequentially. [541975] The "Creating Many Profile in a Batch" topic is missing from the FAQ. [543436] Some command line operations give the rather cryptic error message "Couldn't find node licenses in" when there is no license installed. [543560] When using Firefox, page formatting is messed up if a report is generated, the server is stopped, the Admin page is accessed, the server is started, and the Admin page is accessed again. [543657] Profiles with a field named "level" give an error "invalid identifier" when used with Oracle. [543851] An internal string management issue could cause crashes during database builds or updates, when using a SQL database server. [544036] The "Session ID" profile option is not customizable in the web UI. [544295] Database updates can give an error, "Unable to allocate X bytes of memory (allocating preconversion buffer)" where X is a huge number. [545134] Running the Windows installer to upgrade overwrites the preferences.cfg and default_profile.cfg files from the previous installation. [546690] The label of the report element filter field has a typo: "epxression" [548666] Log data with pseudo-W3C headers starting with "# Date\tTime" (like some Exchange 2000 logs) does not parse, resulting in empty reports. [550414] Exporting PDF tables containing certain data can give an error, "Unknown HTML entity"
[553375] Improved performance of building indices, for the internal database, for large datasets. [553713] Memory usage can be very high on profiles using the internal database, with very complex fields (fields with many unique values).
New features in 8.0.7:
[538958] Added a -pp ("path page") command-line option for specifying the focus page for command-line export of the Paths Through A Page report. [539823] Added better detection of mobile web browsers. [541059] Enhanced Firewall-1 (fw log -ftn export) Log Format to handle logs with lines starting with dates; added some new fields. [545835] Added support for Linksys VPN Router log format [546006] Added the option to disable cross-reference tables individually, for better control over database build and report performance. [548920] Enhanced support for IPtraf logs, to support a variant with single-digit days. [550396] Improved performance of report generation in most cases. This is partly due to an increase in the default value of the "maximum paging caching buffer memory usage", and partly due to other optimizations. Profiles will benefit most when setting "maximum paging caching buffer memory usage" to the new default value of 64MB. In one example (a large pivot table), report generation speed increased from 13 minutes to 2 minutes; memory usage increased by less than 2x. [553685] Improved progress reporting for cross-reference builds and index builds, to show more granular progress while building each xref table or index (especially when using the internal database).
Version 8.0.6, shipped March 19, 2009
Bugs fixed in version 8.0.6:
[532899] Corrected typo that was breaking sessioning (sessions_id -> session_id) and restored session_id to database.fields. Renamed the field clips to successful_accesses. Removed c_rate from numeric fields because it is a value relative to 1 which means the standard rate. c_rate is now the Visitor Systems->Client Rate report instead of the Client Rate column. [533310] Date filter strings of the form last1d, last1m, last3m, etc (with single-letter units) generate an error, "The date filter is not in valid date filter syntax." [534897] The Paths Through A Page report uses much more memory than it should (several GB).
When using a prefix or suffix with MS SQL or Oracle or MySQL, the
expiration query fails with a "no such table" error.
The "file by file" option for log processing is not editable in the web UI.
Database build performance is slow in some cases due to splitting
processing across N+1 threads (when N is the number of processors or cores)
instead of the more efficient N.
Log parsing uses arbitrarily large amounts of memory when reading log data
from corrupt log data or log data with extremely long lines.
Command-line authentication login fails with error,
"No Permission
You don't have grants to view this page or profile. Please contact your system administrator for more details."
Profiles using filter_finalization give an error on database build,
"Internal: sourceFileNode=NULL during ConfigNode::ParseInfixStatements()"
Database build generates an error,
"Error in writing ODBC table main_table: ODBC error: rec1:
SQLstate: 22003; msg=[Microsoft][ODBC SQL Server Driver]Numeric value out of range;"
when importing data into
MS SQL which has negative integers in it.
Browsing to a UNC pathname like \\pub\pub\logs, on Windows, shows
nothing in the right panel of the File Browser.
Viewing reports sometimes gives an error,
"Attempt to get node number 0 from node v.temp_x_ticks."
Browse button gives the directory above the entered pathname,
if the pathname ends with a slash.
Global page headers and footers do not appear in reports.
The "Automatically update database when older than"
option is not saved properly in the UI.
Database builds could sometimes give an error, "Unable to read contents of
directory LogAnalysisInfo/TemporaryFiles/DeleteMe_...
(No such file or directory)"
Tasks scheduled to run at the same time run in sequence instead.
Temporary files are not cleaned up quickly enough, resulting in large numbers
of DeleteMe files in the TemporaryFiles directory.
Memory usage is very high when zooming to large reports.
Date/time graphs with one-minute granularity show no data.
"Default date filter" (global per-profile date range filter) has no effect.
Imported Sawmill 7 profiles sometimes give error,
"Couldn't find node 0 in language.english.lang_stats.weekdays_short."
Removing all cross-reference groups gives an error,
"Couldn't find node xrefs in v.fp."
The Paths Through A Page report takes a very long time, and
a huge amount of disk space.
The date filter expression "last1month" (and similar ones)
give an error about incorrect date range format.
A database removal operation on a MS SQL database gives an error,
"Unable to Execute ODBC Query='delete from main_table x using main_table x where 1=1'; diagnostics=ODBC error: rec1: SQLstate: 37000; msg=[Microsoft][ODBC SQL Server Driver][SQL Server]Line 1: Incorrect syntax near 'x'.;"
Months report of imported Sawmill 7 profile, where the date/time report
had been edited in Sawmill 7, gives an error,
"Unable to read file LogAnalysisInfo\Databases\\main\Tables\rows_...o\header.cfg (No such file or directory)"
Microsoft Media Server plug-in gives wrong results for the c-rate field.
Generating a report with a customer date range filter string gives an error
"Couldn't find node in."
Reports give an error, "Couldn't find node modification_time in v.lang_stats_file_info," when English language is not installed.
Reports show wrong numbers for fields containing single field values
greater than 2 billion, when using multiprocessor database builds
on a 32-bit system.
The "Process subdirectories" checkbox erroneously states that it is for
"local folders only."
ODBC log sources do not support LONGVARCHAR (a.k.a. TEXT) or BIGINT fields.
Microsoft Media Server profiles show 0 for session duration.
W3C log data (like IIS log data) is very slow to parse,
when using multiprocessor database building.
Date filter strings of the format "last1d", "last1m", and others using
single-letter units, give an error, "The date filter is invalid."
The Profile menu in an action in the Scheduler is not alphabetized.
The "automatically update when older than" option sometimes has no effect.
The "configure" script does not complain when there is no C++ compiler,
when building Sawmill from encrypted source, which causes the compilation
to fail later when it attempts to compile the C++ files.
New features in 8.0.6:
[486806] Added new plug-in to support the Palo Alto Networks Firewall Traffic log format.
A new option has been added for doing LOAD DATA imports into local MySQL
database servers, which is faster than the default LOAD DATA LOCAL INFILE,
and works on servers where local infile is disabled, but requires the MySQL
server to be on the Sawmill server.
A new variant of Unix Daemon Messages log format is now supported.
The name of the Sawmill executable has been changed to just "sawmill"
on non-Windows platforms, instead of having the version number in the name.
Sawmill now determines the amount of physical RAM on the system,
and when splitting queries across multiple threads with "auto,"
it ensures that each thread has at least 2GB of RAM.
A new option, "maximum read block size," controls how much memory
can be allocated to holding a single line of log data.
Version 8.0.5, shipped February 26, 2009
Bugs fixed in version 8.0.5:
Fixed a bug which could cause an error when searching the documentation
in Windows.
Fixed a bug which could cause an error on builds with MS SQL databases, if there was
a database field called "rule."
Fixed a bug which could cause database corruption if the main table of the database
was more than 4GB, with 32-bit versions of Sawmill. This generally happened
with datasets more than about 30 million lines. It could cause a variety of symptoms,
including out-of-memory errors while building indices, crashes during reporting,
and incorrect numbers in the reports.
Fixed a bug which could cause an error "Parsing server returned itemnums list for unknown database field 'fieldname'"
when building two databases simultaneously on Windows.
Fixed a bug which could cause an error "Unable to read file ... sessions_join"
when no valid log entries were found in the dataset.
Moved the MySQL DLL to the Sawmill installation directory, to reduce conflicts with PHP and other programs
which install different versions of the MySQL DLL.
Added support for MySQL on the x86 Linux ES4 platform, where is was disabled.
Restored the "create many profiles" script (now in LogAnalysisInfo/util), which was missing earlier
versions of Sawmill 8.
Fixed a bug where multiprocessor log processing would fail with a parsing server error,
if the "Server hostname" was not empty in Preferences.
Fixed a bug where lines of log data could be split incorrectly,
for log formats where quoted values can span multiple lines,
if the lines spanned the boundary between log reading blocks.
This could result in slightly low numbers.
Fixed a bug in Shoutcast 1.8 processing, which could cause a regular expression error on parsing.
Fixed a bug which could cause incorrect sorting of rows in tables on 32-bit systems,
if they contained large floating-point numbers (more than about 4 billion).
Fixed a bug which caused the graph tick marks and labels to be in the wrong locations, when rendered by Firefox.
Fixed a bug which could cause an error if all log filters were deleted.
Fixed a bug where the Create Profile Wizard incorrectly validated the pathname
when regular expressions were used, causing a "no such pathname" error in some cases
where the pathname was correct.
Added the "maximum paging caching buffer memory usage" option to the web UI.
Removed some database optimization options which no longer have any effect in Sawmill 8.
New features in 8.0.5:
Enhanced support for Smarter Mail log format to handle dates containing dots.
Enhanced support for Wowza log format to handle large negative x-duration values (due to logging or server bugs).
Enhanced support for Limelight log format, to handle spaces in #Fields lines, in addition to tabs.
Added the "integer bits" option for database fields to the web GUI.
Improved handling of v7-to-v8 database conversion in the case where the database directory
was overridden--it now prompts for a new directory for the Sawmill 8 database directory, so it won't
overwrite the Sawmill 7 one.
Version 8.0.4, shipped February 13, 2009
Bugs fixed in version 8.0.4:
Fixed a bug which could cause progress reporting to halt partway through long database builds,
on 32-bit Windows.
Fixed a bug in the Intermapper Event Log Format plug-in that caused an error if Events was not selected as a numeric field during profile creation.
Fixed a bug where reporting would hang in CGI mode, if a "temporary directory" was set.
Added "always include bottom-level items" to the database fields editor.
Fixed a bug which could cause an error when processing Blue Coat W3C logs, if the log data contained
both a cs-uri-path and a cs-uri-stem field.
Fixed an error, "Attempt to join main_table to filtertmp_5652_1 on 's.session_id = filtertmp_5652_1.itemnum', but there must be one column from each of the two tables in the ON condition, and there doesn't seem to be", which could occur when zooming in on a particular session in DIndividual Sessions.
Fixed a bug which would cause an error if a template was edited in an Advanced installation,
though template editing is permitted by Advanced licensing.
Fixed a bug which could cause an overflow when reporting processing time in ISA 2004 log data.
Fixed/improved cleanup of temporary files in LogAnalysisInfo.
Fixed a bug where PDF generation would fail if the report contained a pie chart with a legend.
Fixed a bug where "-a rdi" (rebuilding indices from the command line) gave an error when used with the internal database.
Corrected references to SawmillCL.exe, which is now called Sawmill.exe (in version 8).
Fixed a bug where the Cities report showed States instead, when reporting on W3C log data (like QTSS).
Fixed a bug where the Logout link did not work on the first click, in Firefox.
Fixed a bug where the Support Email Address in Config -> Miscellaneous -> Support & Action Email, did not save.
Fixed a bug where the Config -> Reports page would not display when logged in as a Manager.
Fixed a bug which could cause crashes when using the page tagging server on Windows.
Fixed a bug which could cause 0s in reports, when clicking single date item while displaying a
date report at a different level (e.g., clicking a month, while displaying the Days report).
Fixed a bug which could cause an error when there was a database field named "count".
Fixed a bug where long values could overlap the columns to the right of them, in HTML tables.
Fixed a bug where database updates reported themselves as "Rebuild Database" in the Database Info page.
Fixed a bug where the command-line progress display did not show the number of lines processed.
Fixed a bug where the Sessions Overview report did not change when filters were applied.
Fixed a bug which could cause some log data not to parse, if the "sessions visitor ID" field was empty.
Added the End User License Agreement file to the distribution directory (it was in the installer, but not the installation).
Fixed a bug which could cause an overflow when displaying time-taken for Blue Coat W3C logs,
on a 32-bit system.
Fixed a bug which would cause an error when filtering a report, if there was a SQL table prefix or suffix specified
in the profile.
Fixed a bug which would cause 0 reports when using a regular expression filter on a non-itemnum field (like hour of day).
Fixed a bug which could cause an error, "Unknown variable 'lang_stats.field_labels.time_stamp' in expression",
when reporting on Common Access log data.
Fixed a problem with the formatting of FAQ pages.
Fixed a bug in iMail parsing which caused an error:
'Unknown configuration group "from" in node "profiles.qmail.log.fields".'
New features in 8.0.4:
Modified the Intermapper Event Log Format plug-in to find the year in the log file name it contains the pattern LogYYYYMMDD.
Added an option to RBAC to disable password changing.
Extended Sidewinder Firewall reporting to show countries, regions, and cities for each IP.
Added suport for Squid format logging to Unix Syslog, with a Squid timestamp.
Added support for a new variant of Mirapoint SMTP log format.
Improved the performance of some internal database operations used for indices and uniques tracking.
Improved the performance of some internal database queries involving the sessions table.
Version 8.0.3, shipped January 23, 2009
Bugs fixed in version 8.0.3:
Fixed a bug which could cause database builds to terminate unexpectedly. This occurred with ASA log data, but
could theoretically happen with any log data.
Fixed an issue with the copyright in the Czech translation.
Fixed/enhanced the TaskLog file to suppress the execute_sql_query, which were numerous and mostly useless.
Added documentation for the "Remove Reloads From Sessions" option.
Corrected the MacOS Install.txt file to remove incorrect upgrade instructions.
Fixed a bug which would cause an error when zooming on a session user, and zooming to the Session Pages report.
Fixed a bug which could cause an error, "Error in get_progress_state.cfv" when displaying the Overview.
This bug was caused by a failure during the database build, so profiles showing this behavior will need
to have their databases rebuilt.
Fixed a bug which could cause database builds to hang, when built from Config -> Database Info,
if an error occurred while the database was being deleted.
Fixed a bug with the v7-to-v8 profile converter, which would cause errors when viewing reports,
if the v7 profile contained references to database fields which were not translated in Sawmill 8.
Fixed a bug which could cause an error during database build, if the database directory
was on a different drive or partition from the LogAnalysisInfo (installation) folder.
Fixed a bug where the current_log_pathname() function did not work when using parsing servers
(multithreaded database builds).
Fixed a bug which would cause an error, "Syntax error: Expected ) in expression; found '" when
creating a profile from BlueCoat W3C data.
Fixed a bug which could cause an error when relocating the database directory.
Fixed a bug which could cause sporadic (though harmless) crashes of the web server.
Fixed a bug which could cause an error, "Unknown configuration group 'cs_uri_stem_bottom_level_items'" when viewing
reports for a Microsoft Media Server profile.
New features in 8.0.3:
Improved the Windows installer to omit the minor "rcN" from the version number,
to simplify the Sawmill version number display, and make it clear that these
is a production releases (successful release candidates), not pre-production releases (unsuccessful release candidates).
Improved documentation of "session events" and "sessions."
Added section to documentation, "Adjusting date/time values for Daylight Savings Time".
Enhanced the Limelight plug-in to handle arbitrary W3C headers, allowing variable field layout.
Enhanced the Squid plug-in to handle "action" field values containing spaces (like "TCP MISS").
Version 8.0.2, shipped January 16, 2009
Bugs fixed in version 8.0.2:
Fixed a bug where the "automatically update if older than" option was not saved in Config.
Fixed a bug which caused an error when creating an ISA profile: 'Unknown configuration group "filterinfo" in node "profiles.isa_logs.log.fields".'
Fixed a bug where some hierarchical reports would show no results when drilled into.
Fixed a bug where direct login by URL would sometimes give an error.
Fixed a bug which would give the error 'column "" does not exist' when analyzing Bluecoat log data.
Fixed a bug which could cause an error when using a literal backslash in a custom report expression.
Fixed a bug which could cause an error while analyzing Nortel ACD log data:
'Unknown configuration group "avg_agent_time_busy" in node "profiles.test.log.fields.'
Fixed a bug which could cause an error when clicking Scheduler after importing v7 profiles
with field names containing numbers.
Fixed a bug where full-month filters on the Overview gave zero results.
Fixed errors compiling the encrypted source on SuSe 11, and other recent operating systems.
Fixed a bug where the progress display for one profile could show in another profile's Reports display,
if there were two simultaneous profiles in use.
Fixed a bug where x86 Windows was reported as x64, and vice versa, in automated bug reports.
Fixed several places where images were broken in CGI mode.
Fixed a bug where very long field values could generate an error, when using ODBC.
Fixed a bug which could cause an error when analyzing Sidewinder log data with Microsoft SQL Server.
Fixed a bug where Log Detail report was empty when using ODBC databases.
Added support for not splitting queries at all, in the web interface.
Fixed a bug where some URLs used HTTP, even when running Sawmill in CGI mode under an HTTPS server.
Fixed a bug where parsing servers could linger after the process that spawned them was gone,
in the event of a parsing error.
Fixed a bug which caused an error, 'Internal Error: Empty node name,' when setting a session field to "(None)".
Fixed a bug which caused an error when generating the Top Malware report in Ironport S-Series logs,
'Couldn't find node display_format_type in sessions_cache.xyz.profiles.profilename.extended_profile_dat.'
Fixed a bug which could cause an error like, 'Unable to read file LogAnalysisInfo\Databases\profile\main\Tables\rows_1_6248o\header.cfg'
on location reports, in imported v7 profiles.
Fixed a bug which could cause sporadic crashes of the web server process.
Added the update_xrefs_on_update option to the web interface.
Fixed a bug which caused the Help link to sometimes have no effect.
Fixed a bug where pivot tables would sometimes show NULL rows.
Fixed a bug which could cause errors when using a SQL prefix or suffix.
Fixed a bug which could cause an error, "Unknown variable 'lang_stats.field_labels.ms_ras_client_name' in expression."
Fixed a bug which could cause an error when analyzing Sidewinder logs with MS SQL.
Fixed a bug where Config/Reports did not prompt to save changes.
Converted some hard-coded strings to language module strings, for internationalization.
Fixed a bug which would cause the error, "Couldn't find node next_pages in v.query_result.data" when
viewing a Session Paths report with no data.
Fixed a bug which prevented error messages from being viewed when not logged in.
Fixed several broken images in the documentation, in CGI mode.
Fixed a bug where non-root-admin users could not see the version number of Sawmill in the web UI.
Fixed a bug which could cause the error, "Unable to read...\main\Tables\sessions_join\header.cfg (Day of week)."
Added the missing language module variable lang_admin.log_filters.simplify_playerid_label.
Fixed a bug which would cause error when rebuilding a database, if it was stored a different
drive or partition from the LogAnalysisInfo folder.
Fixed a bug which would cause database build errors when the log data contained extremely large or infinite numbers.
Restored the missing FAQ about emailed report and Outlook 2003.
Fixed the FAQ about resetting the admin password to describe the new method.
Fixed a bug where the SPARC Solaris 9 distribution did not have the necessary libraries included.
Fixed a bug where Config -> Log Source did not support multiple log sources in Professional tier.
Fixed a bug where once a trial license expired, it would not accept another license key.
Restore the Support page from Sawmill 7, to Sawmill 8.
Restored the HTML comment describing the filters, from Sawmill 7, to Sawmill 8.
New features in 8.0.2:
Updated the Czech, German, and Polish translations.
Enhanced Radware DefensePro plug-in to handle ip:port variation.
Enhanced Flash Media Server plug-in to support analysis when there is no cs_uri_stem field.
Implemented a custom action, reset_root_admin, for resetting the root administrator password from the command line.
Added more documentation about real-time log importing.
Version 8.0.1, shipped December 24, 2008
Bugs fixed in version 8.0.1:
Fixed a bug where once a trial license expired, Sawmill would not accept a new license.
Fixed a bug where a Sawmill 7 license would be called "invalid" instead of being reported as a valid older license, no longer valid for Sawmill 8.
Fixed the upgrade instructions in the README of some platforms, which were still describing the Sawmill 7 upgrade method.
Fixed a bug where Sawmill 8 would not install its service, if Sawmill 7 was already installed, and would uninstall the Sawmil 7 service when Sawmill 8 was uninstalled.
Fixed a bug in the Create Profile wizard, which would cause an error if the Pathname was not literally a valid existing pathname,
even if it contained wildcards which should have matched valid pathnames. This almost always caused wildcard or regular expression log sources to fail on profile creation.
Fixed a bug where wildcard and regular expression report filters did not work (generated an error when attempted) with Microsoft SQL Server profiles.
New features in 8.0.1:
Added a separate chapter to the documentation about real-time importing.
Added a Support link to the Admin page.
Added the Salang expression of the current filter to the HTML of the report, as an HTML comment (useful for creating command-line or scheduled filters, or for debugging).
Chopped off the "rcN" part of the version number in the web interface, to make it look better.
Switched the default port of the web server to 8988. This makes it much simpler to run Sawmill 7 and Sawmill 8 together on the same system.
Enhanced support for ISA W3C format, to handle a significant variant (2007)
Added support for Unicode with Microsoft SQL Server as back-end database, so non-ASCII log data can be imported
and stored in MS SQL, and queried from outside Sawmill.
Version 8.0.0, shipped December 12, 2008
Bugs fixed in version 8.0.0:
New features in 8.0.0:
Changed the GUI concept from html frames to single pages.
Added report fields for more flexibility and fine tuning of report elements and table data.
Added a simplified date_filter syntax (i.e: 2m, last2m, etc.) for date/time filtering.
Added a new caching system which caches various report components and database data independently.
Added RBAC (Role Based Access Control)
Added support for sequential actions per schedule in scheduler.
Added a "Run Now" button in the Scheduler, to run any task immediately.
Added log fields editor
Added database fields editor
Added session fields editor
Added report fields editor
Added new field wizard (which allows to create a log field, database field and report field at once)
Added URL support to view reports by URL definition by defining: profile name, report name, date filter, filter expression, filter comment
Added support to view any hierarchical depth of a hierarchical database field as non-hierarchical report. This allows i.e.: to view a months report or a region or city report. (These reports have been added by default)
Added the calendar as optional report.
Added support to dynamically create a pivot table within the reports GUI
Added support for different sort field and sort direction of the drill down field.
Added support to drill down data on a table with multiple string fields.
Added a new date picker which combines single date, date range and relative date selection.
Changed the zoom concept in that zoom automatically adds the zoomed item to filters.
Added support to zoom to multiple items at once.
Added support to save filter items as filter group
Improved the filters editor.
Added support to email a report within the reports GUI
Added support to define the row numbers and aggregation rows when exporting a table within the reports GUI
Added min and max aggregation rows to tables.
Added a row_visibility_expression per report element. This expression allows to show/hide table rows by an expression, i.e. show only rows where page_views > 300 and page_views < 1500.
Added support for a default date filter per profile.
Added support for a date filter per report or per report element.
Improved the Customize Report Element form/options.
Added table column info support.
Added table row selection support (to mark a row in yellow color).
Added support for 3D pie charts
Added support for antialiased PNG graphs
Added support for use of MS SQL or Oracle databases as back-end databases (where processed log data is stored, and reports generated from).
Added support for use of MS SQL, Oracle, or MySQL database as log sources
Implemented multiple scalability improvements, to allow Sawmill to process log data with less memory usage. These largely lift the restrictions of log processing on 32-bit systems, so any amount of log data can be processed without exceeding the address space of a 32-bit system.
Added real-time reporting. Reports can be generated while data is being imported, and will be up-to-the-moment, based on the latest imported data. Data can be streamed continually into the database, without any reporting downtime.
Implemented major SQL performance improvements, especially in the building of cross-reference tables, and in the performance of database update.
Enhanced the internal database to support SQL syntax, for universal SQL querying of any Sawmill database.
Added support for reading log data from a SFTP server
Added directory recursion on (S)FTP servers, so a log source can point to a directory, and all subdirectories will be processed.
Added filtering of reports on numerical fields; e.g., show all events where bytes > 1000
Implemented multiprocessor splitting of report queries: report calculations are split across multiple processors for better performance.
Enhanced multiprocessor and multi-system log processing. Log process is now done more efficiently on a single system, and does not use the disk as much; and it is also possible to split log processing across multiple "parsing servers" on the same network.
Added database import and export.
Added support for user-created actions (-a options), with fully customizable parameters and behavior.
Version 8.0.0b8, shipped December 06, 2008
Bugs fixed in version 8.0.0b8:
Fixed a bug which would cause an error when creating a BlueCoat W3C profile.
Fixed a bug where "start time" and "end time" were integers in imported MySQL profiles, rather than bring formatted timestamps.
Fixed a bug where W3C log data (and other log data with header lines) would not be imported properly
(some lines would be dropped) when using multiprocessor log parsing.
Fixed a bug where the "index" attribute of database fields (whether to index the database field) was not editable in the web interface.
Fixed a bug where the database name was required in Config -> Database -> Server, when it should have been optional.
Fixed a bug where the web interface did not have an option to *not* use distributed parsing.
Changed xref tables to be hierarchical by default, which results in faster report generation, especially for large datasets,
but somewhat slower xref builds on import.
Improved performance of the "session pages" report, and other reports which use indexed joins internally.
Fixed a bug where the "use overview for totals" option had no effect.
Fixed a bug where the "use overview for totals" option was not editable from the web interface.
New features in 8.0.0b8:
Version 8.0.0b7, shipped November 30, 2008
Bugs fixed in version 8.0.0b7:
Fixed a bug where the Reset/Collapse All button did not display properly in the Session Paths report.
Added the "User Overview For Totals" option to the Config interface.
Added editing of the Index option to the Database Fields editor (to specify which database fields should be indexed).
Fixed multiple bugs in reporting, which caused error when generating reports with a SQL database table prefix or suffix.
Fixed a bug where Windows pathnames were shown with slashes doubled, in error messages.
Fixed a bug where entering a directory pathname ending with a slash, then clicking Browse, would browse the parent of the directory, rather than the directory itself.
Fixed a bug where the Report Menu would not be displayed, when viewing a report with no elements.
Fixed a bug with Internet Explorer 7, where the Date Picker did not display properly.
Fixed a bug which caused an error when viewing reports for profiles with a database fields called "from" (or other SQL keywords), e.g., Postfix.
Improved performance of Squid log processing.
Fixed a bug where the expiration date always appeared as Feb 4, 2008, in the Licensing page.
Added links to documentation examples, from the Report Filters editor.
Fixed a bug where Config -> Database -> Server required a database name to be entered, even though it should default to the profile name when empty.
Fixed a bug where the Date picker showed a red "undefined" as the first day of the week.
Fixed a bug where the ODBC form required a username to be entered, even of the DSN had one embedded.
Removed "log processing threads" from the web interface; it is deprecated.
Fixed a bug where some windows did not have title bars, in CGI mode.
Fixed a bug where CGI mode would fail when running queries, if they were large enough to be split across multiple processors.
New features in 8.0.0b7:
Version 8.0.0b6, shipped November 29, 2008
Bugs fixed in version 8.0.0b6:
Fixed a bug where the Windows install included an unnecessary DLL in the installation directory.
Fixed a bug where the Calendar report was not enabled on imported v7 profiles.
Fixed a bug which could cause an "Unknown hash function" error when building some types of profiles,
including Microsoft Media Server.
Fixed a bug which could cause progress reporting errors, if there was error during a database build or report generation.
Fixed a bug which could cause a crash during autodetection, if there was an error in autodection (instead of reporting the error).
Fixed a bug which could cause random results in the Overview, if the filter set contained no rows, and the database
type was Microsoft SQL Server.
Fixed a bug where Microsoft SQL Server databases were not updated properly during a database update; in particular,
the itemnum (normalization) tables were not updated.
Fixed a bug which could cause an error when zooming on session fields, and zooming to session reports.
Fixed a bug where MySQL databases were not imported successfully from Sawmill 7 profiles.
Improved autodetection of Instagate log format, so it doesn't detect Apache Combined as Instagate.
New features in 8.0.0b6:
Version 8.0.0b5, shipped November 27, 2008
Bugs fixed in version 8.0.0b5:
Fixed a bug where relative date filters (e.g., "last 4 months") could cause an error.
Fixed a bug which could cause an error when displaying pivot tables with no data in them.
Fixed a bug which caused the Overview to contain random numbers, when there was no data in the filter set.
Fixed a bug which could cause incorrect numbers in the Sessions Overview, when there were no session events in the database.
Improved the performance of the Sessions Overview report.
Fixed a bug which could cause a very high "elapsed time" value (e.g., 39 years) to appear in the progress display at the beginning of some tasks.
Fixed a bug where MS SQL database updates did not automatically update the xref tables.
Fixed a bug where generating a PDF report could cause an error about "width:-8".
Fixed a bug where individual characters could be skipped during database updates.
New features in 8.0.0b5:
Version 8.0.0b4, shipped November 25, 2008
Bugs fixed in version 8.0.0b4:
Fixed problems with real-time profiles which prevented successive reports from showing the latest information.
Fixed several cosmetic issues with the bug report text.
Fixed a bug where FTP profiles could not process the log data, if there was no leading / in the pathname.
Fixed an issue which could cause errors during database builds, on some 32-bit systems (especially, Solaris).
Fixed a bug where the Use Sawmill icon was not properly removed on uninstall.
Fixed and improved the progress display for database builds.
Fixed a bug with v7-to-v8 profile conversion, which could cause an error when clicking Config -> Database.
Fixed a bug with the conversion of v7 MySQL databases to v8.
Fixed a bug where Sawmill did not see files in an FTP log source, in some circumstances (especially, when accessing a Microsoft FTP server).
Fixed a bug which could cause an error when viewing the Single-page summary with a session field filter.
New features in 8.0.0b4:
Version 8.0.0b3, shipped November 23, 2008
Bugs fixed in version 8.0.0b3:
Fixed an uninstaller bug where the Sawmill 8 icon remained on the desktop after uninstall.
Fixed/improved the progress display for database builds, so it includes all steps, and shows better descriptions of each step.
Fixed a bug with v7-to-v8 profile converter, which did not set up the database tuning options properly, resulting in an error
when viewing the Database section of Config.
Fixed conversion of v7 MySQL database in the import wizard.
Fixed a bug where Sawmill could not see files on a Microsoft FTP server.
Fixed a bug which could cause an error when zooming on session fields, and displaying reports generated without cross-reference tables.
Fixed a bug where indices were completely rebuilt after database updates; they are now properly incrementally updated
from the new new data.
Fixed a bug which could cause a "duplicate key" error when viewing reports from a MS SQL database.
Added support for input of "node" licenses through the web interface.
Fixed bug where the database build would fail if no entries were accepted.
Fixed error which could occur when building from Microsoft Media Server logs.
Fixed a bug where CSV export in CGI mode had a broken link for the CSV file.
Fixed a bug which could cause an error when building a database from Ironport S-Series logs.
Fixed a bug where references to non-existent template pages would give an error "no node 'templates' in 'templates'".
Fixed the timestamp of emailed reports sent from Windows.
New features in 8.0.0b3:
Version 8.0.0b2, shipped November 19, 2008
Bugs fixed in version 8.0.0b2:
Fixed bug which would cause a database build to abort with an error, if the -v f option was used, and a log field value contained a $.
Added ODBC driver manager libraries used by the x64 Linux (ES5) version, to eliminate a "libodbc.so not found"
error.
Fixed a bug where "start time" and "end time" were formatted and named incorrectly in imported MySQL profiles.
Fixed a bug where some images were broken in CGI mode.
Fixed a bug where command-line authentication did not restrict profiles or permissions properly.
Fixed a bug where a "garbage" line appeared at the top of printer friendly pages.
Fixed a bug where the Database Info page would generate an error if the database no longer existed.
Fixed a bug which would cause an lang_stats error when reporting on PIX logs.
Fixed a bug which would cause an error when using "log processing threads" > 0 in Advanced tier.
Fixed a bug which could cause an error with Log Detail, for certain datasets.
Fixed a bug where CGI mode did not display reports.
Fixed an installer issue where a necessary DLL (libeay32.dll) was not installed properly on 64-bit Windows.
Fixed a bug which would cause an error when filtering on items containing a backslash.
Fixed a formatting problem of the date in the licensing page.
Greatly improved performance of filtered reports which must query the main table.
Fixed a bug where the Cross Reference Groups editor showed "undefined" next to all numerical fields.
Fixed a bug where the Session Exits field did not have a default "Session field" value in the report fields editor.
Fixed a bug where the charset could not be changed in the Log Processing options.
Fixed a bug where password were not masked when entering SQL database information.
Fixed a bug where the Send Email window in the Scheduler incorrectly asked for an "output directory."
Fixed a bug where the Import Wizard did not report the error message, in the case of an import failure.
Fixed a bug where the text "Saving..." appeared below (or above) reports, and messed up the zoom formatting somewhat.
New features in 8.0.0b2:
Version 8.0.0b1, shipped November 1, 2008
Bugs fixed in version 8.0.0b1:
New features in 8.0.0b1:
Changed the GUI concept from html frames to single pages.
Added report fields for more flexibility and fine tuning of report elements and table data.
Added a simplified date_filter syntax (i.e: 2m, last2m, etc.) for date/time filtering.
Added a new caching system which caches various report components and database data independently.
Added RBAC (Role Based Access Control)
Added support for sequential actions per schedule in scheduler.
Added a "Run Now" button in the Scheduler, to run any task immediately.
Added log fields editor
Added database fields editor
Added session fields editor
Added report fields editor
Added new field wizard (which allows to create a log field, database field and report field at once)
Added URL support to view reports by URL definition by defining: profile name, report name, date filter, filter expression, filter comment
Added support to view any hierarchical depth of a hierarchical database field as non-hierarchical report. This allows i.e.: to view a months report or a region or city report. (These reports have been added by default)
Added the calendar as optional report.
Added support to dynamically create a pivot table within the reports GUI
Added support for different sort field and sort direction of the drill down field.
Added support to drill down data on a table with multiple string fields.
Added a new date picker which combines single date, date range and relative date selection.
Changed the zoom concept in that zoom automatically adds the zoomed item to filters.
Added support to zoom to multiple items at once.
Added support to save filter items as filter group
Improved the filters editor.
Added support to email a report within the reports GUI
Added support to define the row numbers and aggregation rows when exporting a table within the reports GUI
Added min and max aggregation rows to tables.
Added a row_visibility_expression per report element. This expression allows to show/hide table rows by an expression, i.e. show only rows where page_views > 300 and page_views < 1500.
Added support for a default date filter per profile.
Added support for a date filter per report or per report element.
Improved the Customize Report Element form/options.
Added table column info support.
Added table row selection support (to mark a row in yellow color).
Added support for 3D pie charts
Added support for antialiased PNG graphs
Added support for use of MS SQL or Oracle databases as back-end databases (where processed log data is stored, and reports generated from).
Added support for use of MS SQL, Oracle, or MySQL database as log sources
Implemented multiple scalability improvements, to allow Sawmill to process log data with less memory usage. These largely lift the restrictions of log processing on 32-bit systems, so any amount of log data can be processed without exceeding the address space of a 32-bit system.
Added real-time reporting. Reports can be generated while data is being imported, and will be up-to-the-moment, based on the latest imported data. Data can be streamed continually into the database, without any reporting downtime.
Implemented major SQL performance improvements, especially in the building of cross-reference tables, and in the performance of database update.
Enhanced the internal database to support SQL syntax, for universal SQL querying of any Sawmill database.
Added support for reading log data from a SFTP server
Added directory recursion on (S)FTP servers, so a log source can point to a directory, and all subdirectories will be processed.
Added filtering of reports on numerical fields; e.g., show all events where bytes > 1000
Implemented multiprocessor splitting of report queries: report calculations are split across multiple processors for better performance.
Enhanced multiprocessor and multi-system log processing. Log process is now done more efficiently on a single system, and does not use the disk as much; and it is also possible to split log processing across multiple "parsing servers" on the same network.
Added database import and export.
Added support for user-created actions (-a options), with fully customizable parameters and behavior.
|
 |
