You’re receiving this newsletter because during the
downloading or purchase of
Sawmill, you checked the box to join our mailing list. If you wish to
be removed from this list, please send an email, with the subject line
of “UNSUBSCRIBE” to newsletter@sawmill.net
(please include the entire message, as the identifying information is
at the bottom).
News
Sawmill 8.1.3 shipped on February 12, 2010. This is an bug-fix
release--it fixes a number of
bugs. This release is free to existing Sawmill 8 users. It is
recommended
for anyone who is experiencing problems with Sawmill 8.1.2 or earlier.
Sawmill 8.1.2 has a known issue with some Windows installations. It
requires the Microsoft Visual Studio 2008 redistributable package, but
does not install it. Most Windows systems have this package already,
but for those that don't, Sawmill will not run. If you are experiencing
this problem, upgrade to 8.1.3, or install the redistributable package (x86
or x64).
You can download Sawmill 8.1.3 from http://sawmill.net/download.html
.
Sawmill 7 users can upgrade to Sawmill 8 for half of the license price;
or if you have Premium Support, the upgrade is free. Major features of
Sawmill 8
include
support for Oracle and Microsoft SQL Server databases, real-time
reporting, a completely redesigned web interface, better
multi-processor and multi-core support, and role-based authentication
control.
This issue of the Sawmill Newsletter describes the process of
deleting database fields to simplify the database.
Get The Most Out Of Sawmill With Professional Services
Looking to get more out of your statistics from Sawmill? Running short
on time, but need the information now to make critical business
decisions? Our Professional Service Experts are available for just this
situation and many others. We will assist in the initial installation
of Sawmill using best practices; work with you to integrate and
configure Sawmill to generate reports in the shortest possible time. We
will tailor Sawmill to your environment, create a customized solution,
be sensitive to your requirements and stay focused on what your
business needs are. We will show you areas of Sawmill you may not even
be aware of, demonstrating these methods will provide you with
many streamlined methods to get you the information
more quickly. Often you'll find that Sawmill's deep analysis can even
provide you with information you've been after but never knew how to
reach, or
possibly never realized was readily available in reports. Sawmill is an
extremely powerful tool for your business, and most users only exercise
a fraction of this power. That's where our experts really can make the
difference. Our Sawmill experts have many years of experience with
Sawmill
and with a large cross section of devices and business sectors. Our
promise is to very quickly come up with a cost effective solution that
fits your business, and greatly expand your ROI with only a few
hours of fee based Sawmill Professional Services. For more information,
a quote, or to speak directly with a Professional services expert
contact
consulting@flowerfire.com.
Tips & Techniques: Deleting Database Fields
Most log formats contain dozens of fields in the data: each event may
include timestamps, source and destination IPs or addresses, usernames,
pages or filenames, and a myriad of less essential parameters like
server responses, error codes, MIME types, bytes transferred (sometimes
in multiple directions), packets lost, transmission quality, codecs,
versions of software and hardware, etc. A few formats have hundreds of
fields.
In most cases, only a handful of these fields are useful. In a proxy
server analysis, for instance, you might want to know the source IP,
username, destination URL, timestamp, and a of couple others. But
exactly
which fields are useful depends on the application, and the
installation, and the user, so Sawmill's default behavior is to report
them all, to ensure that the fields you want, and the reports you
want, are there in the default profile. The remainder of the fields are
still in the database, however, and they take time to populate into the
main table of the database, time to normalize, time to index, and time
to cross-reference; they also use disk space for all of these purposes.
To optimize the size and performance of your database, you can remove
unused database fields. This is a somewhat involved process, as many
other parts of a profile depend on each database field, but following
the steps below will remove the field, and all its dependencies.
Step 1: Determine If It Is An Aggregating Field
Several of the steps are different for an "aggregating" field vs. a
"non-aggregating field," so first you need to determine which it is.
Aggregating fields, sometimes called "numerical" fields, are fields
like "hits" or "bytes" or "page views", which can be totaled in some
way, often by adding them together, but also by maxima, minima, or
computing number of uniques as in a "unique client IPs" or "visitors"
field. If it makes sense to ask, "what is the total X?", then X is an
aggregating field. Non-aggregating fields are all the others, things
like timestamps, filenames, IP addresses, or anything else that can't
be totaled. Non-aggregating fields are the main fields of reports
(e.g., the "File Types") report; aggregating fields are columns of
reports (e.g., the Visitors columns of a report).
Step 2: Remove The Field From Reports
In Config->Reports, either:
For non-aggregating fields, delete all report elements, and the
reports that contain them, and the report menu items referring to those
reports, which contain the field. For instance, if you're deleting the
File Type database field, delete the File Types report. Also, delete
the File Types report element of the Single-Page Summary. Then, remove
the field from Log Detail, and any other reports it appears in as an
extra column (but default, this does not usually happen for reports
other than Log Detail). Remove it by using the Manage Fields link in
the report element--just unchecking it isn't enough. Finally, delete
the report menu item that points to the File Types report.
- or -
For aggregating fields, remove the field from all reports where
it appears as a column. Remove it by using the Manage Fields link in
the report element--just unchecking it isn't enough.
Step 3: Remove The Field From Cross-Reference Groups
This works just like Step 2, but you remove it from Config ->
Cross-Reference Groups instead. If it's a non-aggregating field, you'll
typically remove its main cross-reference group, and remove it from any
other cross-reference groups it's in; if it's an aggregating field,
you'll remove it from all cross-reference groups it's in.
Step 4: Delete The Report Field
In Config->Report Fields, delete any report fields that use the
database field. There will usually be only one, but if there are
several, delete them all.
Step 5: Delete The Database Field
Now you can finally delete the database field itself, in Config ->
Database Fields.
Step 6: (optional) Delete The Log Field And Log Filter
Steps 1-5 are enough to save the database space, but if time is being
spent parsing and filtering the field, you can speed up the build a bit
by deleting any log filter that sets the field, in Config -> Log
Filters, and then deleting the log field itself, in Config -> Log
Fields.
Advanced Topic: Creating a Custom Action To Delete a Database Field
As a demonstration of custom actions, and to provide a convenient way
to do all the steps above, we have created a custom action,
delete_database_field, for this newsletter. Custom actions are
implemented with CFG files in the Actions folder of LogAnalysisInfo, so
to install this custom action, just drop the attachment into the
Actions folder. Custom actions are invoked from the command line, in
this case like this:
sawmill -p profile -a ddf -dfn dbfieldname
e.g.:
sawmill -p my_profile -a ddf -dfn file_type
The attached action automatically performs all steps 1-5 above (step 6
still needs to be done manually, if desired), deleting the database
field, any report fields that depend on it, any report elements that
use it, and reports that contain only such report elements, and any
report menu items that point to such reports. It also removes the
database field from all xrefs, and deletes any xrefs that are useless
after deleting the field.
This custom action is a particularly sophisticated example of profile
manipulation using a custom action (at 240+ lines, it is twice as long
as any of the other custom actions currently installed). Any programmer
can create custom actions, by writing a file like the one attached.
This custom action will be a standard part of Sawmill 8.1.4 and later.
Professional Services
This newsletter describes the procedure for deleting database fields,
and all dependencies. If
you need
assistance with this or other profile customization, or with creating
custom actions,
or
with
any other Sawmill tasks, our Sawmill Experts
can help. Contact sales@sawmill.net
for more
information.
![]({=fileref("picts/docs/newsletter/2010-03-15/part1.01090803.04040709@flowerfire.com.gif")=})