Newsletters
 |
Sawmill Newsletter May 15, 2008 |
Welcome to the Sawmill Newsletter!
You’re receiving this newsletter because during the downloading or purchase of Sawmill, you checked the box to join our mailing list. If you wish to be removed from this list, please send an email, with the subject line of “UNSUBSCRIBE” to newsletter@sawmill.net .
News
Sawmill 7.2.14 shipped on March 26, 2008. This is a minor "bug fix" release, and it is free to existing Sawmill 7 users. It is not a critical update, but it does fix a number of bugs, adds support for several new log formats, and adds a few small features (mostly notably, a new "Save To Menu" button in the reports, and the "Use Overview for totals" option discussed below). It is recommended for anyone who is experiencing problems or slow reports with Sawmill 7.2.13 or earlier. You can download it from http://sawmill.net/download.html .
This issue of the Sawmill Newsletter describes the new CFGA file format, which can be used to selectively override CFG files (log formats, profiles, or any other CFG files).
Get the Most out of Sawmill with Professional Services
Looking to get more out of your statistics from Sawmill? Running short on time, but need the information now to make critical business decisions? Our Professional Service Experts are available for just this situation and many others. We will assist in the initial installation of Sawmill using best practices; work with you to integrate and configure Sawmill to generate reports in the shortest possible time. We will tailor Sawmill to your environment, create a customized solution, be sensitive to your requirements and stay focused on what your business needs are. We will show you areas of Sawmill you may not even be aware of, demonstrating these methods will provide you with many streamlined methods to get you the information more quickly. Often you'll find that Sawmill's deep analysis can even provide you with information you've been after but never knew how to reach, or possibly never realized was readily available in reports. Sawmill is an extremely powerful tool for your business, and most users only exercise a fraction of this power. That's where our experts really can make the difference. Our Sawmill experts have many years of experience with Sawmill and with a large cross section of devices and business sectors. Our promise is to very quickly come up with a cost effective solution that fits your business, and greatly expand your ROI with only a few hours of fee based Sawmill Professional Services. For more information, a quote, or to speak directly with a Professional services expert contact consulting@flowerfire.com.
Tips & Techniques: Using CFGA Files Incrementally Override CFG Files
Important Note: The feature described in this article was
introduced in Sawmill 7.2.11, and will not work in earlier versions.
Sawmill uses its CFG format (CFG stands for "ConFiguration Group,"
as it is usually a group of configuration options or values) in most
situations where it needs to store formatted, hierarchical, textual
data on disk. These include profiles (in LogAnalysisInfo\profiles), log
format plug-ins (in LogAnalysisInfo\log_formats); the files used to
match spiders, search engines, and worms (spiders.cfg, worms.cfg, and
search_engines.cfg, all in LogAnalysisInfo); preferences; language
modules (LogAnalysisInfo\languages), and more.
Simple uses of Sawmill do not require direct editing or viewing of
CFG files at all, but advanced uses often require editing profiles,
creating or editing log format plug-ins, and performing other CFG file
edits.
Editing CFG files works fine until you upgrade to a new release of
Sawmill. At that point, if you've edited search_engines.cfg to add a
new search engine to Sawmill's detection, you'll have a choice: keep
your edited search_engines.cfg, or use the latest one. If you use the
latest one, you'll get whatever is new in Sawmill; but you'll lose your
own edits, and have to re-do them in the new version of
search_engines.cfg. If you keep yours, you'll keep your edits, but you
won't get anything that's new, including new search engines added, or
bugs fixed, as part of the Sawmill upgrade.
CFGA Files Are Patches Applied To CFG Files
The solution is to use CFGA files, instead of editing CFG files.
CFGA files are "ConFiguration Group Addition," so called because they
add new information (or modify existing information) to a CFG file,
from Sawmill's perspective. They allow you to edit the contents of a
CFG file as Sawmill sees it, without actually editing the
original text file. This lets you add a new search engine, for
instance, without actually editing search_engines.cfg.
To use CFGA files, create a file next to a CFG file, with the same
name except for the CFGA extension. In that file, use the same
structure as the CFG file (the same internal groups), and any options
you select will automatically be patched into the CFG file whenever
Sawmill uses it.
Example: Adding A Search Engine To search_engines.cfg
For example, suppose search_engines.cfg looks like this (this is a
simplified version of the usual search_engines.cfg, which contains many
more entries):
search_engines = { yahoo = { name = "Yahoo" substring = "yahoo." regexp = "yahoo\\.[^/]+/.*[&?]p=([^&]*)" } # yahoo lycos = { name = "Lycos" substring = "lycos.com" regexp = "lycos\\.[^/]+/.*[&?]query=([^&]*)" } # lycos google2 = { name = "Google" substring = "google." regexp = "google\\.[^/]*/.*[&?]q=([^&]*)" } # search_engines |
search_engines.cfg
Now, suppose your favorite search engine isn't there. You want to add
MSN Search. You could do it by adding it to search_engines.cfg
directly, like this:
search_engines = { yahoo = { name = "Yahoo" substring = "yahoo." regexp = "yahoo\\.[^/]+/.*[&?]p=([^&]*)" } # yahoo lycos = { name = "Lycos" substring = "lycos.com" regexp = "lycos\\.[^/]+/.*[&?]query=([^&]*)" } # lycos google2 = { name = "Google" substring = "google." regexp = "google\\.[^/]*/.*[&?]q=([^&]*)" msn_search = { name = "MSN Search" substring = "search.msn." regexp = "search\\.msn\\.[^/]*/.*[&?]q=([^&]*)" } # msn_search } # search_engines |
search_engines.cfg (with MSN Search added)
That would work--you'd see "MSN Search" in your Search Engines report.
But the next time you updated to the newest Sawmill, your change would
be overwritten by the newest search_engines.cfg. A better solution is
to create a file in LogAnalysisInfo (next to search_engines.cfg) called
search_engines.cfga:
search_engines = { msn_search = { name = "MSN Search" substring = "search.msn." regexp = "search\\.msn\\.[^/]*/.*[&?]q=([^&]*)" } # msn_search } # search_engines |
search_engines.cfga
When Sawmill goes to look at search_engines.cfg, it will automatically
add the information from search_engines.cfga, as it reads it, and the
effect will be the same as if you had modified search_engines.cfg. On
the next update, search_engines.cfg will be overwritten, but your
modifications will not be lost, because they are in
search_engines.cfga, which is not overwritten (because it is not part
of the standard distribution of Sawmill).
Example 2: Adding A Log Filter To A Log Format Plug-in
Suppose you always want to discard spider traffic from your Apache
logs. You could do this by adding a log filter like this one, each time
you create a profile, to the log_filters section (log.filters):
(... beginning of file omitted ...) reject_spiders = `if (spider ne '(not a spider)') then 'reject';` (... end of file omitted ...) |
But that would affect only that profile; it would have to be re-done for any future profiles. So a better solution is to add that filter to the apache_combined.cfg plug-in, in the same place as you would add it in the profile (log.filters). That would add the filter to all future profiles created for that log format. But the plug-in change would be overwritten when you upgrade to a new release of Sawmill, so you would have to re-do the plug-in edit after each upgrade. So the best solution is to create a new file, in LogAnalysis\log_formats, called apache_combined.cfga (next to the apache_combined.cfg log format plug-in file), which contains this:
apache_combined = { log.filters.reject_spiders = `if (spider ne '(not a spider)') then 'reject';` } # apache_combined |
This has the same effect as adding that line to apache_combined.cfg--it adds the filter to the bottom of the list of any new profile created from that plug-in--but when you upgrade to a new version, the plug-in change will not be overwritten. So this has to be done only once, and will add that filter to all future profiles for that format, for all future upgrades of Sawmill.
Other Uses Of CFGA Files
This can also be used to:
- Override or add final_step in a log format plug-in, to make any other changes desired to the profile
- Package up a set of desired changes to a profile, and turn the
changes on or off by moving or renaming the CFGA file
- Add or modify the spiders, worms, server responses, or other lists
- Modify the language modules in LogAnalysisInfo\languages
- Anything else that would normally require a CFG edit
[Article revision v1.0]
[ClientID: 46]