{= include("docs.util"); start_docs_page(docs.technical_manual.page_titles.newsletters); =}

Sawmill Newsletter

March 15, 2008

Welcome to the Sawmill Newsletter!

You’re receiving this newsletter because during the downloading or purchase of Sawmill, you checked the box to join our mailing list. If you wish to be removed from this list, please send an email, with the subject line of “UNSUBSCRIBE” to newsletter@sawmill.net .

News

Sawmill 7.2.13 shipped on February 21, 2008. This is a minor "bug fix" release, and it is free to existing Sawmill 7 users. It is not a critical update, but it does fix a number of bugs, adds support for many new log formats, and adds a few small features. It is recommended for anyone who is experiencing problems with Sawmill 7.2.11 or earlier. You can download it from http://sawmill.net/download.html .

This issue of the Sawmill Newsletter describes how to use log filters to exclude your own traffic from a web site analysis.

Get the Most out of Sawmill with Professional Services

Looking to get more out of your statistics from Sawmill? Running short on time, but need the information now to make critical business decisions? Our Professional Service Experts are available for just this situation and many others. We will assist in the initial installation of Sawmill using best practices; work with you to integrate and configure Sawmill to generate reports in the shortest possible time. We will tailor Sawmill to your environment, create a customized solution, be sensitive to your requirements and stay focused on what your business needs are. We will show you areas of Sawmill you may not even be aware of, demonstrating these methods will provide you with many streamlined methods to get you the information more quickly. Often you'll find that Sawmill's deep analysis can even provide you with information you've been after but never knew how to reach, or possibly never realized was readily available in reports. Sawmill is an extremely powerful tool for your business, and most users only exercise a fraction of this power. That's where our experts really can make the difference. Our Sawmill experts have many years of experience with Sawmill and with a large cross section of devices and business sectors. Our promise is to very quickly come up with a cost effective solution that fits your business, and greatly expand your ROI with only a few hours of fee based Sawmill Professional Services. For more information, a quote, or to speak directly with a Professional services expert contact consulting@flowerfire.com.

Tips & Techniques: Excluding Your Own Traffic With Log Filters

If you have a web site, you probably visit it yourself, frequently. You might want to ensure that it is functioning properly (through an automated system, or manually), or you might be looking for information on your own site, or there might be portions of your own site that are private and for your use only. When you analyze your web site with Sawmill, your own traffic will normally appear in the reports. If the purpose of the analysis is to determine how other people use your site, your own traffic can be a distraction, or can skew the statistics. Therefore, it is often best to exclude your own traffic, when doing an analysis with Sawmill, so the reports show only external traffic.

At first, it might seem that the way to do this is using a Report filter, by excluding your own IP address using the Filters icon at the top of the reports. That will work, but it is usually not the best way. Report filters do their work while the reports are being generated, after the logs have been imported into the database, and therefore slow down report generation. Reports with complex filters usually require a full table scan, which is a slower report generating method, whereas unfiltered reports can be generated much faster, because they can be generated from cross-reference tables. So if you intend always to discard your own traffic, you should do it with a Log Filter, so you can use unfiltered reports, and get better performance. Log Filters run while the log data is being imported, so they do not affect reporting speed; and a simple log filter doesn't hurt import speed much.

Note: because this technique requires Log Filters, this cannot be done with Sawmill Lite.

Using A Log Filter To Discard Your IP Address

For now, we'll assume that your traffic always comes from a particular IP address. Maybe it's the IP address of your firewall, or maybe it's the IP address of your own computer. The first step is to find out what IP address is yours. If you're behind a firewall, the IP address of your computer will not be the IP address that web servers see. If you're not sure, one way is to go to whatsmyip.com, which will display your IP address as it appears to web servers on the Internet. Let's assume that your IP address is 12.34.56.78. Now we'll create a log filter to reject all events from that IP. First, click View Config next to the profile name, in the Admin page, to go to the Config section of your profile: There, click Log Data and then Log Filters, in the left menu, to see the current list of log filters. Click New Log Filter, in the upper right of the log filters list, to create a new log filter, and type a name for it, like "Reject my IP address":

Now, click the Filter tab, and click New Condition to set up the condition for the filter. The condition is met when the IP address is 12.34.56.78, so choose the IP address field from the Log field menu (this will vary depending on log format; it will be "Hostname" for Apache logs, and "Client IP" for IIS logs), choose "is equal" from the Operator menu, and enter 12.34.56.78 in the Value field:

Click OK, and click Add Action to add the action this filter is to take when the condition is met. The action should be to reject this entry (we're rejecting all entries from your IP address), so choose "Reject log entry" from the Action menu:

Click OK. At this point, you could click Save and Close--the log filter is done. But for performance reasons, and to ensure no earlier filters explicitly accept the events you're trying to reject, it's best to have rejecting filters at the top of the list of log filters, so before you save this, move it to the top by clicking Sort Filters, and clicking Up [ + ] until the new filter is at the top:

Now click Save and Close, and the log filter will be saved permanently to the profile.

If you view reports now, you won't see any change, because log filters only have an effect while importing log data. So it is now necessary to rebuild the database, which you can do by clicking Rebuild Database at the top of the Config page. After you rebuild the database, all reports will show only hits from IP addresses other than 12.34.56.78.

Advanced Topic: More Sophisticated Filters

The simple example above rejects only a single IP address. That is sufficient for some purposes, but if your internal traffic is not always from the same IP, you will need a more complex filter. Log Filters of any complexity can be created in the Log Filter Editor. For instance, if your internal traffic is from any of 12.34.56.76, 12.34.56.77 or 12.34.56.78, you can create a filter with three conditions (by clicking New Condition three times) to reject hits from all three IPs:

This same filter could be done using a regular expression, by choosing "Matches regular expression" from the Operator menu in the Condition page, creating a filter which uses a regular expression to determine which IP addresses to reject:

By customizing the regular expression, you can reject any class of IP address which can be described by regular expressions (which are very flexible).

The same filter could also be implemented with even more flexibility, by choosing "Advanced expression syntax" from the Filter Type menu. This allows you to enter an arbitrary Salang expression. Salang is the "Sawmill Language," and allows fully general programming language syntax, including conditions, loops, variables, subroutines, and more. It's overkill for this example, but is useful for complex ranges, and for conditions which involve more than just the IP address. The three-IP filter could be implemented as an IP range in Salang using this expression:

Questions or suggestions? Contact support@sawmill.net. If would you like a Sawmill Professional Services expert to implement a customization, contact consulting@sawmill.net.

[Article revision v1.0]
[ClientID: ]

{= end_docs_page() =}