{= include("docs.util"); start_docs_page(docs.technical_manual.page_titles.newsletters); =}

Sawmill Newsletter

  November 15, 2007



Welcome to the Sawmill Newsletter!

 You're receiving this newsletter because during the downloading or purchase of Sawmill, you checked the box to join our mailing list. If you wish to be removed from this list, please send an email, with the subject line of "UNSUBSCRIBE" to newsletter@sawmill.net .


News

Sawmill 7.2.10 shipped on August 4, 2007. This is a minor "bug fix" release, and it is free to existing Sawmill 7 users. It is not a critical update, but it does fix a number of bugs, adds support for many new log formats, and adds a few small features. It is recommended for anyone who is experiencing problems with Sawmill 7.2.9 or earlier. You can download it from http://sawmill.net/download.html .

This issue of the Sawmill Newsletter describes the Create Many Profiles feature, which can be used to greatly simplify the creation and maintenance of many similar profiles.


Get the Most out of Sawmill with Professional Services

Looking to get more out of your statistics from Sawmill? Running short on time, but need the information now to make critical business decisions? Our Professional Service Experts are available for just this situation and many others. We will assist in the initial installation of Sawmill using best practices; work with you to integrate and configure Sawmill to generate reports in the shortest possible time. We will tailor Sawmill to your environment, create a customized solution, be sensitive to your requirements and stay focused on what your business needs are. We will show you areas of Sawmill you may not even be aware of, demonstrating these methods will provide you with many streamlined methods to get you the information more quickly. Often you'll find that Sawmill's deep analysis can even provide you with information you've been after but never knew how to reach, or possibly never realized was readily available in reports. Sawmill is an extremely powerful tool for your business, and most users only exercise a fraction of this power. That's where our experts really can make the difference. Our Sawmill experts have many years of experience with Sawmill and with a large cross section of devices and business sectors. Our promise is to very quickly come up with a cost effective solution that fits your business, and greatly expand your ROI with only a few hours of fee based Sawmill Professional Services. For more information, a quote, or to speak directly with a Professional services expert contact consulting@flowerfire.com.


Tips & Techniques: Using "Create Many Profiles" to Create and Maintain Many Similar Profiles

In large multi-user environments, like web hosting companies, Sawmill is used to provide one profile per customer. In other situations, Sawmill is used to manage one profile per server, or one profile per device. In these and similar situations, the Sawmill installation can have hundreds or thousands of profiles, all of them very similar. Creating all these profiles manually, using the Create Profile wizard, can be very time-consuming. When a change needs to be made to all of them, it can take a very long time if it is done separately for each profile through the Config interface.

The solution is to use Create Many Profiles, a feature of Sawmill which lets you create as many similar profiles as you like, in a single step. Create Many Profiles works in three stages: create a template profile, create the create_many_profiles.cfg file, and run the command to generate or regenerate the profiles.

Stage 1: Create  a Template Profile

The first step is to create the template profile. For this example, we'll assume that we're creating only three profiles for the web sites, site1.com, site2.com and site3.com, on a Windows server where the logs are at C:\logs\site1, C:\logs\site2 and C:\logs\site3. The template profile is usually a separate profile from any of the final profiles, so we'll call it "template". Start by creating this "template" profile, and pointing it to the log data for site1.com. This first profile is created using the Create Profile wizard (the usual way, through the web interface). Enter C:\logs\site1 as the log source for the profile. This template profile won't actually process logs or generate reports, but it needs to see some log data so it knows what the format is, so all the format-related options can be propagated to the other profiles. Finish creating the profile, and call it "template." Do not build the database or view reports; this template exists only to be a model for other profiles.

Stage 2: Set up create_many_profiles.cfg

Now, using a text editor like Notepad, edit the file LogAnalysisInfo\miscellaneous\create_many_profiles.cfg . This is the file which describes the profiles you want to create from the template. In this case, it might look like this:


  create_many_profiles = {
                                                    
    template_profile_name = "template"
  
    profiles = {
  
      site1 = {
  
        changes = {
          label = "Site 1"
          log.source.0.pathname = "c:\\logs\\site1"
        }
  
      } # site1
  
      site2 = {
  
        changes = {
          label = "Site 2"
          log.source.0.pathname = "c:\\logs\\site2"
        }
  
      } # site2
  
      site3 = {
  
        changes = {
          label = "Site 3"
          log.source.0.pathname = "c:\\logs\\site3"
         }
  
      } # site3
  
    } # profiles
  
  } # create_many_profiles                      


The parts of this file are the following: First, the whole thing is enclosed in the create_many_profiles group, which just wraps the rest of the data into a CGF file.


  create_many_profiles = {
           
    ...

  } # create_many_profiles                      


The following line means that the profile called "template" (the internal profile name, as it appears when you look in LogAnalysisInfo\profiles, without the .cfg extension) is used as the template to create all the other profiles.


  ...
    template_profile_name = "template"
  ...            


Within the create_many_profiles section, there is a "profiles" section, which contains information about the profiles to create:


  ...
    profiles = {
 
    ...
  
    } # profiles
  ...

And within the "profiles" section, there are three sections, one for each profile, like this one:


  ...
      site1 = {
  
        changes = {
          label = "Site 1"
          log.source.0.pathname = "c:\\logs\\site1"
        }
  
      } # site1
  ...

The section above (site1) means that the Create Many Profiles operation should create a profile whose internal name is "site1" (e.g., it will be site1.cfg in the profiles folder), and whose label is "Site 1" (i.e., it will appear as "Site 1" in the web interface). All settings will be copied from the template profile--it will be an exact clone of the template profile, except for the settings specified here, which are the internal name, the label, and the log.source.0.pathname option. That option (log.source.0.pathname) refers to the "pathname" option in the "0" group of the "source" group of the "log" group of the profile, which, for a profile with one log source, is the pathname of the log data. (If you look in the LogAnalysisInfo\profiles\templates.cfg file, you can see this structure, with the curly-bracketed "log" group containing the "source" group, etc.) So this will create a profile based on "template", but with internal name "site1", with label "Site 1", and which reads its log data from C:\logs\site1 (the single \'s must be replaced by \\'s within CFG file string options).

Similarly, the site2 and site3 sections create the Site 2 and Site 3 profiles.

Stage 3: Create the profiles

To create the profiles, you need to run the following command line on Windows. Open Command Prompt, use the "cd" command to change to the Sawmill installation directory, and run Sawmill with the "-dp templates.admin.profiles.create_many_profiles" option to create the profiles, like this:

  cd C:\Program Files\Sawmill 7
  SawmillCL -dp templates.admin.profiles.create_many_profiles

Or on non-Windows:

  cd <sawmill directory>
  ./sawmill -dp templates.admin.profiles.create_many_profiles

When the command completes, all the specified profiles will have been created, just as though they had all been created separate using the Create Profile wizard. Now you can proceed with building databases, viewing reports, etc., for each profile.

Modifying all the profiles

Now maintenance of the profiles becomes very easy. Suppose you want to add a log filter to all profiles. Just open the "template" profile in Config, edit it to add the log filter, and go to Stage 3 again, above, to recreate the profiles. Recreating the profiles does not delete or rebuild the database, so it can be safely done at any time. It can even be scheduled using an external scheduler like cron or Windows Scheduler, to recreate all profiles every night, to pick up the previous day's changes to "template", or to create_many_profiles.cfg .

Adding another profile

Adding another profile is also very easy. Just add a new section (e.g., site4) to create_many_profiles.cfg, and repeat Stage 3 to recreate all profiles. It won't affect the other profiles, and it will create the new profile.

Automation

Because the Create Many Profiles feature uses a text file and the command line, it is very easy to automate from a scripting environment. Just have your script edit, or rewrite, the create_many_profiles.cfg file to list all profiles and their modifications, and have the script run the command line to regenerate all the profiles any time something changes.


Questions or suggestions? Contact support@sawmill.net. If would you like a Sawmill Professional Services expert to implement this, or another customization, contact consulting@sawmill.net.


[Article revision v1.2]
[ClientID: ]


{= end_docs_page() =}