FAQ: Resource Usage

How much memory/disk space/time does Sawmill use?

Short Answer

It depends on how much detail you ask for in the database. It uses very little if you use the default detail levels.

Long Answer

Memory usage depends mostly on the complexity of your data set (not the size). If your database has fields with millions of unique values, it will use many megabytes for each of those fields. It's uncommon for any particular field to require more than 100M, but in extreme cases, fields can use over 1G.

Disk usage is roughly 200% and 300% the size of your uncompressed log data. In some cases, you may need 400% of your uncompressed data. So if you're processing 500 GB of log data, you'll need about 1500 GB of disk space to hold the database.

The time to process a dataset is roughtly proportional to the size of a database. As of 2004, on a moderately fast single-CPU system, Sawmill typically processes between 5,000 and 10,000 lines of log data per second.