SAWMILLFORUM

Well...I am going to speak out on this one. I for one would like to see MS-SQL sooner then later.

* Add SQL/IIS log source that extracts log data from a SQL database.

* Add support for MS SQL (e.g. as a back-end database)

Anyone else?

Don't worry about that feature overriding the Log Filters editor or the Reports Editor -- it's on the long-term list (which means that barring extraordinary demand, it won't appear until at least Sawmill 8). The Log Filters editor and Report Editor are on the short-term 7.0.x features list, for implementation in 7.0.x versions. In fact, the Log Filters editor is now implemented in 7.0.10, which shipped yesterday, and it includes functionality similar to that found in the Sawmill 6 Log Filters Editor. The Report Editors is the main feature planned for Sawmill 7.0.11, so you shouldn't have to wait long for that one either.

-> azarc <-

I guess in version 8??

Thanks,

Tariq

I should mention in passing that SQL Log Source is available to the adventurous programmer -- all you need to do is write a script which accesses the SQL server, and dumps the log data from there to the standard output stream, formatted as W3C data (like the text logs IIS normally generates). Then use that script as a Command log source, and you've implemented the feature. When we implement it, we'll probably integrate it into Sawmill, but for now it can be done manually if you have SQL and scripting experience.

However, MS SQL as Backend Database is much, much harder. We're still tuning and fixing our MySQL implementation, and it would be somewhat foolhardy of us to begin implementing another database before that's done. I think we're pretty close on that -- we're doing one last revision to the scheme and we'll probably lock it down and turn our attention to Oracle, or maybe MS SQL.

Oracle was the "natural" next step for us after MySQL because it seems to be the industry standard for a Enterprise database, and SQL is currently available only in Sawmill Enterprise. But given the amount of interest in MS SQL, I'm wondering if we would be better to implement that first. We don't have much of an idea how hard either will be -- in theory, they should be easy, because we already have all the SQL queries put together as part of the MySQL implementation, so it should just be a matter of dealing with the different connection types, and tuning the syntax for the differences between servers. But in practice, I think it will be much harder than it sounds. I'm going to open up an internal conversation on this again, to see if MS SQL might really be the best choice for our "second" database.

The To Do List is an internal list which we maintain -- we don't currently have a public version. Roughly speaking when the first person requests a feature, we add it to the list (which is about 300 items long right now), and give it priority 1. Additional requests bump the priority up by two, so it goes 1, 3, 5, 7, 9 (I'm not sure why it goes by two...). Once it passes 9, priority is set to 10 and in theory we implement it in the next major release. To continue to gauge interest, we keep bumping the number up as more requests arrive, to 11, 12, etc. For comparison, there are 13 priority-10 items, six priority-11 items, two priority-12 items, and nothing above that, except for MS SQL, which is now priority-16. The expected difficulty of the feature makes me reluctant to consider it in Sawmill 7 -- it seems like the sort of thing that should really be in v8 -- but we'll see what's possible.

I should mention that we do have an official "pay-for-priority" policy where in theory, *any* feature can be requested for immediate implementation, for a fee. A feature of this magnitude would have a king-sized pay-for-priority fee, and still would take months to implement, but if some's just DYING to see this in Sawmill, and has some cash to burn making their dream come true, email me at ferrar@flowerfire.com and I'll put together a quote. Maybe you could all chip in....

Thanks

BTW: IT people seem to have much more free reign with MS-SQL implementation then choosing any other product. Personally, I can not choice IBM DB2 or Oracle on a window server without jumping through many flaming hoops. MS-SQL is very easy as there are really no guidelines set around deployment.

It'll let me run reports combining log data with our transactional stuff very easily and allow us to move the database onto our cluster which has a lot more power available.

We seriously considered ODBC but decided that performance was paramount for our application, which frequently needs to add thousands of lines of data per second to the database, and needs to optimize queries in any way possible (including database-specific methods). So for the first three databases (MySQL, MS SQL, and Oracle), we will use the direct C interfaces to provide maximum performance. ODBC is tentatively scheduled for after that, and will provide compatibility with PostgreSQL and other SQL servers, at a cost in performance.

Any idea when Sawmill is going to support MS SQL Databases.
I've read something about the end of this year? Is that correct, or how long will I have to wait for it?

Flo

Thanks,
Jason

You will be giving users the ability to easity access data froma lot from sources. I think this will lead to a lot "custom log" support work. Since FlowerFire now charges for this type of work, I would like to strongly recommend that the feature be put along side the sql priority in the dev list

http://www.sawmill.net/dcforum/DCForumID4/192.html>

I really think a field mapping engine slash scheme definiation wizzard will be a extremely popwereful tool for users to define and map the data they want to leverage sawmill to produce reports. I regoinize the great owr that has been done to the new profiel wizzard and I do not see this mapping tool as a replacement at all but rather a power user type tool maybe even a seperate tool.

This could at least improve the ability of your cutsomers to get some of these data sources operational in sawmmill without exta cost. Of course custom dev work will always be needed,but right I feel that as a customer I'm helpless but to call support each time.

1. There is a major new chapter in the Technical Manual describing plug-in creation in great detail, with examples.

2. Plug-ins are much simpler than they were a year ago, thanks to a lot of work on "default" values, especially in log fields and database fields, so it is usually sufficient to just list a field name in the fields list, without giving it any of the half-dozen attributes that previously had to be included; the attributes will be chosen reasonably for you.

Number 1 especially functions as a basic "log format wizard"; it can be used by moderately technical users to create custom plug-ins without our assistance. Number 2 eliminates a lot of the aggrevation that used to accompany plug-in creation. Having created at least 300 plug-ins myself, I can definitely say that there is a world of difference between plug-in creation now, and one year ago.

I should also point out that you don't have to create a plug-in at all if you have control over the log format; just make it CSV, or W3C, and use the existing "generic" CSV or W3C plug-in, and it will import if automatically. This may be the simplest approach in most cases. Programmers who don't want to learn Sawmill's environment can create programs in their favorite language which convert a particular format to W3C, for instance.

The wizard still is high on our list; but it probably won't make it in v8. ODBC is working now in the v8 pre-release, and that's important enough that we want to get v8 stable and out the door as soon as possible.

ok since I purchased sawmill7 a year ago, what will I need to do to get Sawmill 8 when it comes out? Do I need a licenses upgrade??

Thanks again guys

If you don't, it will be a reduced-cost upgrade. In the past, upgrading from a previous version has been 25% to 50% of the full price. So if you upgrade to a $1000 Sawmill 8 license, it will probably cost between $250 and $500.

There will be a "grandfather" period where people who buy v7 will get v8 free if it ships just afterward; that period has been typically been 3 months, so if we do that again, and Sawmill 8 ships July 1, those who buy Sawmill 7 after April 1 will get it free. This is not guaranteed; first, we don't know for sure when it will ship, and second, we don't know how long the grandfather period will be for v8 yet.