[Issues] [mod_log_sql 0000077]: Please integrate attached patch to support configurable logging tables like in libapache-dbilogconfig-perl
issues at outoforder.cc
issues at outoforder.cc
Fri Oct 19 11:54:59 EDT 2007
A NOTE has been added to this issue.
======================================================================
http://issues.outoforder.cc/view.php?id=77
======================================================================
Reported By: korn
Assigned To:
======================================================================
Project: mod_log_sql
Issue ID: 77
Category: Feature Request
Reproducibility: always
Severity: feature
Priority: normal
Status: new
Apache Version: unspecified
======================================================================
Date Submitted: 10-17-2007 07:30 EDT
Last Modified: 10-19-2007 11:54 EDT
======================================================================
Summary: Please integrate attached patch to support
configurable logging tables like in libapache-dbilogconfig-perl
Description:
Hi,
This patch (against 1.101) adds support for a customizable logging table,
similar to what the Apache::DBILogConfig perl module can do. We've been
using it in production for months now, and it appears to work.
Unfortunately, it isn't fully backwards compatible with the original
version (the LogSQLTransferLogFormat directive needs to be adjusted
somewhat). None of the option letters were changed, but the list of
supported letters has been expanded and it's now possible to specify what
each field is called in the SQL table.
This makes it a lot easier to migrate from an existing dbilogconfig setup
to mod_log_sql.
We are willing to "support" the patch in the sense that if changes are
needed to better integrate it with mod_log_sql, we'll see what we can do.
Should you decide to include the patch in your distribution, we'd be
grateful if you could list Allied-Visions GmbH
(http://www.allied-visions.de/) in the credits.
Best regards,
Andras
======================================================================
----------------------------------------------------------------------
urkle - 10-18-07 08:09
----------------------------------------------------------------------
Please check out SVN trunk of mod_log_sql I have some pending changes in
there for a 1.102 release that adds basically what your patch does (for
the configurability of the SQL fields) I would just need to add in the
extra *extract* functions. Please check it out and see if it does
everything you need.
Summary of new commands
Here is the summary of the new commands.
LogSQLShowConfig logs/sqlconfig.txt
This command will cause a file to get created (in this case
sqlconfig.txt in the apache log directory) that will contain a dump of
the mod_log_sql registered functions and field mappings. You can use
this as interim documentation for the list of TransferLogItem full names.
LogSQLRegisterItem myitem cookie mycookie myfield INT 4
This will register a new item "myitem" using function mycookie with
argument "mycookie" (ie find cookie arg 'mycookie') and store it in the
SQL field myfield of data type INT w/ a size of 4.
Currently the size parameter is NOT being used, but will in the future
when I update the auto-create table code to use this configuration to
create tables and ALTER them:)
LogSQLTransferLogItems +myitem
This command says "log all the standard sql items (or what was
configured via a previous LogSQLTransferLogFormat) and ADD to it myitem.
All of the old one letter log items now have a full named version that are
migrating to this new command.
----------------------------------------------------------------------
korn - 10-19-07 11:54
----------------------------------------------------------------------
Um, if the trunk is http://svn.outoforder.cc/svn/mod_log_sql/trunk (Last
Changed Date: 2007-07-16 14:45:04 +0200), then I don't see what changes
you mean; svn diff http://svn.outoforder.cc/svn/mod_log_sql/tags/1.101
http://svn.outoforder.cc/svn/mod_log_sql/trunk shows just this:
Index: mod_log_sql_mysql.c
===================================================================
--- mod_log_sql_mysql.c (.../tags/1.101) (revision 174)
+++ mod_log_sql_mysql.c (.../trunk) (revision 174)
@@ -147,10 +147,10 @@
SIGNAL_RELEASE
return LOGSQL_QUERY_SUCCESS;
}
- log_error(APLOG_MARK, APLOG_ERR, 0, r->server,
- "mysql_query returned (%d)", retval);
+ real_error = mysql_errno(dblink);
+ log_error(APLOG_MARK, APLOG_ERR, 0, r->server,
+ "mysql_query returned (%d) \"%s\"", real_error,
MYSQL_ERROR(dblink));
/* Check to see if the error is "nonexistent table" */
- real_error = mysql_errno(dblink);
if (real_error == ER_NO_SUCH_TABLE) {
log_error(APLOG_MARK,APLOG_ERR,0, r->server,"table does
not exist, preserving query");
Surely you must be referring to something else?
Anyway, what our patch mainly adds is the ability to change the database
table field names a data item goes to.
For example, we had been logging, using libapache-mod-dbiconfig-perl, to a
MySQL table with the following structure:
+-------------+--------------+------+-----+-------------------+----------------+
| Field | Type | Null | Key | Default | Extra
|
+-------------+--------------+------+-----+-------------------+----------------+
| id | int(11) | NO | PRI | NULL |
auto_increment |
| client_ip | varchar(15) | YES | | NULL |
|
| accesstime | timestamp | NO | | CURRENT_TIMESTAMP |
|
| httpmethod | varchar(4) | YES | | NULL |
|
| uri | varchar(200) | YES | | NULL |
|
| querystring | varchar(150) | YES | | NULL |
|
| result | int(3) | YES | | NULL |
|
| referer | varchar(200) | YES | | NULL |
|
| useragent | varchar(100) | YES | | NULL |
|
+-------------+--------------+------+-----+-------------------+----------------+
The submitted patch allows this existing table to be re-used with
mod_log_sql using
LogSQLTransferLogFormat "%w=client_ip %{%Y-%m-%d %T}t=accesstime
%m=httpmethod %U=uri %a=querystring %s=result %{Referer}i=referer
%{User-Agent}i=useragent"
Correct me if I'm wrong, but this wouldn't have been possible, especially
the field with the "timestamp" type, using the old code, and it looks to
us like it still wouldn't be possible using the trunk version (our patch
applies cleanly to that as well, provided we're looking at the correct
trunk).
Our patch adds a new layer to the transferlogformat handling that builds a
linked list of what data to log in what SQL field. This allows us to:
* log the values of arbitrary envvars (as set by apache) to SQL, and
* use a single access log table to log arbitrary incoming and outgoing
headers as well.
E.g. I could say "%{Content-Type}i=incoming_content_type
%{Content-Type}o=outgoing_content_type" to have these two headers logged
in these two fields.
This adds some complexity when parsing the configuration but incurs no
performance loss during logging.
I hope this clarifies what the patch is about. :)
Issue History
Date Modified Username Field Change
======================================================================
10-17-07 07:30 korn New Issue
10-17-07 07:30 korn File Added: dbi.v1.101.patch
10-17-07 07:30 korn Apache Version => unspecified
10-18-07 08:09 urkle Note Added: 0000094
10-19-07 11:54 korn Note Added: 0000095
======================================================================
More information about the Issues
mailing list