Logging

January 21st, 2009 Leave a comment Go to comments

Group chats

IRC is alittle “special”. With its channels, it suports group chatting. IMSpector copes with this in the following way.

The local ID is always the nickname of the person making the IRC connection. The remote ID will be a channel for channel messages, and the other persons nick for private messages. When a remote user says a message on a channel, the event data is prefixed with their nick and a colon followed be the message said.

MSN and Yahoo group chats are logged in a simular way; in both cases a remote party is created “on demand” to hold the group. ICQ/AIM support within IMSpector currently does not support group chats.

File log format

Logs are filed under a path like this:

{protocol}/{local id}/{remote id}/{year}-{month}-{day}

A prefix to this path is stored in the config file:

file_logging_dir=/var/log/imspector

This directory must be owned by whoever the imspector program will run as.

{protocol} is the name of a supported protocol (currently MSN, ICQ-AIM, Jabber, Yahoo or Gadu-Gadu), {local id} is the local identity who sent or received a message (ie the person “behind” the machine running IMSpector, {remote id} is the person “in front” of the IMSpector machine, and the last item is obviously the date of the chat.

The local and remote IDs are different depending on the protocol. For MSN it is the persons registered passport email addresss, for AIM and ICQ it is just a simple string (ICQ is now the same as AIM but has a numeric user ID).

An actual log file will look something like this:

192.168.0.4:62013,1160853400,0,1,0,,yep
192.168.0.4:62013,1160853401,0,1,0,,:~)
192.168.0.4:62013,1160853407,1,1,0,1 smoothie;,smoothie should be in the other rack

Columns are:

  1. IP address:Port of the client machine.
  2. UNIX timestamp.
  3. Outgoing. 1 for an outgoing event, 0 for an incoming event.
  4. Type of event. Currently supported are:
    • 1 – Message.
    • 2 – File transfer. Some protocols only.
    • 3 – Typing event. User is typing a message. Some protocols only.
    • 4 – Webcam event. User requested a webcam session. Yahoo only presently.
  5. Filtered. 0 means the message was not filtered (blocked), 1 means it was. Does not relate to content manipulation, but rather ACLs and other filter plugins.
  6. Message categories. This is usually empty, but if a message has been filtered or censored by one of the filtering plugins, then the “category” of the message will be here. For instance, the badwords filter records the count of replaced words here.
  7. The event data, which for messages happens to be what ever was said. For file transfers, the name and length is logged here. Both the typing events and webcam events do not use this field. Note that the un-manipulated text is logged, ie. before it has been censored for bad words, if the option is enabled. If the message includes embedded new-lines, they are backslashed as \n. Note that, depending on the client software and protocol, the message may contain embedded HTML or other formatting elements.

The log files are never held open, so for the purposes of log rotation, can be truncated or removed freely.

Category file logging

Most people can ignore this little IMSpector feature.

This is a special logging plugin which is not generally needed, but is useful in some situations.

cats_logging_filename=/var/log/imspector/cats

This will log, to a flat file, all events where the category field is not empty. The purpose is to have a seperate log of all questionable messages.

The format of this file is identical to the ordinary file logging plugin, except that each line will be prefixed with the protocol, the local id and the remote id.

MySQL logging

MySQL support is not built unless you invoke the make mysqlloggingplugin.so target. Of course, the client side C MySQL library is required to build this plugin. It adds the following options to the config file:

  • mysql_server – The hostname of the DB server.
  • mysql_database – The database name in the server.
  • mysql_username and mysql_password – Login details for the DB.

The table in the database should be called “messages”. The following SQL can be used to be create it:

CREATE TABLE `messages` (
`id` int(11) NOT NULL auto_increment,
`timestamp` int(11) NOT NULL default '0',
`clientaddress` text NOT NULL,
`protocolname` text NOT NULL,
`outgoing` int(11) NOT NULL default '0',
`type` int(11) NOT NULL default '0',
`localid` text NOT NULL,
`remoteid` text NOT NULL,
`filtered` int(11) NOT NULL default '0',
`categories` text NOT NULL,
`eventdata` blob NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;

If this table does not exist, and the DB user is capable of doing so, this table will automatically be created when IMSpector starts.

Note that only MySQL version 4 and 5 has been tested. Prepared statements are used for efficiency reasons and I believe that these will not work on version 3. Testing on this is welcome, as I am neither an MySQL or SQL bod. Then again, MySQL 3 is probably considered ancient.

The plugin supports some level of resiliance against server disconnects and other errors; if the server is remote and the connection between IMSpector and the DB server is broken, IMSpector will queue the events and periodically attempt a reconnection. When the connection is reestablished, the queued events will be logged at the database in one hit.

PostgreSQL logging

PostgreSQL support is not built unless you invoke the make postgresqlloggingplugin.so target. Of course, the clientside C PostgreSQL libary is needed to build this plugin. It adds the following options to the config file:

  • pgsql_connect – Connection paramaters.

The connection parameters are varied and it depends on the details of your client library and so on, but an example would be:
pgsql_connect=host=localhost dbname=imspector user=dbuser password=Password
The table in the database should be called “messages”. The following SQL can be used to be create it:

CREATE TABLE messages (
id serial primary key,
"timestamp" timestamp with time zone default now(),
clientaddress varchar,
protocolname varchar,
outgoing int default 0,
type int default 0,
localid varchar,
remoteid varchar,
filtered int default 0,
categories varchar,
eventdata text )

If this table does not exist, and the DB user is capable of doing so, this table will automatically be created when IMSpector starts.

The PostgreSQL plugin supports the same form of disconnected operation as the MySQL plugin.

SQLite logging

SQLite support is not built unless you invoke the make sqliteloggingplugin.so target. It adds the following options to the config file:

  • sqlite_file – The filename of the DB.

Once again, a “messages” table is required:

CREATE TABLE messages (
id integer PRIMARY KEY AUTOINCREMENT,
timestamp integer NOT NULL,
clientaddress text NOT NULL,
protocolname text NOT NULL,
outgoing integer NOT NULL,
type integer NOT NULL,
localid text NOT NULL,
remoteid text NOT NULL,
filtered integer NOT NULL,
categories text NOT NULL,
eventdata blob NOT NULL
);

This table is automatically created if it dosn’t already exist.

  1. kaan
    March 30th, 2009 at 10:04 | #1

    First, I wish success. Was a very good work. Thanks to the team.

    My issue is,
    if actively block_files, file is not automatically canceled.
    In the log file every minute to appear again.
    The remote user canceled even log every minute is ongoing.

    TEST environment
    remote user: current live messenger
    local user: mac messenger 7.02
    imspector: 0.8
    operating system: OpenBSD
    output log every minute:
    …..
    …..
    192.168.3.39:52916,1238315065,0,2,1,,50.jpg 477297 bytes
    192.168.3.39:52916,1238315076,0,2,1,,50.jpg 477297 bytes
    192.168.3.39:52916,1238315077,0,2,1,,50.jpg 477297 bytes
    192.168.3.39:52916,1238315089,0,2,1,,50.jpg 477297 bytes
    192.168.3.39:52916,1238315090,0,2,1,,50.jpg 477297 bytes
    192.168.3.39:52916,1238315092,0,2,1,,50.jpg 477297 bytes
    …..
    …..

  2. March 30th, 2009 at 12:03 | #2

    This is unfortunately the nature of MSN. There is no way (that I know of) to get IMSpector to tell the client to stop sending the files. It should eventually give up, though.

  3. September 10th, 2009 at 00:14 | #3

    Hi, a log like this it’s possible?

    {protocol}/{local id}/{remote id}/{year}-{month}-{day}-{hour}-{min}-{sec}

    It would be interesting..

    Thank’s…

  4. September 22nd, 2009 at 15:36 | #4

    I’m not sure why you would want that. Each message would be in a different file?

    Maybe I’m missing something. :)

  1. February 21st, 2009 at 17:42 | #1
  2. March 4th, 2010 at 15:53 | #2
You must be logged in to post a comment.