# HG changeset patch # User heiko # Date 1216643437 0 # Node ID d14cfa1c1298fcf9646ee6c5f3cde55310dd48c0 # Parent dcef47ed5ec2128ae1661fe9e6d74b309ef69581 - changed logging (now via syslog) - output elapsed time to syslog diff -r dcef47ed5ec2 -r d14cfa1c1298 mail2db --- a/mail2db Mon Jul 21 08:03:11 2008 +0000 +++ b/mail2db Mon Jul 21 12:30:37 2008 +0000 @@ -13,10 +13,10 @@ use File::Temp; use Unix::Syslog qw(:macros :subs); use Text::Iconv; +use Time::HiRes qw(gettimeofday tv_interval); use if $ENV{DEBUG} => "Data::Dumper"; -my $DSN = "DBI:mysql:mail:hostname=schnuffi"; -my @CREDENTIALS = qw(mail PWUle5Eimi); +my $T0 = [gettimeofday()]; my $OUTPUT_CHARSET = "UTF8"; my $DEFAULT_INPUT_CHARSET = "ASCII"; @@ -27,6 +27,7 @@ my $opt_dsn = ""; my $opt_dbuser = ""; my $opt_dbpass = ""; +my $opt_debug = 0; my $DBH; @@ -35,8 +36,12 @@ sub get_headers($); sub decode_headers($$); + + MAIN: { - openlog( "mail2db", LOG_PID | ( -t STDERR ? LOG_PERROR: 0 ), LOG_MAIL ); + openlog( "mail2db", LOG_PID | ( -t STDIN ? LOG_PERROR: 0 ), LOG_MAIL ); + $SIG{__DIE__} = sub { die $@ if $^S; syslog(LOG_ERR, "ERROR: %s", join "", @_); exit 2 }; + $SIG{__WARN__} = sub { syslog(LOG_WARNING, "%s", join "", @_) }; if ( -f ( $_ = "/etc/mail2db.conf" ) ) { open( X, $_ ) or die "Can't open $_: $!\n"; @@ -49,6 +54,7 @@ "h|help" => \$opt_help, "m|man" => \$opt_man, "n|dry" => \$opt_dry, + "debug!" => \$opt_debug, ) or pod2usage(); pod2usage( -verbose => 1, -exitval => 0 ) if $opt_help; @@ -59,6 +65,9 @@ { RaiseError => 1, FetchHashKeyName => "NAME_lc", AutoCommit => 0 } ) or die; + # The $message contains the in-core representation, the MIME parser + # works on it. The $tmpfile (handle) will be used for access to the + # original unmodified message. my ( $tmpfile, $message ) = get_message(); decode_headers( $message, $OUTPUT_CHARSET ); @@ -92,11 +101,11 @@ # first insert the message and get the database message id my $msg_id; { - seek( $tmpfile, 0, 0 ); local $/ = undef; + seek($tmpfile, 0, 0); $insert_message->execute(<$tmpfile>); $msg_id = $DBH->last_insert_id( undef, undef, message => "id" ); - syslog( LOG_DEBUG, "message id: $msg_id" ); + syslog( LOG_DEBUG, "message id: $msg_id" ) if $opt_debug; } # now insert the message headers @@ -108,7 +117,7 @@ ) { $header =~ s/\s*$//; - syslog( LOG_DEBUG, "$tag\[$idx]\n" ); + syslog( LOG_DEBUG, "$tag\[$idx]\n" ) if $opt_debug; # first we'll give it a try, but it may fail, because the # header_field.id is missing @@ -126,6 +135,7 @@ } $DBH->commit if not $opt_dry; + syslog(LOG_NOTICE, "inserted message $msg_id (%.1fs)", tv_interval($T0)); } @@ -134,14 +144,19 @@ # we'll create a tmp file containing the complete message # if speed matters we should use a ram disk. # unfortunely the MIME::Parser may temporary files too - my $tmpfile = new File::Temp( TEMPLATE => "mail2db-XXXXXX" ); + my $tmpfile = new File::Temp( TEMPLATE => "/tmp/mail2db-XXXXXX" ); local $_ = <>; + die "No input" if not defined; + if ( !/^From\s/ ) { my $nl = /\r?\n$/; print {$tmpfile} "From - @{[scalar localtime]}$nl", $_; } - $/ = undef; - print {$tmpfile} $_, <>; + + { + local $/ = undef; + print {$tmpfile} $_, <>; + } $tmpfile->autoflush(1); seek( $tmpfile, 0, 0 ); @@ -221,14 +236,20 @@ =head1 SYNOPSIS -mail2db [--dsn=I] [--dbuser=I] [--dbpass=I] [-n|--dry] +mail2db [--dsn=I] + [--dbuser=I] [--dbpass=I] + [--[no]debug] [-n|--dry] [message] mail2db [-h|--help] mail2db [-m|--man] =head1 DESCRIPTION -B reads a RFC822 message from stdin and saves it into a database. +B reads a RFC822 message and saves it into a database. +If a file name is passed on the command line, it's assumed to be the +message file. Otherwise the message is expected on STDIN. Please see +the paragraph about LOGGING below. + For more information please see the source code itself. =head1 OPTIONS @@ -245,6 +266,10 @@ The credentials to access the database. (default: "") +=item B<--[no]debug> + +More output (to syslog/stderr), could help debugging. (default: 0) + =item B<-n>|B<--dry> Do not modify the database, just start a transaction but do not commit it. @@ -259,6 +284,12 @@ =back +=head1 LOGGING + +Log output goes to syslog. Some of the output lines are logged with priority +"DEBUG". Errors are logged as "ERR", and warnings as "WARNING". If standard +input is connected to a terminal, additional logging goes to STDERR. + =head1 DATABASE LAYOUT Beside having transactions and real support for foreign keys, the