POPauthd

Authenticating Roaming Users For SMTP Relaying Using POP

by M.D. Brownsworth

(Updated 4 May 02)

For the impatient, you can skip the chin-music below and download all files in tar.gz format.

Mirriam-Webster's defines paripatetic as "movement or journeys hither and thither," and itinerant is defined as "traveling from place to place." Both terms accurately describe many users, who nevertheless expect full access to their e-mail accounts on their journeys. This includes being able to relay mail through servers hosting their accounts. These nomadic account holders, known as "roaming users," present special problems for system administrators.

We want our local users, whether they be hither or thither, to be able to relay mail through our servers, but we also want to slam the door with alacrity on spammers. However, authenticating roaming users is quite a vexing problem. Some trusting -- make that foolish -- system administrators address the issue by configuring sendmail with "promiscuous relay," i.e., lax relay rules. Unfortunately, that's just putting out the welcome mat for opportunistic spammers to abuse their servers.

Beginning with sendmail version 8.9, relaying (SMTP forwarding) of messages is not permitted by default. A means of control over users who are allowed to relay mail through the server is provided by the access database, using m4 FEATURE(access_db). There are indeed other means of authentication but, sadly, they are easily exploited. The access file (/etc/mail/access) is essentially a list of all connecting hosts -- IP addresses or hostnames -- who are allowed to relay. To be accurate, it's really access.db that's used by sendmail; access is the human-readable file. An entry has the following form:

240.141.21.140   RELAY
Sometimes the hostname is used instead:
regex.goodhack.com   RELAY
The access database authentication system works exceptionally well: An SMTP relay request is received and the sender's origin IP or hostname is checked against the access database. If it's in the list the message is relayed and the sender is happy, otherwise, the message is rejected. Unfortunately, the system administrator must edit the access file, make an entry for the host to be allowed, and then rehash the database, all manually. It's difficult enough getting the connecting host information from users that stay put. Obtaining ever-changing connecting host information directly from roaming users is impractical, and very close to impossible. What's needed is a system that will authenticate roaming users -- in fact, all users -- automatically.

Enter POPauthd, a Perl-based daemon that uses POP to authorize connecting hosts for SMTP relaying, or forwarding, of mail through the server. The daemon runs in the background, watching the syslog for successful POP logins. When it sees one it enters the IP in sendmail's access database, giving the host authorization to relay through the server. The entries can be expired after a prescribed, configurable period of time with a companion script run via crontab. The best part: The entire system is automatic; no intervention by the system administrator is required.

POPauthd is designed to run on a FreeBSD server, but should function properly on most Unix systems with few, if any, minor changes to reflect filesystem differences. It will work with any POP3 daemon that's capable of writing successful POP logins to a log file.

An excellent choice is Qpopper, available free at http://www.eudora.com/qpopper/. The latest version as of this writing is 3.1.2. Qpopper will be used for the purposes of this how-to. If you decide to use another daemon you may need to change certain variables accordingly. Qpopper will not write to a log file by default; you must use a compile directive to make it do so.

# ./configure --enable-log-login
# make
The newly compiled popper daemon will be in the popper directory. It will log POP login entries to /var/log/messages by default, although specifying a different file with a configure directive is possible.

By default, inetd.conf expects the POP3 daemon to reside in /usr/local/libexec/. Qpopper has no "make install" so the daemon must be copied there manually:

# cp -p popper/popper /usr/local/libexec/
You'll need to enable the following line in /etc/inetd.conf:

pop3    stream  tcp     nowait  root    /usr/local/libexec/popper       qpopper -s
Here's an example Qpopper log entry:

Mar 16 10:54:52 straylight popper[21040]: (v3.1.2) POP login by user "michelle" at (c885447-a.duckburg.or.home.com) 240.141.21.140
Okay, folks, now that we have a POP3 daemon that will log accesses here's the star of the show, POPauthd. The suggested location for it is /usr/local/libexec/.

[ Download this file in text format (shift-click for Windows, option-click for Macs). ]

#!/usr/bin/perl

# popauthd
#
# by M.D. Brownsworth
# 
# Version 2.7
# 6 Feb 02
#
# Authenticates SMTP relay using POP
#
# Daemon runs in background looking for successful POP logins in
# syslog.  When one appears, if user is already in database, only
# timestamp in access.info file is updated; otherwise, entry with
# user's IP is added to access file, database is rehashed, and 
# date, userid, IP, and timestamp are added to access.info.  A
# companion cron script runs periodically to prune expired entries.
#
# Example of qpopper entry for successful login:
# Mar 16 10:54:52 straylight popper[21040]: (v3.1.2) POP login by user "michelle" at (c885447-a.duckburg.or.home.com) 240.141.21.140

require 5.004;

use IO::Seekable;
use Fcntl qw(:DEFAULT :flock);

$syslog = "/var/log/messages";
$maildir = "/etc/mail";
$access = "$maildir/access";
$makeaccess ="$maildir/makeaccess";
$pidfile = "/var/run/popauthd.pid";

open(PID,">$pidfile") or die "Can't open $pidfile: $!\n";
print PID "$$\n";
close(PID);

open (SYSLOG, $syslog) or die "Can't open $syslog: $!\n";
while(1) {
    while(<SYSLOG>) {
        if (/^([A-Za-z]+\s+\d+\s+\d+\:\d+\:\d+).+POP login by user \"(.+)\".+\s(\d+\.\d+\.\d+.\d+).*$/) {
            $date = $1;
            $userid = $2;
            $ip   = $3;
            $timestamp = time;
            $dup = `grep '\\<$ip\\>' ${access}`; # Check to see if IP is already in access file
            if ($dup) { # Duplicate found, update timestamp on existing entry in access.info
                system("sed -e 's/^.*\t$userid\t$ip\t[0-9].*/$date\t$userid\t$ip\t$timestamp/' ${access}.info > ${access}.temp1");
                rename("${access}.temp1","${access}.info");
            } else { # No duplicate found, add new entry in access and access.info
                open(ACCESS, ">>$access") || die "Can't open $access: $!\n";
                flock(ACCESS, LOCK_EX); # Lock in case someone is editing file
                print ACCESS "$ip\tRELAY\n";
                flock(ACCESS, LOCK_UN);
                close(ACCESS);
                system("$makeaccess"); # Rehash access.db
                open(INFO, ">>${access}.info") || die "Can't open ${access}.info: $!\n";
                print INFO "$date\t$userid\t$ip\t$timestamp\n";
                close(INFO);
            }
        }
    }
    sleep 1; # Sleep one second
    SYSLOG->clearerr();
}
close(SYSLOG);

exit(0);
NOTE 1: Feedback from one user indicated that a recent version of Qpopper uses a slightly different log format, which necessitated using a modified regular expression in POPauthd:

if (/^([A-Za-z]+\s+\d+\s+\d+\:\d+\:\d+).+Stats\:(.+).+\s(\d+\.\d+\.\d+.\d+).*$/) {
NOTE 2: Users who wish to use ipop3d instead of qpopper might try the following regular expression:
if (/^([A-Za-z]+\s+\d+\s+\d+\:\d+\:\d+).+Login user\=(.+)\s.+\[(\d+\.\d+\.\d+.\d+).*$/) {
POPauthd utilizes the following small shell script (/etc/mail/makeaccess) to rehash the database.

[ Download this file in text format (shift-click for Windows, option-click for Macs). ]

#!/bin/sh
# makeaccess

/usr/sbin/makemap hash /etc/mail/access < /etc/mail/access
Now, we can't allow the database to fill up forever, can we? Some of the IP's will become outdated, so we need a way to prune out the deadwood from time to time. The following script is run periodically by cron to delete expired entries from the database.

[ Download this file in text format (shift-click for Windows, option-click for Macs). ]

#!/usr/bin/perl

# delexpired.pl
#
# by M.D. Brownsworth
# 
# Version 2.7
# 6 Feb 02
#
# Companion crontab script to popauthd SMTP authorization daemon.
# Deletes expired entries from sendmail access files, rehashes db.
#
# Example entry from access.info file:
# Mar 20 14:03:01    michelle    24.14.231.140   985125786
#
# Example crontab:
# Delete expired entries in relay allow file at 15 minutes after hour
# 15  *    *    *    *    root  /etc/cron_scripts/delexpired.pl

require 5.004;

use IO::Seekable;
use Fcntl qw(:DEFAULT :flock);

$maildir = "/etc/mail";
$access = "$maildir/access";
$makeaccess ="$maildir/makeaccess";
$newtimestamp = time;

# Uncomment desired expiration value below
#$expires = 3600; # 1 hour
#$expires = 10800; # 3 hours
#$expires = 21600; # 6 hours
#$expires = 43200; # 12 hours
#$expires = 86400; # 1 day
#$expires = 129600; # 3 days
$expires = 604800; # 1 week
#$expires = 1209600; # 2 weeks
#$expires = 1814400; # 3 weeks
#$expires = 2419200; # 1 month

open(ACCESS, "$access") || die "Can't open $access $!\n";
flock(ACCESS, LOCK_EX); # Lock file to prevent other updates
open(INFO, "${access}.info") || die "Can't open ${access}.info: $!\n";
flock(INFO, LOCK_EX); # Ditto

open(TEMP, ">>${access}.temp2") || die "Can't open ${access}.temp2: $!\n";
while(<INFO>) {
	($date,$userid,$ip,$timestamp) = split(/\t/, $_);
	chomp($timestamp); # Trim trailing newline char
	next if (($newtimestamp - $timestamp) > $expires); # Discard expired entry
	print TEMP; # Recent entry, keep
}
close(TEMP);

system("awk -F\\\t '{ print \$3 \"\tRELAY\" }' ${access}.temp2 > ${access}.relay"); # Intermediate file
system("[ ! -r ${access}.static ] && { touch ${access}.static; }"); # Create static file if none exists
flock(ACCESS, LOCK_UN);
close(ACCESS);
system("cat ${access}.static ${access}.relay > $access"); # Combine static and relay files
system("$makeaccess"); # Rehash access.db
system("rm ${access}.relay"); # Tidy up
rename("${access}.temp2","${access}.info");
flock(INFO, LOCK_UN);
close(INFO);

exit(0);
In addition to relay permission, the access file is used to reject hosts or networks, using entries such as this:

badspammer.com	REJECT
66.66.66	REJECT
However, when running POPauthd you'll need to place hosts or networks to be rejected in a separate file, named "access.static." In addition to REJECT entries, access.static may be used to ensure that an address will be included in the access list. Any static, unchanging address should be placed in this file. The first time it runs, delexpired.pl will consolidate ALL relay and static entries into the access file.

POPauthd needs to be restarted whenever the syslog is rotated, so that it won't continue watching the old file. Here's a small script that will do the deed. It will be run periodically by cron also.

[ Download this file in text format (shift-click for Windows, option-click for Macs). ]

#!/bin/sh
# restart_popauthd.sh

pid=`cat /var/run/popauthd.pid`

# Kill some time until the syslog rotation is finished
until [ -r /var/log/messages ]
do
	sleep 15
done

kill -9 $pid

/usr/local/libexec/popauthd &
Add the following entries to /etc/crontab:
# Delete expired entries in relay allow file at 15 minutes after hour
15	*	*	*	*	root	/etc/cron_scripts/delexpired.pl
# Restart popauthd after syslog is rotated
0	*	*	*	*	root	/etc/cron_scripts/restart_popauthd.sh
You'll probably want POPauthd to startup automatically on boot, so put the following startup script in /usr/local/etc/rc.d, named "popauthd.sh":

[ Download this file in text format (shift-click for Windows, option-click for Macs). ]

#!/bin/sh
# popauthd.sh

[ -x /usr/local/libexec/popauthd ] && { 
        /usr/local/libexec/popauthd &
        echo -n ' popauthd'
}
Okay, the various elements are in place, the scripts are in their proper locations and set to executable, so we're ready to take POPauthd for the first spin around the block. If you haven't rebooted the server so that our daemon was started automatically, you can invoke it from the command line (making sure to put it in the background with &):

/usr/local/libexec/popauthd &
A ps x should contain a line similar to the following:

573  ??  I      0:00.26 /usr/bin/perl /usr/local/libexec/popauthd
Now use your mail client to check your mail. Afterward, you should see a brand-new file has been created in /etc/mail/: access.info, an information file important to the operation of POPauthd. If you examine this file you should find it contains a fresh entry from your recent POP to the server. An entry has four fields separated by tabs: date, userid, IP, and timestamp; the file will have one entry per line. Here's an example:

Mar 26 18:29:32    michelle   240.141.21.140    985660172
In addition, access should also have a corresponding entry:
240.141.21.140   RELAY
Finally, access.db should have been remade. Again, access.db is in binary format, readable only by sendmail. The users in these files should now have permission to relay mail. Incidentally, sendmail will use the updated access database without the need for a SIGHUP.

If you see no entries in the access files, check your syslog to make sure your popper daemon is running when invoked by inetd, and that it's logging POP logins to the syslog. If it is logging properly, that would suggest that the problem possibly lies with permissions. Are all scripts executable? Are they all in the correct locations? Did you skip or modify an important step or a script? Go over the steps again and check your work carefully.

When everything is working properly you can sit back, secure in the knowledge that POPauthd is diligently watching for valid POP users, ready to automatically authorize them for SMTP relaying and save you, the overworked system administrator, much tedious and time-consuming file maintenance.

That about does it. Happy authenticating!

© 2002 M.D. Brownsworth