Messages from non-subscribers are held for manual disposition. Since the list posting addresses are in circulation on the Internet, they get spam. The queue of held messages thus quickly accumulate spam. They need to be cleaned on a regular basis -- at least weekly, sometimes daily. Failure to do so results in the queue growing to unworkable proportions depressingly quickly.

To clear the queue: Log in to the admin interface, click "Tend to pending administrative requests", and deal with the resulting pile of crap^W^W^W list of messages. Most messages will be spam. So you generally go along, set each message to "Discard", and then click "Submit".

Quick Links

For convenience, the following links to the various queues may be useful.

• http://mail.gnhlug.org/mailman/admindb/gnhlug-discuss
• http://mail.gnhlug.org/mailman/admindb/gnhlug-jobs
• http://mail.gnhlug.org/mailman/admindb/gnhlug-announce
• http://mail.gnhlug.org/mailman/admindb/gnhlug-org

Tip: Put the above links in a bookmark folder in Firefox, and use the "Open in tabs" option that appears at the bottom of the resulting bookmark menu.

Discard-All Bookmarklet

Most messages end up being discarded. One quickly wishes "Discard" were the default, rather then "Defer". The following bookmarklet will accomplish that; it sets all the checkboxes to "Discard". Of course, you still have to weed through all the spam and set any non-spam to "Approve". A little caution is in order; it is easy to accidentally discard a useful post this way.

Discard All

The above bookmarklet is a kludge; it may break if the Mailman config or version changes, or on browsers other then Firefox 1.5, or if the wind blows.

More automation by Jeff Kinz

As of 11/03/2005, Jeff Kinz is running a curl/bash/bogofilter script which automatically discards 92-95% of these on the gnhlug-org list. The idea for this tool was generated by the thread seen here:

http://www.mail-archive.com/gnhlug-org@mail.gnhlug.org/msg01363.html

The final shape of the script bears almost no resemblance to the ideas in the discussion. The final approach is based on the principle of "Do No harm". No false positives are allowed, no submissions approved without direct human intervention, no notification emails can be generated that might be sent to forged addresses, must not interfere with the existing system's function and the tool must not put extra load on the servers.

Current approach:

1. When Mailman gets a non-member submission it sends an email to gnhmod@Jeff's domain activating a procmail recipe which kicks off the script.
2. A curl command is used to fetch the gnhlug mailman pending requests page, containing fragments of non-member submissions to the list.
3. The script loops through each submission, extracting the fragment and using bogofilter to score its "spaminess".
4. If the score is twenty percent higher than the cutoff for spam, the email is automatically discarded. Because the spam going to the list is very uniform this results in more than 90% of the spam being discarded immediately despite the 20% higher score requirement.
5. Low-scoring, non-discarded fragments are kept until a human can review them using the same tool in a different mode. The human can discard the spam and/or feed the spam into bogofilter so that any more of the same spam will be identified by the system.

Stats as of: Mon 01/02/06 09:43 Total reviewed: 1411, killed: 1320 (93%), ignored: 91

The approach is described here so that it can be reviewed for potential problems. From its it activation to Jan 1, 2006, this system was carefully watched. It seems to be performing well and will not be so carefully monitored. If you notice any real or even potential problems with this approach please let Jeff Kinz know immediately.