Hi all,
maybe this list can give me some feedback on a plugin I've written a few
weeks ago.
The Plugin is based on parts of the 'NiXSpam' project by the German IT
magazine iX. NiXSpam is an elaborate procmail recipe (for more info see
- it's German, though), and it uses a
cool way of computing hashes from the body of mails to detect highly
similar ones (which - propably - are spam).
example:
Given a mail that has at least 16 spaces in it, NiXSpam does the following:
- reduce all duplicate occurences of [:space:]-chars to just one
- remove all characters of the [:graph:]-class
- then compute a MD5-hash and compare that to existing ones.
(For procmail code see end of post)
Now, in NiXSpam this is a purely local thing - hashes are written to a
file, and procmail subsequently simply does a grep on it.
However, in march somebody volunteered to feed the hashes computed by
the iX mail server into a blacklist DNS server. I subsequently wrote a
plugin for SA () and even
managed to set up another DNS server with hashes from our own spam.
Maybe someone is interested in trying out the plugin and report some
results? I, for my part, was surprised to find the spam that hit us to
be apparently quite different from that hitting iX.
BTW: Both tests against our and iX's blacklist hit about 50% of all
incoming mails, and I've still to find a false positive. For me, this works.
Dirk
PS: I think I should mention
- Bert Ungerer, (iX)
- Manuel Schmitt, who hosts iX's blacklist
- KungFuHasi, who posted the perl code computing the hashes @ heise
In procmail, the above mentioned reads as follows:
:0B
* -15^0
# This checksum requires at least 16 spaces/tabs:
* 1^1 [ ]
{
:0 bw
md5hash=|tr -s '[:space:]' \
|tr -d '[:graph:]' \
|md5sum
# Hashsumme bereits in der Datei?
:0 Aw
* ? fgrep -s $md5hash $HASHFILE
{ KNWN=YES }
}

Hi Dirk,

Dirk Bonengel wrote:

Hi all,

maybe this list can give me some feedback on a plugin I've written a
few weeks ago.
The Plugin is based on parts of the 'NiXSpam' project by the German IT
magazine iX. NiXSpam is an elaborate procmail recipe (for more info
see - it's German, though), and it
uses a cool way of computing hashes from the body of mails to detect
highly similar ones (which - propably - are spam).

Correct me if I'm wrong, but this sounds very similar to razor/pyzor/dcc!

Chris T

PGP SIGNATURE
Version: GnuPG v1.2.4 (Darwin)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

S7r6z1HGxEtlwWvGeUnN1ZM=
=2P78
PGP SIGNATURE

Yes, you're right.

The advantage - at least for me - is that, after having set up rbldnsd,
I can easily access the data I get from the spamtrap addresses we run.
With razor you can't run your own server, and if you're actually allowed
to use the client if you use SpamAssassin on a commercial basis is
unclear (to me at least).
With DCC and Pyzor you could set up your own server, but would either
lose the much efficieny of DCC/Pyzor in doing so or would have to tinker
with the C/Python code to use, say, both your own and the 'real' pyzor
data store, I guess.

Chris Thielen schrieb:

Hi Dirk,

Dirk Bonengel wrote:
>
>Hi all,
>>
>maybe this list can give me some feedback on a plugin I've written a
>few weeks ago.
>The Plugin is based on parts of the 'NiXSpam' project by the German
>IT magazine iX. NiXSpam is an elaborate procmail recipe (for more
>info see - it's German, though), and
>it uses a cool way of computing hashes from the body of mails to
>detect highly similar ones (which - propably - are spam).
>
>
>
Correct me if I'm wrong, but this sounds very similar to razor/pyzor/dcc!

Chris T