Systems | Library | University of Waterloo

Spam and SpamAssassin Q & A

What is Spam?
How can I get off a spam mailing list?
Why can’t spammers be tracked down and stopped?
Why do some messages in my mailbox have a subject starting with *****SPAM*****?
What is SpamAssassin?
How does SpamAssassin work?
How can I filter my mail once it's been marked as spam?
Some mail I really want is being marked as spam. How do I stop that?
I don't have a user_prefs file.
or
I made changes to my user_prefs file, but they don't seem to be working
How do I change the way SpamAssassin behaves?

What is Spam?

Not to be confused with the meat product, spam is a term commonly used when describing junk email on the Internet. Junk email is on the rise. An anti-spam software maker recently reported a 600 percent increase in spam detected from May 2001 to May 2002.

Spam floods the Internet with many copies of the same message, in an attempt to force the message on people who would not otherwise choose to receive it. Email spam targets individual users with direct mail messages. Email spam lists are often created by scanning Usenet postings, stealing Internet mailing lists, or searching the Web for addresses.

Unfortunately, spam is an inevitable part of the way email functions and there are no public email addresses that are not susceptible to it. It creates a hassle for the home user who has to waste time sifting through it for legitimate email and it costs companies money to process this extra email that is irrelevant to the operation of their business.

How can I get off a spam mailing list?

You probably can’t. Many spam messages say "please send a REMOVE message to get off our list'' or “click on the link below to unsubscribe”. It’s usually a waste of time and, in fact, may simply verify that you have a valid e-mail address and you’ll get even more spam.

Why can’t spammers be tracked down and stopped?

Efforts have been made to shut down spam-generating senders and some sources have been successfully shut down. But there are so many sites and they’re hard to identify. Spammers know that people don't want to hear from them and put fake return addresses on their messages or use “disposable”' trial ISP accounts.

Why do some messages in my mailbox have a subject starting with SPAM?

Your system administrator has installed a mail filter called SpamAssassin on your mail server. Messages which appear to be "spam" (unsolicited bulk mailings) according to the filter are marked for easy identification and filtering.

What is SpamAssassin?

SpamAssassin is a spam tagging system that scans all mail as it comes into the server, compares it to over 650 known spam characteristics, and assigns it a probability of being junk mail. Users can then set up a filter on their email application to look at this tag in the email header and automatically divert the email into a “junk” folder for quick review prior to deletion. SpamAssassin may tag some valid messages as spam but the tagging success rate is high.

The concept of SpamAssasin is very simple. All of your email is directed through this program that rates the email against hundreds of factors and gives it a rating. If the rating is over a pre-determined value, it labels its as SPAM in the subject line and gives SPAM warnings.

All a user has to do is setup a filter in Outlook or PINE to redirect emails with SPAM in the Subject line to a junk folder.

In the example below, for example, SpamAssassin has identified a message as “probably spam” and indicated the criteria by which it made this identification.

Subject: *****SPAM***** Exclusive Mortgage Leads!

SPAM: -------------------- Start SpamAssassin results ----------------------
SPAM: This mail is probably spam. The original message has been altered
SPAM: so you can recognise or block similar unwanted mail in future.
SPAM: See http://spamassassin.org/tag/ for more details.
SPAM:
SPAM: Content analysis details: (22.1 hits, 6.5 required)
SPAM: Hit! (2.7 points) Subject contains lots of white space
SPAM: Hit! (1.0 point) From: ends in numbers
SPAM: Hit! (0.6 points) From: does not include a real name
SPAM: Hit! (0.5 points) Subject has an exclamation mark
SPAM: Hit! (1.9 points) BODY: Contains word 'guarantee' in all-caps
SPAM: Hit! (1.5 points) BODY: Asks you to click below
SPAM: Hit! (0.1 points) BODY: List removal information
SPAM: Hit! (0.5 points) BODY: A WHOLE LINE OF YELLING DETECTED
SPAM: Hit! (0.6 points) BODY: 2 WHOLE LINES OF YELLING DETECTED
SPAM: Hit! (-0.3 points) URI: Includes a link to send a mail with a subject
SPAM: Hit! (1.3 points) URI: Includes a link to a likely spammer email address
SPAM: Hit! (1.9 points) URI: Includes a URL link to send an email with the subject 'remove'
SPAM: Hit! (1.3 points) URI: Includes a 'remove' email address
SPAM: Hit! (4.8 points) BODY: Frontpage used to create the message
SPAM: Hit! (1.8 points) BODY: Tells you to click on a URL
SPAM: Hit! (0.0 points) BODY: Includes a URL link to send an email
SPAM: Hit! (1.4 points) Message text disguised using base-64 encoding
SPAM: Hit! (0.5 points) Received via a relay in ipwhois.rfc-ignorant.org
SPAM: [RBL check: found 83.1.207.200.ipwhois.rfc-ignorant.org., type: 127.0.0.6]
SPAM:
SPAM: -------------------- End of SpamAssassin results ---------------------

Good Morning -
[full text of message would follow – deleted from this example]

How does SpamAssassin work?

SpamAssassin uses several spam-identification tactics, including:

header analysis: Spammers use a number of tricks to mask their identities, fool you into thinking they've sent a valid mail, or fool you into thinking you must have subscribed at some stage. SpamAssassin tries to spot these.
text analysis: Spam mails often have a characteristic style (to put it politely), and some characteristic disclaimers and text. SpamAssassin tries to spot these, too.
blacklists: SpamAssassin supports many useful existing blacklists, such as mail-abuse.org, ordb.org or others.
Razor: Vipul's Razor is a collaborative spam-tracking database, which works by taking a signature of spam messages. Since spam typically operates by sending an identical message to hundreds of people, Razor short-circuits this by allowing the first person to receive a spam to add it to the database -- at which point everyone else will automatically block it.

A list of SpamAssassin tests with their default values can be found at http://spamassassin.org/tests.html.

How can I filter my mail once it's been marked as spam?

You can set up a filter on Outlook or a procmail filter on Pine. Note: you would usually create either an Outlook or a Pine filter, not both. If Outlook is your primary email application, create only an Outlook filter.

A. Creating a filter in Outlook

For the Office 2000 version of Outlook, select Tools from the main menu at the top of the screen, then Rules Wizard. A new window will appear.

To set up a mail filtering rule, click on the New... button, then “Move messages based on content” and select or fill in appropriate information as you work your way through the questions.

To move all messages marked as spam to a "junkmail" folder (the SpamAssassin filter may incorrectly identify some mail you want to read as spam, so don't just throw it away), you would set up a rule like: Where the Subject line contains *****SPAM***** , move it to the junkmail folder. When asked “What do you want to do with the message?”, select the action you want to happen (e.g. “move it to the specified folder”) and ALSO select “Stop processing more rules”.

Click the “Run now” button to apply the rule to all messages currently in your Inbox and click the OK button to save the rule to be automatically applied to all incoming messages in the future.

B. Creating a filter in Pine

Pine has a filtering system that enables you to move or delete messages from one mailbox to another without having to manually move every message. Each filter you set within Pine will perform one action on messages within folders (such as the inbox) you specify. Pine filters only operate when you have Pine running. To set up Pine filters:

From the Pine Main Menu, type S to get to the Setup menu.
In the Setup menu, type R for Rules.
At the bottom of the screen, four options will be presented to you, one of which is Filters. Type F.
You will see a list of commands at the bottom of the Setup Filtering Rules screen that appears. To add a new filter, type A.
The next screen, “Add A Filtering Rule”, will appear. You will need a name for the filter action you wish to perform. This name you choose will be listed on the previous, Setup Filtering Rules screen. To choose a name, make sure Nickname is highlighted. If it is not highlighted, use the up and down arrow keys to highlight that field. Type C. At the bottom of your screen, you will be prompted to enter replacement text. Type a name for your filter and hit Enter.
Following the Nickname field are a number of patterns, which refer to certain parts of an email.
- The first six patterns refer to parts of the header of an email or, in the case of News pattern, a newsgroup message. When you set a value for any of these patterns, the filter will then look through the designated part of the header for a text string that matches your value. You are allowed to set multiple values for a single field, by separating each with commas. The next pattern, Recip, will search the To and Cc fields for a match. Partic will search the From, To and Cc fields. AllText will search the whole header and body. The final field, Score interval, refers to Scoring values that you can configure Pine to assign to your messages.
Recommended for the SpamAssassin rule: select a match by Subject
To set a value for any of the patterns, make sure the pattern is highlighted, or selected. You can select a specific item by using the up and down arrow keys. When you have done that, type C to change the value. Type the string you want to match, and then press Enter. For a SpamAssassin rule, type: *****SPAM***** Be sure to add a space at the end.
Below that list is the Current Folder Type setting. This will allow you to choose what folder, or folders, you wish to filter. All will filter all of your mailboxes and newsgroups. News will filter only newsgroups, while Email will only filter mailboxes. If you select Specific, you will only filter one email folder, like your inbox.
To select one of the four choices, highlight the type you wish to filter and then type *. If you choose Specific, you must also designate the folder you wish to filter.
Following the Current Folder Type section are four message status settings. They can filter messages based on whether or not they are important, new, deleted, or answered. For each of these criteria, you can select Don't care, always matches, Yes, or No by using the same method you used to set the Current Folder Type.
Finally, you need to set the Filter Action. All qualifying messages that fit the defined criteria can either be deleted or moved to another folder. Select either Delete or Move. If you choose Move, you will then have to choose the folder you want all messages that qualify in this filter to be moved to. To set a folder, highlight the appropriate field and type C. You will be prompted to write the folder name. If you do not currently have a folder with your desired name, Pine can make it for you. Type Enter.
Type E to exit the settings for this filter. Pine will ask you if you want to commit the changes. If you are certain about the settings of the filter, type Y.
Now, back at the Setup Filtering Rules screen, you can create more filters or you can exit the screen. To exit, type E. To save your changes, type Y.

Some mail I really want is being marked as spam. How do I stop that?

You should have a directory under your home directory on the Library Unix server called .spamassassin and a file in that directory called user_prefs. If you want to make sure all messages from a particular source (we'll use joe@good.example.com for this example), edit your .spamassassin/user_prefs file to include a line like this:
whitelist_from joe@good.example.com
and all future messages from joe@good.example.com should not be marked as spam.

Or to whitelist all mail originating from UW Library staff:
whitelist_from @library.uwaterloo.ca

I don't have a user_prefs file.
or
I made changes to my user_prefs file, but they don't seem to be working

Mail which arrived before you updated your user_prefs file will not be re-filtered through SpamAssassin even if you haven't read it yet. If you're certain you updated your preferences file before the message arrived or you don’t have a user_prefs file, contact the Systems Help Desk at x2405.

How do I change the way SpamAssassin behaves?

You can reconfigure your SpamAssassin settings by going to this URL:

http://mailservices.uwaterloo.ca/spam/prefs.php

Select “library” from the dropdown box for your server, signon with your UWDIR id. On the subsequent page you can change all of your settings. The “required_hits” item is the value you would want to change if you find the current threshold of 4.0 is too high or too low. Any value you enter will override the global settings.

Alternatively, you can change SpamAssassin's behaviour using the .spamassassin/user_prefs file in your home directory on the Library Unix server. This file should by default contain helpful comments on things people commonly wish to change, but a few are listed here for reference. If you are not familiar with Unix commands, please contact the Systems Help Desk at x2405 before making changes to the file.

whitelist_from	If a message arrives from an address matching this string or pattern, assume it is not spam.
required_hits	This is the minimum score (scores are provided in a header called X-Spam-Status:, which you should be able to see by looking at the full headers or "source" of a message) required to mark a message as spam. Raising this number will reduce "false positives" (messages incorrectly marked as spam) at the cost of increasing "false negatives" (spam which is not tagged as such), and lowering it will do the converse.
report_header	Set this to 1 to put spam reports in the header of the messages, 0 to have them appear in the body of messages.
subject_tag	This is the exact text which is put in front of the original subject line of messages tagged as spam.
use_terse_report	If this is set to 0, a brief description of each test which caused SpamAssassin to mark a message as spam is added to the headers or body of the message. If it is set to 1, only symbolic test names are provided.
defang_mime	If this is set to 1, change the MIME type on potentially dangerous messages (with the side effect of making such messages possibly difficult to read or extract attachments).

February 23, 2007

Systems Department