Methods to Combat SPAM

"Spam" is the term for junkmail, i.e. emails that are sent en masse to a huge number of recipients. They are as unwanted as advertising leaflets in your mailbox, and since it costs virtually nothing to send them, they have become a major Internet problem. Spam and Virus Protecting the mailservers Protecting the Recipients Prudent Behaviour Is this a losing battle? Limiting the choice of mailserver Regarding SPAM-filtering To Bounce or not to Bounce How about paying for wanted mails? Spam over Cellular Phones?

1. Spam and Virus One need, first of all, to differ between spam on the one hand, and malicious software on the other. Malicious software are small programs, such as "viruses", "worms" and "trojan horses", which are often distributed by way of spam mail. Spam can (and certainly does!) be generated by these software categories, but we will limit this discussion to just "spam". Malicious software is a broad subject that should be treated separately. "Spam" is the accepted term for mass-distributed junkmail, and, as demonstrated by the "SobigF"-virus attack, it is not only annoying and time-consuming to deal with, it can also bring down mailservers and thus block email-traffic! 2. Protecting the mailservers Spam can cause mailservers to be: Clogged by an avalanch of incoming mail, insofar as they cannot handle the mail at the rate they arrive. Filled in excess of their storage capacity. Each of these is sufficient to bring the server down. The first problem has the side-effect that messages to the mailserver get queued, and if the queues become too long, timers in the sending computers will interrupt the proceedings after about one minute or so. If the sender is another mailserver, it will keep trying for at least another 24 hours. If the sender is your mail handler, you will most likely get an error message, to the effect that "the server is not available". A good remedy for this is difficult to implement. Many mailservers have filters that either reject or delete messages from certain other servers, but in order to do this, each incoming message has to be examined regarding its source address, which still takes time. The only way to go is to increase the communication handling capacity of the mailserver, and this is usually done by installing the anti-SPAM-program on a separate computer, with lots of RAM, which receive all incoming mail ahead of the mailserver. The second problem is easier. There are today spam filters that can do a lot of things, such as: Deleting mail from certain sender addresses. Deleting mail containing certain words. Deleting mail containing certain headers. Deleting certain attached files. These methods are very efficient; they keep the servers going and also help the recipients to deal efficiently with their junk mail. On top of this, mailservers can help each other by filtering and not forwarding mails that fill certain set criterias for being spam.

3. Protecting the Recipients An unfortunate side effect of the tide of spam is that many people delete received messages that they suspect of being spam without reading them. What they look at are: The sender address The subject line As most people have discovered by now, the sender address is not to be relied upon as a criteria for deleting a received mail, because: The address in the "From"-field can easily be faked by the sender, since this field is manually edited in his mail program. And it is usually faked by spammers. The sender can have an infected computer, which means that he/she is sending junk or virus without knowing it. The subject line is more telling, because you generally know what your friends or aquaintances usually write there. Spam and virus-generated junk usually have standard headings that you, your anti-spam-program and/or the filtering program in your mailserver recognizes. So; a good and relevant subject line is pretty vital to prevent your e-mail being inadvertently dismissed as spam by some readers. It is even more important when you consider the increasing use of rule-based e-mail filters that use very unforgiving software to classify incoming messages as spam or not-spam. Thus, be sure to avoid certain subject lines. For instance, don't use money in the subject line; e.g. "Buy this for only $50". Many of your readers will have spam filters than kill off anything with a dollar sign in the subject line. So, to make your life as a mail recipient easier: Use an ISP (= Internet Service Provider) that have efficient and updated spam filters. This service could be worth paying extra for. Use a good anti-spam-program that can be easily programmed to automatically bounce and/or delete unwanted mail (but only after you have seen the mail listing and approved of this action for each individual mail. Bouncing mails (i.e. returning them to the sending address) might not be a good idea, however, because: The sender (spammer) will thus see that you exist, and double his efforts to spam you. The sender might be an innocent aquaintance with an infected computer. He/she is not aware that his/hers computer is sending these mails. The senderīs purported address might be faked, which means that the mail will be returned to you (again) by the receiving mailserver, as "undeliverable." It is usually better to leave the "bouncing" to the mailservers. They can make it look like your address doesnīt exist. Read more in chapter 8 on this page about this.

4. Prudent Behaviour By keeping a low profile, and not brandishing your email address around, you can, to some extent, avoid spammers. But most of us want to be visible on the web, usually in the line of business. In many instances you will want to put your email address on your website, even if you provide a form as a contact method. This is useful because it increases the number of ways that someone can contact you. Why is that important? Because presumably if you have a website which you want to show the world - and also to communicate over - and this communication goes both ways. If you have a commercial site, then the answer should be obvious - someone may want to purchase something. So, of naturally your website needs to contain email addresses. But, of course; the problem with directly including your email addresses on your webpage is spam harvesters. What they do is search through the internet, looking at web sites and pages for email addresses to add to those million email address collections that you see advertised (mostly in spam) all over the web, usually for a price. There is no ironclad way to prevent these programs from scanning your web site for email addresses. Changing your email address every few months is counter-productive, as you will discover when your friends and business acquaintances time and again have to update their address registers. You will be regarded as an annoying fellow. There are a number of techniques, however, to make it a little more difficult for the spam harvesters. The best of these is to code your email addresses in something called Unicode. Other methods use "enail address cloaking", where the address is hidden, using JavaScript, encryption and the like. But; as the address harvesting programs used by spammers get increasingly smarter, they will decode most of these. There are, however, small scripts like Privacy Notes Spambot Buster, which use a roundabout way over a server to get at the email address, and this method has so far been successful. When no mail address is available, direct or encoded, it canīt be harvested.

Many people contend that spammers and virus-producers are ruining the usefulness of Internet in general and email in particular. We will be so wary of virus and tired of spam that we will not use the great facility that email is, to its fullest extent. But this attitude is wrong, for two reasons; The counter-measures noted above will be gradually more efficient and more common, as time goes by. And the email-user who takes the trouble to learn how email and spam function can do a great deal to rid his mailbox of spam.

Law enforcement in this area is on the rise. More and more countries makes laws against spammers, and against Internet Service Providers that do not take apprpriate counter-measures. Thus; spamming is a criminal offence in U.K. since December 2003, fining spammers up to 5000 pounds. Italy in 2003 passed a similar, though tougher, anti-spam law. In Italy, a spammer can be fined 90,000 Euros and sentenced to three years in prison. Spammers can of course operate from mailservers in countries that do not have these kind of laws. But then, such mailservers could easily be blacklisted by other mailservers, making them more or less inoperable.

Newsflash, 11 May 2004: CANADA urges international cooperation to fight spam Canadian officials has suggested that international efforts, possibly including a treaty, are necessary to fight the growing problem of spam. Lucienne Robillard, Canada's Industry Minister, said, "Alone, country by country, we cannot solve this problem," noting that 95% of all spam received by Canadians originates in other countries. According to Robillard, an international treaty on spam could include extradition of those accused of sending spam. Richard Simpson, director general of e-commerce for Industry Canada, compared a potential international agreement on spam to existing tax treaties, which countries use in collecting taxes and "countering other forms of activities like money laundering." A spam treaty is also being discussed at the Asia Pacific Economic Cooperation forum.

One method of identifying spammers, which has been implemented on most ISPs (= Internet Service Providers) by now, is to allow access to the SMTP-service only over the mailserver which belongs to the same ISP and modem pool which the sending client is connected to. It functions as shown in the example to the right. Letīs assume that the sending client is connected, by way of a modem and the telephone network (fixed or mobile phone network, it does not matter), to the modem pool belonging to ISP1. He can still fetch his mail from his own mailserver, no matter where on the Internet it is. But when he tries to send email, he might get into difficulties, if he has not configured his mail client program correctly. Outgoing mail, as well as incoming mail, has to go through a mailserver, but not through any mailbox, so the sender wonīt need username or password for that; he will just use the SMTP-protocol for his sending. But the ISP he is connected to will (likely) not permit him to use any other mailserver for his outgoing mail than the one in ISP1. If he tries to use the mailserver in ISP2, he will get an error message. Likewise, if he is connected to the modem pool in ISP2 and tries to use the mailserver in ISP1 he will get an error message. In this way, no matter that he tries to fake his sending address, the receiving mailserver will know that he is likely a subscriber at ISP1, and thus the sysop at the receiving end can address a letter of complaint to (in this case) ISP1. There is an exception to this rule. If the sender knows that the receiver has his mailbox at ISP3, he can in most instances access this mailserver (at ISP3) directly. But that wonīt be good enough for a spammer.

7. Regarding SPAM-filtering

Todayīs spam-filters can be very sophisticated, with a host of options. There are two kinds. Those that are meant for installation on mailservers. The most advanced have a myriad options, and should preferably be installed on a separate computer, between the Internet and the mailserver. They are called "mail agents". Those that are meant for installation on client computers. They are meant to make initial contacts with the mailservers, and check received mail, before the regular mail program does the same thing, i.e. contact the mailservers. It is interesting to note that most filtering is done at the receiving end. If mailservers were to do more filtering before sending, the total traffic load on the Internet could be considerably reduced. Spam-filters that work together with mailservers basically do two things: They can check attached files in incoming mailes for virus. If found, the attachments are removed, and a message is inserted in the mail, to the effect that "An attachment was removed, please contact the sender for re-transmission by other means".

They can examine the mail content and produce a pretty exhaustive report; which can look like the example to the right. It will be accompanied by an inserted message, reading something like this: "This mail is probably spam. The original message has been attached along with this report, so you can recognize or block similar unwanted mail in future. See http://spamassassin.org/tag/ for more details." One can naturally wonder; is not this an invasion of your mail privacy? Well, it probably is, but if youīre not a crook, you will probably conclude that it is worth it. If you are a crook, you will probably be encrypting your email-messages anyway. Handling of Email-attachments Regarding deletion of mail-attachments; many e-mail-users have their mail programs set to automatically process executable attached files of various kinds, often without knowing it. This creates an inroad for virus, which we are not dealing with on this page. But there is thus three good reasons for server-based anti-spam programs to remove some of these attachments: To eliminate the risk of virus-infection in recipientsī computer. To reduce download time when fetching mail. To free storage space on the mailserver. Usually, files whose names end with ".exe", ".com", ".bat", ".pif" and ".scr" are eliminated. So, if the sender wants to get around this, he/she can always ZIP the file to be attached to a mail. A zipped file is harmless until the recipient, voluntarily, does something with it after unzipping (there are of course other methods of compressing files than zipping them).

Depending on the anti-SPAM-software installed at your mailserver, and on how it is configured, you could find one or more of the following message lines in your received mails. Content analysis details: (20.10 points, 5 required) FROM_NUM_AT_WEBMAIL (2.9 points) From address is webmail, but starts with a number BANG_EXERCISE (3.1 points) BODY: Talks about exercise with an exclamation! MONEY_BACK (0.7 points) BODY: Money back guarantee HTML_FONT_COLOR_RED (0.1 points) BODY: HTML font color is red HTML_80_90 (0.5 points) BODY: Message is 80% to 90% HTML HTML_MESSAGE (0.1 points) BODY: HTML included in message HTML_FONT_BIG (0.2 points) BODY: FONT Size +2 and up or 3 and up MIME_LONG_LINE_QP (0.3 points) RAW: Quoted-printable line longer than 76 characters MIME_HTML_NO_CHARSET (0.6 points) RAW: Message text in HTML without specified charset REMOVE_PAGE (0.3 points) URI: URL of page called "remove" MAILTO_TO_SPAM_ADDR (0.6 points) URI: Includes a link to a likely spammer email address HTTP_EXCESSIVE_ESCAPES (1.1 points) URI: Completely unnecessary %-escapes inside a URL MISSING_MIMEOLE (0.5 points) Message has X-MSMail-Priority, but no X-MimeOLE FORGED_MUA_AOL (4.3 points) Forged mail pretending to be from AOL MIME_HTML_ONLY (0.1 points) Message only has text/html MIME parts UPPERCASE_25_50 (1.5 points) message body is 25-50% uppercase.

The figure above illustrates what we have been talking about. Letīs assume that we run the mailserver on Unix, and use a script procedure called "procmail". Then: 1) The arriving email gets picked up by the spam filter, which is usually installed on a separate computer. The spam filter does three things. Deletes "offending" attachments Assigns spam points to the email Add lines of text to the email to reflect what has been done.	2) The mail is handed over to the mailserver, which sort it into the appropriate mailbox. 3) If there are scriptfiles for this recipient, these are activated. The script called "forward" directs the mail to be processed by the script mentioned above. This script can sort the mail into thre categories: To be deleted To be saved a number of days, then deleted To be stored in the mailbox.	4) The recipient checks his mail, using his anti-spam program. 5) Using this program, he can instruct his mailserver to bounce and/or delete some mails. 6) After this process, he fetches remaining emails using his regular email handler. 7) The recipient can, now and then, check and delete temporarily, spam-marked emails on his mailserver. Although this might sound tedious, the recipient is only involved in steps 4 to 7, and, if he uses good scripts on the server, he can very well skip steps 4, 5 and 7, and just use his mail handler in the same way as he did in the good old days, before spam. The "flow" of this process on the mailserver is illustrated in the figure at left.

	8. To Bounce or not to Bounce

Anti-spam programs offer an opportunity to bounce received junk mail, in addition to deleting them. It is of course tempting to do just that, but one has to consider: The sending address (in the "From:" field) is usually faked. Some innocent guy will get your bounce, not understanding why. And he might bounce it right back to you. The sending address might be non-existent, in which case the receiving mailserver will just send it back to you. Often spam mails are accompanied by attachments carrying virus. By bouncing, you will just help spread the virus. This bouncing will only generate more useless traffic on the Internet. Now; the purpose of bouncing might not be to "get back" at the spammer, but rather to give the spammer the impression that the recipient address does not exist, and thus scratch that address off his list. Beacause thatīs what recieving mailservers do when they receive a mail to a non-existent address; they bounce it back to the sender, usually adding an explanation.

Does this scheme work? Not really. First off, the spammer has seen to it that the bounced mail, whether from a mailserver or from you, does not reach him, because he has faked the sending address as described above. Second, he doesnīt care to read, or even check the returned mail. Understandably, he would never have time to do anything like that. He will just delete all received mail en masse. But, the curious person might want to know: if the spammer scrutinizes the header, could he then see if the bounced mail got bounced because the address was really non-existent, or because it bounced by a recipient? Yes, he could, if he would really go to that trouble (which he wonīt, if he is a reasonably(??) sane person. I performed a simple test, and sent two mails from mailserver A to mailserver B, one mail with a faked and one with a real address. Both addresses purportedly belonged to the domain "swedetrack". The mail with the faked address got bounced by mailserver A (and not the destination-server B), the one with the real address a bounced myself, back to mailserver A (Since A was put as SMTP-server in my mail program). Both bounces were then fetched att receiving mailserver C, and compared.

The two mails have, of course, travelled over different routes, but there are more telling differences. 1) The faked-address mail that got bounced had the "From"-field: "Internet Mail Delivery, postmaster (etc.)mailserver A" with the Subject line: "Delivery notification: Delivery has failed". 2) The real-address mail bounced by me had the "From"-field: "Mailer-daemon@swedetrack.com", with the Subject line: "Returned mail: User unknown". So; clue number one: the real-address mail (not the bounce, but the mail itself!) reached its destination mailserver. It shouldnīt had done that if the address was non-existent! Clue number two: The faked-addressed bounce contains the text "Your message cannot be delivered to the following recipients: (etc.). Reason: Remote SMTP server has rejected address Diagnostic code: smtp;550 (etc.) User unknown in virtual alias table. That revealing info does not appear in the bounce that I performed myself.

The figure at right illustrates what I have described in the text above. 1) Both mails get sent to server A. 2) The mail with existing mail address (blue) is forwarded to destination server B. 3) Server B sends it to the recipient, who bounces it back (red). 4) The bounced mail gets forwarded to its destination mailserver (C), as calculated from the address in the mailīs "from"-field. 5) The manually bounced mail gets fetched by the sender. 6) The mail with the non-existing mail address (violet) is forwarded to the senderīs mailserver (C), as calculated from the address in the mailīs "from"-field. 7) The mail bounced by mailserver A gets fetched by the sender.

9. How About Paying for Wanted Mails?

Instead of blocking unwanted messages, two California-based companies are offering services aimed at limiting spam by ensuring delivery of wanted messages, despite increasingly stringent Internet Service Provider (ISP) filtering programs. "IronPort" offers clients "bonded" e-mail, for which clients attest that their messages are directed at recipients who want to receive them.

"IronPort" works with ISPs to ensure delivery of those messages, but if unwanted mail is delivered, the client is liable for costs, even if complaints are not proven. Habeas offers a service, free to private users and licensed to businesses for up to $5,000 per month for bulk e-mailings, that embeds Haiku into messages. There is embedded text that identifies those messages as valid to those ISPs who agree to the service.

Analysts question whether these services that guarantee legitimate e-mail and challenge the future of free e-mail will succeed, and whether they will adequately address the problem of vast and increasing volumes of unsolicited e-mail. But; itīs up to everyone to pay for what he considers worth having.

10. SPAM over Cellular Phones?

It is so easy to be enthusiastic over technological advances that one tends to forget that there are always people who are prepared to use the new technology for personal gain, regardless that they destroy the usability for other people in the process. When supermarkets replaced small shops, this radically reduced prices, until shoplifters became such a problem that security measures had to be taken. The cost for that, and for all pilfering, has to be paid for by the honest customers. E-mail over the Internet is a tremendous saver of time and money but, as has been seen, this is now being largerly offset by the time-consuming task of scanning and deleting junk mail.

What about the emerging broadband wireless technology? GSM-phones are now gradually being replaced by the enhanced 3G-phones. When will spammers discover how easy it will be to deliver voice messages to your telephone answering machines? It will take about one minute of listening, before you realize that this is not a new potential customer, but spam. 30 such messages waiting on your answering machine, and you will have wasted half an hour. It will be awfully difficult to design spam filters that can separate junk messages from those that you want to hear. The sending address can be faked. Voice, wording and intonation of a message are those criteria that an anti-spam filter would have to check, there are no other useful criteria.

Already today we have seen the emergence of junk-SMS-messages, that occupy the narrow bandwith allotted to SMS. They also needlessly occupy the message buffers of the cellphones, so that they have to be checked and deleted pretty regurlarly, so that there will be buffer space for genuine SMS-messages. One can object that sending such messages will cost money to the spammers, in difference to IP-based spam over Internet, which is virtually free. That may be true, but then one forgets about VoIP, Voice over IP, which allows anyone to use the Internet IP-protocol to send voice messages. It wonīt be free of charge to use the 3G technology, but cheap enough to tempt spammers.