Are Your E-Mails Getting Lost?

Anti-spam issues are becoming critical and there are many spam-blocking packages in use, effectively blocking junk mail up to 95% and more. But an important side effect of the anti-spam measures has now started showing up. Even genuine e-mails can be bounced or deleted.

Are your e-mails getting lost in cyberspace? I have seen this happen thrice
in last few months. In the first instance, I stopped receiving the fortnightly
newsletter from a leading chain of bookstores when they outsourced the job to a
bulk mailing service. Then I stopped receiving e-mails from a client after he
registered his domain name with an e-mail hosting service and switched to an
e-mail ID with his own domain name instead of the VSNL ID. A few weeks later I
stopped receiving e-mails from another client who had changed the e-mail hosting
service provider. 

Note two important things here, which are common to all three instances of
lost e-mails. Firstly, I use the usa.net messaging service for my e-mail, which
has a very strong anti-virus and Spam-blocking policy. Secondly, all three cases
happened after the senders of e-mails had switched to a different e-mail service
provider. This prompted me to investigate a little deeper into the problem of
lost e-mails. What I found was something unexpected. The root of the problem was
common to all the three cases - improperly implemented mail servers. Unexpected,
because two of the mail hosting service providers were well known companies! The
bulk mailing service provider turned out to be a typical Spammer, with a
falsified server identity.

WHAT SMTP IS

All e-mail communication takes place on the basis of the Simple Mail
Transfer Protocol (SMTP). The SMTP servers on the sender and receiver side must
follow this protocol to successfully communicate with each other. The SMTP
design is based on the model of communication, which goes like this-

When a user wants to send mail to a recipient, a mail request is generated;
the sender-SMTP establishes a two-way transmission channel to a receiver-SMTP.
The receiver-SMTP may be either the ultimate destination or an intermediate
server, which will relay the mail further. To be able to provide the relay
capability, the SMTP-server must be supplied with the name of the ultimate
destination host as well as the destination mailbox name.

At the time the transmission channel is opened, there is an exchange of
commands and responses to ensure that the hosts are communicating with the hosts
they think they are. The HELO command is used while opening the channel and the
QUIT command is used in closing the channel. In the HELO command, the host
sending the command identifies itself; the command may be interpreted in plain
English as saying "Hello, I am ". A sender-SMTP
supporting SMTP Service Extensions starts an SMTP session by sending the EHLO
command instead of the HELO command.

HELO OR EHLO

In order to achieve maximum interoperability, the Extended SMTP clients
using EHLO must be coded to check for server connection closure after EHLO is
sent, either before or after returning a reply. If this happens, the client must
decide if the operation can be successfully completed without using any SMTP
extensions. If it can, then a new connection can be opened after sending a RSET
(Reset Command) and then the HELO command. The HELO or EHLO command and an OK
reply to it confirm that both the sender-SMTP and the receiver-SMTP are ready to
start the mail transaction.

If your SMTP server is not properly implemented, but the receiver-SMTP server
on the other side is implemented to follow SMTP standard strictly and to block
Spam, your mails can get lost in cyberspace or get bounced. The mail transaction
can get terminated right at the start with the HELO or EHLO command itself.

Here
are a few examples of the HELO and EHLO commands and their results:

Received: from 210.18. 136.31 (HELO yahoo.com) (210.18.136.31) - Originating
IP address does not match with the HELO identity. Detected as falsified
identity. Mail gets diverted to Bulk Mail folder or gets deleted.

Received: from 66.218.93. 171(HELOweb42003.mail. yahoo.com) (66.218.93.171) -
Originating IP address matches with the HELO identity. Mail gets delivered to
Inbox.

Received: from 202.144. 65.72 (EHLO Tetraspace) (202.144.65.72) - Valid IP
address, but missing ".com", mail gets bounced with a message
"Remote host said: 501 Invalid host name Tetraspace". Some receiver-SMTP
servers can receive this e-mail if they ignore the '.com'. Yahoo! ignores
this but usa.net doesn't. 

If a service provider detects a large percentage of invalid e-mail coming
from one IP, the mail stream may be identified as spam and blocked. Due to all
the mischief from Spammers, some genuine mail also gets blocked or bounced

if the sender-SMTP is not properly implemented. Reason for improper
implementation of SMTP servers is not that SMTP is some kind of Rocket Science.
It is just plain carelessness. Some service providers correct their mistakes
when they are noticed; others just carry on, because many receiver-SMTP servers
also do not follow the standard very strictly.

Ashok Dongre is an independent
consultant

CHECKLIST FOR MISSING MAILS 

1. Find the reason for the bounce from the bounced message coming from
the Mailer-Daemon.

2. If the mails are bouncing from one address (usa.net) but not from some
other address (yahoo.com), check the e-mail headers to find if there is
something wrong with the HELO command.

3. If the mails are getting lost, disable the Spam-blocking feature
temporarily and see the result.

4. Check if the IP address of the outgoing SMTP server is blacklisted as
a Spam server.

5. Take up the matter with the e-mail hosting service provider. Ignoring
the problem may result in a lost business opportunity some day.