We get a number of calls from clients about emails. These often involve emails not being delivered, receiving unwanted emails, or outgoing emails not getting to recipients. Many of these issues can be traced back to how email filtering works.
Email filtering is incredibly useful. It is a vital layer of cyber security and also saves time, blocking unwanted spam emails landing in your inbox.
However, it is not perfect. Certainly, straight out of the box it can provide some frustration. The good news is modern email filtering problems are intuitive and highly customisable. With a little time and effort, you can get your email filtering working exactly how you want it to.
What is email filtering?
Email filters organize email according to specified criteria. Originally, filters were designed primarily to identify spam and block it or place it in the spam folder. Today, some mailbox providers use email filters to categorize messages for inbox organization purposes (e.g., Gmail categories or Microsoftโs Focused inbox).
Spam is annoying, no doubt, but it can also be dangerous. Malware and phishing are hugely profitable for scammers and can be costly for mailbox providersโ customers, as well as the mailbox providers who face intense market competition. Practically speaking, spam filters drastically reduce the load on server resources.
According to Spamlaws Spam accounts for 14.5 billion messages globally per day. In other words, spam makes up 45% of all emails. Some research companies estimate that spam email makes up an even greater portion of global emails, some 73% in fact. The United States is the number one generator of spam email, with Korea clocking in as the second largest contributor of unwanted email.
How Email Filtering Works
Email filtering works by using a set of protocols to determine which of your incoming messages are spam and which are not. There are several different types of spam filters available, but they all do basically the same thing. They scan the email header information for evidence of malice, look up senders on blacklists of known spammers and filter content for patterns that point to junk mail.
Deciphering Header Data
Header information is the text at the top of an email that you never have to see, and looks something like this:
Received: by 10.107.191.69 with SMTP id p66csp1537538iof
X-Received: by 10.107.175.218 with SMTP id p87mr2784731ioo.80.1477075567036
Fri, 28 Jul 2019 11:46:07 -0700 (PDT)
Contained within the text is important information. It shows things like the IP address of every server that touched the email, date and time stamps, security signatures and the like. The user doesnโt necessarily need to know any of this but is useful in understanding where that mail came from. Spam filters look for attempts to deceive the recipient (e.g., g00gle.com instead of google.com) and compare addresses to blacklists of known spammers to automatically filter out those that match.
Blacklists
Blacklists are lists of known spammers collected by internet service providers (ISPs), email providers and server administrators. The most popular ones, such as SpamCop, Spamhaus and URIBL, have the most credibility but anyone can create and publish a blacklist. The reason itโs important to label emails as spam is because these lists are created from those labels. By reporting spam youโre helping keep everybodyโs inbox clear.
The smarter spammers disguise header information and make their messages look genuine. However, not all spammers are particularly smart so header analysis catches much of the most obvious spam. Even those that are good at cloaking information may overlook some easy to spot details. If delivery reporting is disabled, for example, itโs a sign that the sender is transmitting a large volume of mail and doesnโt want to be bothered with bounce messages. Thatโs a possible spammer.
Content Filters
The clever part of how email filtering works comes into play when analysing the contents of a message. This is where the best filters shine, but itโs also where legitimate messages can end up incorrectly marked as spam.
Some content tactics are almost certain to land a message in the spam folder. Emails containing attached executable files or links to blacklisted websites are sure giveaways, as are the most common spam keywords. To avoid getting your own marketing emails blocked common spam keyword lists are available online.
If these things are so easily detectable you might wonder why spammers continue to use them. Unfortunately, there are enough gullible people out there that even a very low hit rate can be profitable. High-volume spammers donโt expect more than about a .1 percent open rate, but that still translates to 1,000 people for every 1 million messages sent.
Outgoing Mail
Whilst the main focus of how email filtering works is on blocking incoming spam, malware and phishing emails most will also filter outgoing emails.
When multiple users send email through one mail server, email receivers on the internet canโt trust anything other than the IP address of the mail server. This is because spammers can provide fake email addresses or even impersonate legitimate users.
However, an outbound spam filter installed within your own network can be programmed to identify individual users based on their authentication credentials. For instance, whether they entered the correct password to send mail through the server. By tracking individual users, the outbound spam filter can identify spam-like behaviour on a user-by-user basis, and prevent spam from leaking out of the mail serverโs IP address. This protects the mail server from becoming blacklisted and legitimate email being blocked.
Outbound spam filtering involves more than just analysing message content and rejecting the spam. A good outbound spam filter knows how to identify the actual sender of each message, and to record the long-term behaviour of each sender, looking for suspicious patterns of behaviour. A good outbound spam filter also takes great care not to make mistakes, because mistakes hurt your own users, rather than someone elseโs.
Identifying Poor Practice
Another way outgoing filtering can be used is to look at the activity of individual members of staff. Again, this might not be malicious activity but perhaps some naivety. We’d advocate giving staff training on what could constitute spam, but a little technological backup doesn’t go amiss.
You may not think this is necessary, especially in small businesses, as you’ll believe your employees can be trusted not to send out spam or sensitive material. You might even think of it as spying. However, it can be remarkably easy to end up inadvertently sending spam and getting your IP address blacklisted.
For example an email with a misspelled title or the overuse of spam-related keywords or characters (โ!โ or โ$โ for example) could be identified as spam by an over-zealous spam filter. If enough over-zealous spam filters report your IP address as a source of spam, it could be added to a global blacklist.
The global blacklist is shared between all major spam filtering service providers. This means that all your business emails would be caught by the recipientsยด email filters, and only delivered once your organisations name had been added to a whitelist โ or removed from the global blacklist.
Whilst removing your IP address from a global blacklist is not difficult it can certainly be time-consuming. Whilst your IP address is blacklisted, the flow of communication between your business and your customers are delayed โ potentially harming profitability.
The Perfect Email Filterโฆโฆ
Doesnโt exist! Although no spam filtering solution is 100% effective, a business email system without spam filtering is virtually unusable. Youโll probably need to put some effort into tailoring the solution exactly the way you want it. Scanning spam reports and white listing your contacts and legitimate emails, whilst blocking and reporting anything nasty that slips helps the filter โlearnโ and provides you with a better experience over time.
We provide email filtering as standard within our fully supported IT service. The cost of email filtering is included in your quotation, with nothing extra to pay. The email filtering system we provide is highly customisable, allowing users to decide report frequency, add and remove content to be blocked and white-list contacts.
If youโd like to find out how Your IT can help you with all elements of your IT please complete the contact form or call us on 0115 8220200.