How Email Can Get Horribly Lost

27.02.2014 Malcolm Newman

Email is today the most ubiquitous business communication tool available. Yet the infrastructure that enables email is often taken for granted and assumed by many to be reliable enough for critical communications. In reality, this couldn’t be further from the truth.

Email is stored first and foremost in files and folders on central server computers. These can be at your company office, on a Google, Yahoo, Hotmail (or 'telco') server, and for home-only users, often on just your home PC. If those "centrally" located" files are damaged in certain ways, then without backups (i.e. copies made when they were undamaged) some or all of that email may be lost forever. This can be a major problem for business, and more than just a nuisance for personal users.

However the servers aren't really a problem in this regard, because they are (generally) well managed, backed up by professional IT services, and secure from tampering or most natural disasters.

The BIG problem is the way you access the email. If you only use a browser (such as with Gmail or other popular online service) then this rarely causes a problem because it's just a window into the server.

But people and organisations mostly use much more sophisticated software such as Microsoft Outlook or Apple Mail (or IBM/Lotus Notes etc) which takes copies, performs virus and junk mail scanning, calendar and other office suite integrations, all in the name of making your life easier (which for the most part they do very well).

However, there is a dark side to these productivity tools. In order to achieve this, these smarter email client programs have a multitude of very private and sophisticated "conversations" behind your back with the servers. Sometimes, these conversations can end very badly, mostly due to unexpected interactions or just plain defects (known as "bugs") in the software design. The result of such errors is that the server can lose track of where some or even all of your email is, depending on just where, when and how the "conversation" went bad.

This problem has become much worse in recent times because of the explosion in popularity of mobile devices such as smart phones and tablet computers. There are two problems at play here:

  1. There are many more different types of smart email clients in use than before, not just a couple of (relatively) unchanging PC based clients. So the range of conversations with the servers (and consequently the opportunity for bad things to unexpectedly happen) has increased, and keeps increasing with every email application update (which is typically weekly or even more often).
  2. The sheer volume and rate of conversations is increasing exponentially. The way people now use email (compared to a PC that stays connected to the email server for a long time on a stable corporate network) is vastly different - using mobile devices, the email conversations are now started and finished far more often. This increases the exposure to problems even further, because a wider range of interactions is also happening, and the server "feels" them all.

The problem isn't going to go away soon. The best insurance is to have a strong and workable archiving and backup policy, so that when (not if) disaster strikes your email folders, recovery is only a few clicks (or swipes) away.