Managing Server Diskspace Using IMAP
.... and how to placate your Web Hosts
The Source of the Problem
Auto-Archiving - Best Immediate Solution
Planning for Reducing Diskspace Usage
What To Do Now to Reduce Diskspace Usage
Create a Policy With Rules
This guide is designed to assist the website owner in staying within the diskspace limitation required by the hosting provider.
THE SOURCE OF THE PROBLEM
When setting up email accounts on your Outlook system you are faced with the decision between using a traditional POP (Post Office Protocol) system or the more flexible IMAP (Internet Message Access Protocol). IMAP allows for your current inbox and folders to synchronise between devices which means that office desktop, mobile cellphone and home laptop can all access the same set of data files. To do this, and be acessible from all devices, your downloaded email files and email data has to be stored on the server, and not at your operating locations.
The POP (Post Office Protocol) system is so named simply because it operates much like when you go fetch letter mail from the post office box (server) and take it home (to your desktop) leaving nothing behind. No mail builds up at the server unless you have a rule in place to not delete mail that has already been downloaded. But this is more often a misconfiguration error than a backup plan.
IMAP is a little more complicated to use but for users who are very mobile there are clear advantages in being able to work with your mail from anywhere at anytime and have access to all your current mail as well as your archived mail in folders, sent mail, etc.
Some IMAP users seem to believe that storage at a server (sometimes incorrectly called cloud storage) is a failproof data storage which is most certainly is not. This also means that they are exposing themselves to unexpected business risk as well as running foul of South Africa's new POPI Act regulations.
But there is an inherent problem, often not thought of at setup time, which will present later management issues.
These inbox, sentmail and stored folders will grow at an alarming rate. There is not an ISP around (unless you are prepared to pay heavily) who will allow you to grow those files beyond a certain point. Common limitations, for example, might be a maximum maximum of 10gB per domain and exceeding that would attract a penalty premium of R200 per month. A 20gB per domain limitation would usually mean moving your domain to a mail optimised server which would typically have a cost of R400 per month. If you wanted a 40gB allocation, then your mail premium would increase to R800 per month. These are the hard realities of server diskspace management and they are much the same from ISP to ISP. MWeb, for example, places an initial limit of 2gB per mailbox. Vodacom are 3Gb per user but there are variations between stand alone accounts and conditions for multiple users collectively under domain hosting accounts. Limitations are also placed on how long you can store data for - with 60 days and 3 months being common restraints that can be followed with automatic unconsulted mail deletion.
The bottom line for us is that domain hosting servers have finite diskspace limits which are designed to limit the processing load and file access speeds on the server. Exceed these limits and your website and mail processing performance will drop off. That effects user experience and is also heavily penalised by Google and other SEO evaluations. Your site relevance would suffer and web ranking slips. End result is that your traffic drops, business leads decrease and frustrations mount. None of these are desirable outcomes and cannot be allowed. If your mail storage is 100gB then that mail alone is using 20% of an entire server. The cost implication is easy to work out from there.
Extra costs for clients are rarely welcomed so one needs to consider a number of other solutions to avoid the accumulation of email data on the server and the storage problems that will inevitably follow.
AUTO-ARCHIVING - BEST IMMEDIATE SOLUTION
Emergency Action : Deletion of Old Mail
There are two quick solutions. If you look through the contents and decide that it is largely worthless to keep then we can delete old mail for you automatically. You need to let us know and choose the age limit - 2 weeks, 1 mth, 3 mths, 6 mths, 1 year or 2 years. We can delete that old mail in minutes.But you may be cautious and decide to keep old mail and understand it has to come off the server. For this we recommend enabling and configuring (for Outlook) the auto-archive facility. A detailed guide follows below....
There also the hybrid solution. You ask us to delete all mail over two years old which, to you, is not even worth archiving. Once that is done you put an auto-archive in place (say for all mail over 6 months old) and that is then saved automatically to your PC and removed automatically from the server.
Archiving IMAP eMail in MS Outlook
The simplest and quickest way, with MS Outlook, is to use the auto-archive facility.This will run at set intervals and will move mail from your desktop/laptop inbox and storage folders to the same structure under a new email user profile called Archive. As it moves the mails across it will similtaneously delete that mail from your online mailbox thereby reducing your diskspace usage. Problem solved.
Anytime you need the access stored email that is in the archive age (ie older than 6 months or whatever you choose) then you open the user "Archive" on your user profile list (at the left) and there is all the archived mail. From there you can use the mail as if you were in the standard user profile.
Practical Example on How to Set Up Auto-Archive
You decide that 1 year is a good enough period for mail to be accessed from all devices. Mail older than that must be moved to Archive where it can only be accessed from the users desktop machine. The norm used by most users is 6 months.Open the default mail user account to which the archiving will be applied.
- File > Options > Advanced
- Third item down > AutoArchive
- Select [ AutoArchive Settings ]
- Check the [ ] for "Run AutoArchive every [ 14 ] days.
- The remaining items will fill the dialogue box - leave the defaults "as is"
- "Clean out items older than" 6 months would be adjusted to 12 months.
- Select [ Ok ]
That's it. All is done. Your Outlook will now automatically check every 14 days and move mail older than 12 months and to "Archive" and will then automatically remove the emails from server diskspace usage.
We recommend that you set this up so that only a small amount of very old mail is initially archived. You will then be able to see how the Archive user gets created on your desktop and you will also see, when using that new profile, that your folders etc have also been migrated with the old data.
Imagine that your server email has old stored mail going back 18 months.
We suggest that you initially set the "Clean out items older than" value to 15 months. So it will initially only be mail older than 1 year, 3 months which is moved to your desktop Archive user. Once this has completed you can access via profile Archive and satisfy yourself that all has worked well.
Thereafter please set the value to 12 months, or less (preferably 6), and notify us that the archiving has been initiated. We will then check the server account and confirm back to you that all is good there.
The following page has reference.
- https://support.microsoft.com/en-us/office/archive-older-items-automatically-25f44f07-9b80-4107-841c-41dc38296667
To switch it off (not recommended) you would only need to go into your AutoArchive Settings and uncheck the tickbox alongside "Run AutoArchive every [ xx ]
iPhone mail, Apple Mail, Thunderbird and other mail clients all have facilities similar to the Outlook Auto-Archive to manage the same iIMAP server storage issues.
PLANNING FOR REDUCED DISKSPACE USAGE
- Use IMAP only where necessary
You can use a combination of POP accounts and IMAP accounts on our servers which support both protocols.
- Do you use any mailboxes for monitoring incoming mail ?
There are different reasons for this which vary from employee surveillance to backup and archive control where a copy of every incoming mail is automatically copied to an archive mailbox before delivery to the recipient.
If this method must be used then a quick solution would be to open a Gmail account (called, say, ArchiveYourCompany@gmail.com) and alias the archived mail to that mailbox. That would give you an initial 15gB free storage with very affordable extra storage when and if needed.
You could also, on an annual basis, copy that archive mail inbox back to a local ..pst file. More about that at the end.
- You can setup your storage folders to be on other devices.
For example, you could have a Google Cloud Storage account and when you move data from your inbox to a folder then that folder is not on your domain server, it is actually in your (true) cloud storage.
- Encourage your users to be lean on storage.
Does every incoming email have to be saved?
Can attachments not be saved to a local desktop folder instead of being left in a server folder.?
- Keep track of what your users are doing.
Monitor their usage by asking us for reports on their storage usage.
If you see a particular user becoming heavy on storage then you can analyse that user's setup before it becomes a problem. It also helps reinforce the focus for users by storing only what is necessary. POP users can also have incorrect setups. A common problem is where a POP client has been set to not delete mail which has already been downloaded (or "read"). This is usually done in error but it causes years of incoming mail, with attachments, to build up in the server new mail inbox.
- Understand that domain server mail storage is not infallible backup.
The POPI Act requires you to have a working backup (which could be interpreted as two backups) of your primary business communication records. That will certainly include your email records.
- Discuss this article and our suggestions with your internal PC support personnel.
Your own technical PC support people will usually have other suggestions and methodologies which will be helpful in achieving lower server storage use. They defininitely need to be consulted in this respect.
- Include this issue in your Risk Assessment strategy.
Your Risk & Compliance manager (sorry, POPI again) is required to address the subject of the email backup and security. That POPI requirement needs to be understood and by bringing data local it can be easier to manage and secure.
- Do you use any mailboxes for monitoring incoming mail ?
WHAT TO DO RIGHT NOW TO REDUCE YOUR DISKSPACE USAGE
In today's world it is not that easy for you to give us an instruction to delete all mail older than, say, two years. You can of course do that, but in many instances your client and supplier and other business communication is in your email archives. This needs to be retained for legal reasons and also for compliance with the dreaded POPI Act.
1. BITE THE BULLET AND PAY THE PREMIUM
This could be a preferred option where it is simply not efficient or convenient to reorganise. We prefer this to be a planned long-term decision as we have a hefty background task to move servers.
As an indication (per domain):
- 10gB to 20gB - R400 p/m premium
- 20gB to 30gB - R600 p/m premium
- 30gB to 40gB - R800 p/m premium
- over 40gB - R800 p/m + R150 per 10gB used excess premium
2. CONSIDER WHERE STORED MAIL CAN BE DELETED
Is there a monitoring box for backup purposes.? Perhaps mail in this box over 24 months old can be deleted. We can do this with running a simple script.
Are there redundant mailboxes? Employees that have left?
3. MOVE SELECTED STORED MAIL TO OTHER DEVICES
You can move entire folders from the server to a different storage devices.
We have provided a dedicated page which details the steps required to archive old mail from your server to an archive file on your local PC if you are using MS Outlook.
See - How to Archive Old Mail in MS Outlook
You just need to remember that once you move a folder from the server, say back to your desktop PC, then things will operate much as before with the exception that if you wanted to access that folder from another device, such as your smartphone, then that folder would not be visible. It is no longer on the server but on your desktop - and the phone can only see the server. So whatever you move off the server needs to be a folder that is not required to be visible from all devices. You may have a folder called "Artwork" which is huge. Create another called "Artwork - archive to 2021". The drag over all the older mail from one folder to the other. Now you have a folder to move to your desktop.
So decide how to restructure your folders and then decide where this archived data will be stored.
We will assume here that you want it local where your primary device (office desktop) can access the archived folders for searching, printing, replying, etc. Set that drive up if needed.
To create a personal folder on your desktop, on the Home tab, click New Items, More Items, and Outlook Data File. Choose where you want to save the file, type a name for it, and then click OK.
The Outlook Data File is then added to the folder list as a personal folder.
You can then use simple click and drag to move the contents of a server based IMAP folder to the local folder on your desktop PC.
The following Microsoft tutorial explains that in a video.
- Video: Create and use Personal Folders (Outlook Data Files)
CREATE A POLICY WITH RULES
We recommend that you formulate a policy along these lines and it is understood by all users and your PC support staff.
1. SELECT THE CORRECT PROTOCOL FOR EACH USER.
Use POP as the preferred mail protocol especially where you only access email and download mail at one location.
IMAP should only be used by mobile staff who need to access recent email from using different devices.
2. THE SERVER IS NOT THE ARCHIVE STORE.
It must be understood by all users that the mail server (ie. IMAP protocol) is not a permanent store for archiving email. When set at IMAP it enables you to read email using different devices and access the stored folders but those stored emails are always, one way or another, going to be limited by time.
We recommend that, in for users in IMAP mode, your domain (ie every user) is set to only keep read and stored mail on your server for a maximum of 6 months and thereafter it will be automatically deleted. On request we can set your server so that this is an automatic rule for everyone. It will NOT effect mail that has been downloaded to PCs.
Your PC, however, would continue to maintain an archive of all mail received. But when the server is accessed from another device then that access would only show email going back to the 6 month point.
|
Anything we left out, stuff you don't agree with.? Good article, bad article.? Please give us your comments and suggestions. |
