Betalogue

Mac OS X Server still has a way to go before it can claim to be a truly user-friendly server OS, i.e. a server OS that can be managed by an administrator without requiring knowledge of arcane Unix commands. Here’s an example I ran into yesterday.

I use Panther Server’s Mail service on my Xserve. For the most part, things can be configured using the graphic interface provided by the Server Admin application, which I can even use remotely from my home office over a pokey modem connection (although it’s a bit slow for my tastes, it works).

So far I’ve only created a handful of POP accounts, so that the transition and the learning process for the users and for me (the administrator) are not too abrupt. There’s one person other than myself who already uses her account on a regular basis.

Yesterday, she phoned me to tell me that she wasn’t receiving any new messages, and that the messages she was trying to send out from her computer were apparently not getting delivered. She had no idea why.

I launched Server Admin and looked at the SMTP log, which indeed indicated that all commands related to her account were getting “deferred”. My account, on the other hand, was working fine, so it wasn’t a problem with the Mail service itself, but a problem with her account specifically.

I asked what she had been using the account for in the past 24 hours and she said that she had been receiving a number of pictures as attachments. She knew about the 5 MB limit on her mailbox, but was making sure that people weren’t sending her attachments that were too big, and was checking her e-mail regularly in order to remove the mail from the server by downloading it in Mail on her client machine.

I went to the “Accounts” tab in Server Admin and, sure enough, the number in “Disk Space Used” was over 5 MB, with “-6%” in the “% Free” column. In other words, her mailbox was full.

I asked again and she assured me that she was downloading the messages as they were coming and thereby removing them from the server.

That’s when things got a bit tricky. As far as I know, there is no built-in interface in Server Admin or any of the other server administration tools to examine the contents of a user’s account. In other words, you have to use the command-line interface for this.

It was my first time doing this and I myself was learning at the same time as I was conversing with my user, trying to determine exactly what had happened.

Before trying the Unix route, I launched Workgroup Manager and changed her space quota from 5 MB to 10 MB, vaguely hoping that it would magically solve the problem. It didn’t. (And, unfortunately, Workgroup Manager is much slower than Server Admin to use over a modem connection, so the whole process of exploring this issue was rather painful.) Even though I saved the changes in Workgroup Manager and refreshed the display in Server Admin, the “Disk Quota” for the user in Server Admin still said “5 MB” and things were still stuck.

I put on my (flimsy) Unix boots and bravely logged in remotely using SSH using my administrator’s account. Of course, it put me in my own user folder by default, but I was able to change directories and move back up to the top of the folder hierarchy on my server.

Server Admin, in the “Accounts” pane, indicates the exact location of the user’s mailbox on the server, which is /var/spool/imap/user/[username]. Once I was at the root level on the server, I then tried to get into the /var/ directory, but was told unceremoniously that I didn’t have the permission for this. So I had to use the “su” command to give me root access. After that, I was able to change directories freely and soon managed to get inside the [username] directory for my user’s account.

After taking a quick look at the man page for the “ls” command, I used ls -l to get a list of the directory’s contents that included the file size for each item. Now, you need to realize that I knew nothing about how the mail server stored mail messages and what you could do with the files. However, a quick look at the listing gave me some clues:

total 10952
-rw-------  1 cyrus  mail   987277 17 Mar 13:31 218.
-rw-------  1 cyrus  mail  1368803 17 Mar 13:37 219.
-rw-------  1 cyrus  mail  1524146 17 Mar 13:45 220.
-rw-------  1 cyrus  mail     2179 17 Mar 13:52 221.
-rw-------  1 cyrus  mail  1694662 17 Mar 14:12 222.
-rw-------  1 cyrus  mail     4584 18 Mar 09:11 cyrus.cache
-rw-------  1 cyrus  mail      161 16 Feb 22:03 cyrus.header
-rw-------  1 cyrus  mail      336 18 Mar 11:19 cyrus.index

Clearly the files whose name was a number followed by a period were individual message files (including their attachments) and the last three files were standard files present for each account.

I used the “pico” tool to take a look inside one of the files whose name was a number, and it was indeed a message, with a large photo included as an encoded attachment.

But when I read the date and sender to my user, she said that she had received these messages. In fact they were the last ones she had received before things went wrong. Why were there still on the server then?

That’s when I asked her to go to her preferences in Mail and check the preferences for this particular account. Sure enough, under “Advanced”, the “Remove copy from server” option was set to “After one day”. Now the picture was getting clearer: She had received a slew of e-mails with attachments in a short period of time and, while none of these e-mails exceeded the limit, the total of them did, and since they had all arrived in less than a day (less than an hour, in fact), even though my user had been checking her e-mail regularly, these large messages had not been removed from the server and were now clogging her mailbox and causing it to exceed the disk quota.

She assured me that she never changed that setting, and I believe her, because I seem to remember that, the first time I created a new POP account in Mail, that was the default setting.

How much sense does this make? Most POP account users are told to assume that, once they have downloaded their messages from the POP server, these messages are removed from the server, and that is indeed what my user had assumed. But this assumption was wrong, because of a default setting in Mail that foolishly was not what would have been expected.

I made her change the setting to “Right away”. But that didn’t help unclog her mailbox, since she had already downloaded the messages. I had to figure out how to remove the messages from the server.

Fortunately, the intuitive thing that I tried was the correct one. I just used the “rm” command and removed the files whose name was a number in the user’s directory. I was afraid that this wouldn’t work right because there was probably a reference to these files in the “cyrus.index” file and I didn’t know how to change that. But it took care of things by itself, and things started working again for my user. I sent her a test message, and she got it right away.

However, based on the SMTP log that I had referred to earlier on, there were still a number of messages that people had sent her whose delivery had been “deferred” and that were currently lost in limbo. These messages didn’t arrive in her mailbox right away. But I assumed that the SMTP server would continue to try delivering the messages at regular intervals and told her to wait a bit. Sure enough, within half an hour, the messages had been delivered. And everything was now back to normal.

The lesson here is two-fold. First of all, it is really silly of Apple to use default settings in the Mail application that don’t make sense. It might be an additional “safety” measure to configure accounts by default so that the messages stay on the server for one day after the user has downloaded them — but clearly most users are not even aware that this is possible, and, as my example indicates, such a default setting is much more likely to create problems than to help the user.

The second lesson is about Panther Server. Not only did I have to resort to the command-line interface to get out of this mess (and I am obviously learning through experimentation in that department), but even for the user things were unintuitive. Why didn’t she get a notification from the server immediately that she had exceeded her disk quota? This would have helped a lot. Why didn’t she get an explanation when she tried sending out messages? I can understand why the people who tried to send her messages didn’t get a bounce right away (the server is probably supposed to continue to try to deliver the messages for a few days before bouncing them back), but the user herself should have been notified of the problem, instead of just seeing her own sent messages and the messages that people were trying to send her disappear in limbo. And I, as an administrator, should probably have received some kind of notification that a user was exceeding her disk quota and that things were stuck.

Now, some of these things might be configurable through the command-line interface — but there’s certainly no mention of it in the graphic interface provided by the server administration tools. And the default behaviour certainly doesn’t help: Things just stop working, without any explanation. Even the log messages simply say that the delivery has been “deferred”, without saying why.

Clearly, there’s a lot of room for improvement here. On the other hand, I’ve learned something else, and I will now be better equipped to handle such situations in the future (although I’ll be carefuly to configure everyone’s Mail application so that it removes messages from the server right away!). With the increase in the number of people with high-speed Internet, more and more users are sending out large attachments without realizing that they are way too large for a recipient who only has a modem connection, and that most people’s accounts have a maximum disk quota that cannot be exceeded. So this type of situation will probably occur again in the future, even if my users’ Mail application is configured properly.

This entry was posted by Pierre Igot on Friday, March 19th, 2004 at 12:35 am and is filed under Macintosh. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.