More on automated backups with Tiger using rsync

Posted by Pierre Igot in: Macintosh
August 18th, 2005 • 10:01 am

A couple of weeks ago, I mentioned an excellent article by Richard Hough at O’Reilly describing a simple method to set up automated backups in Tiger without having to purchase any third-party software.

It turns out that the article doesn’t mention a small, yet significant problem with the built-in rsync command that comes with Tiger. The rsync command that’s included in Tiger has a bug that causes it to change the creation and modification dates of the backed up files. Instead of the original file’s creation date, the backed up file has a creation date that is actually the modification date of the original file. And the modification date of the backed up file is… the time of the backup itself.

Needless to say, this can turn out to be a significant problem if you ever need to restore your files from the backup. I decided that it wasn’t acceptable and tried to figure out what to do about it.

I e-mailed Richard Hough, the author of the article, and he very kindly provided me with some pointers. The first one was to install the open source RsyncX utility.

Installing RsyncX does two things: It installs a backup utility with a graphic interface (called RsyncX) in your startup volume’s “Utilities” folder, and it installs a new rsync command in the Unix structure of your Tiger system, which can be used via the Terminal.

I had a quick look at the RsyncX utility, and wasn’t too particularly impressed. In spite of its brushed-metal looks, the interface is far from intuitive, and simply attempting to do a backup of my home folder resulted in a cryptic “Error -2” that I didn’t bother trying to understand.

On the other hand, the new rsync command that RsyncX installs is actually a better version that doesn’t suffer from the bug with the creation/modification dates that affects the original rsync included with Tiger.

The important thing to note here is that the command installed by RsyncX is also called rsync, and that it replaces the original rsync command that comes with Tiger. (The original rsync command is actually kept, but with a new name.) And this new rsync command actually has a syntax that is slightly different, so you cannot just use Richard Hough’s script commands without altering them.

For example, where the original rsync command would use a syntax such as this one:

rsync -aE --delete ~ /Volumes/Files/FilesBackup 2>>~/rsyncErr.txt || echo -n

to back up the home folder to the “/Volumes/Files/FilesBackup” destination, the new rsync command installed by RsyncX needs to use the following syntax:

rsync -a --eahfs --delete ~ /Volumes/Files/FilesBackup 2>>~/rsyncErr.txt || echo -n

(Note the -a -eahfs options after the rsync command itself.)

After installing RsyncX, I tried this new syntax in the Terminal, and it failed, with the following error message:

rsync: --eahfs: unknown option

I immediately suspected that the new rsync Unix command might not have been recognized by Tiger following a simple application install, that the Terminal was still using the old rsync command (for which the “--eahfs” was obviously an unknown option), and that I probably should restart my machine. I did that, and after that the command with the new syntax worked fine in the Terminal.

However, when I tried to incorporate the command with the new syntax in the AppleScript script described by Richard Hough, I discovered that it didn’t work, and that the “rsyncErr.txt” error log also had the same error message:

rsync: --eahfs: unknown option

In other words, it looked like, while the Terminal was able to use the new rsync command installed by RsyncX, the AppleScript script was still using the old rsync command, even after a system restart.

So I sent another e-mail to Richard Hough, and again, he very kindly provided me with the solution, which was to include the full path to the new rsync command in the AppleScript script.

In order to determine the full path to the new rsync command, I had to type:

which rsync

in the Terminal, which gave me:

/usr/local/bin/rsync

I then replaced:

rsync -a --eahfs --delete

with

/usr/local/bin/rsync -a --eahfs --delete

in every line of my AppleScript script and, sure enough, after that change the script ran just fine.

The only trouble with this whole approach is, of course, that you do have to install a third-party utility after all. It’s a free utility, so there’s no expense involved — but it still modifies your Tiger system in a way that will not be preserved after a system reinstall, and might be affected by future Mac OS X system updates. (Apple might decide to fix the bug in the original rsync command one of these days, and to incorporate a new version of the command in one of its system updates, which would likely replace the command installed by RsyncX.)

Still, it remains a 100%-free way to set up an automated backup system in Tiger, and it works perfectly well after the above adjustments are made.

Thanks again to Richard Hough for his help.


16 Responses to “More on automated backups with Tiger using rsync”

  1. Jussi says:

    The rsync command that’s included in Tiger has a bug that causes it to change the creation and modification dates of the backed up files

    Interesting. I’m not seeing this with my backup (which I do with rsync) at least with simple usage of ls or finder. How are you making the dates visible?

  2. Pierre Igot says:

    I am seeing this with the built-in rsync command when using it this way:

    rsync -aE --delete ~ /Volumes/Files/FilesBackup 2>>~/rsyncErr.txt || echo -n

    (as an example). Since the author of the O’Reilly confirmed the problem, I am guessing I am not the only one :). Besides, Apple has since responded to my bug report saying that it was a duplicate of a known issue.

    The wrong dates can be seen in the Finder with the Get Information window or in the Preview column in the Finder.

  3. Jussi says:

    I’m using the builtin rsync like this:


    rsync --perms --group --owner --delete --archive --extended-attributes ~ /path/to/backup/Users/

    and according to ls and finder the modification dates are same in both my homedir and backup volume (for items I have not changed after my backup, previously done on 15th of August) For example:


    $ ls -l /Volumes/Backup/BACKUP/Agnus/Users/hagi/gui_osx.c_tmp
    -rw-r--r-- 1 hagi hagi 140163 Jun 16 04:32 /Volumes/Backup/BACKUP/Agnus/Users/hagi/gui_osx.c_tmp
    pB:~/webkit hagi
    $ ls -l ../gui_osx.c_tmp
    -rw-r--r-- 1 hagi hagi 140163 Jun 16 04:32 ../gui_osx.c_tmp

    But sure I’ve not tested it very well, it could be that some of the swiches I’m using accidentally triggers the correct behaviour. :)

  4. Pierre Igot says:

    Looks like it might be the “extended-attributes” bit. See the discussion on the O’Reilly article for more details.

  5. Jussi says:

    The flag –extended-attributes is same as -E so it is not the cause. But it’s nice to know there are alternatives to the version provided by Apple.

    Looking at the manpage it actually seems that some of the flags I’m using are duplicates that are already set by –archive (-a). Well, perhaps I had better not to touch my command so that I won’t be bitten by the bug.

  6. Pierre Igot says:

    According to the feedback on the O’Reilly page, the bug is a direct result of the combination of -E and -a, so it sounds like you should be experiencing the bug. I can definitely tell you that I experienced it myself with the -aE options

  7. Jussi says:

    OK. I read the O’Reilly page’s comments properly and I tested it myself with a small directory. I can duplicate the behaviour when -a and -E are specified and the copied file has a resource fork. Most of my files do not have it so I did not see this problem earlier. Files without resource fork seem to be transferred perfectly.

  8. Pierre Igot says:

    Thanks for all the testing. Sounds like you’ve narrowed down the problem. Using the built-in rsync command that comes with Tiger with -a and -E on files that include a resource fork results in backed up files with the wrong creation/modification date.

    It’s a pretty big bug, unless you have no files with resource forks. Many files from “legacy” Mac applications still use them. For example, BBEdit still uses the resource fork to store important information about its text files.

    If you are unsure whether your files to backup contain resource forks, your best bet is to replace the built-in rsync command with the one that comes with RsyncX, and modify the syntax of your rsync commands accordingly.

    Thanks for your help, Jussi!

  9. bbushman says:

    Pierre:
    Great information- thanks! Could I bother you for some quick insight into running these rsync commands with an applescript? I keep getting the following error when I try to run them: “can’t make current application into type string”. my event log in the script editor shows that it’s trying to run it within a “current application” tell block, even though per the apple article I read, I tried to keep it out of one. I’m a total Applescript newbie, so I’m sure I’m doing something foolish here… any thoughts?

    more info in my comments at Richard’s original article:
    http://www.macdevcenter.com/pub/a/mac/2005/07/22/backup.html?page=2

  10. Pierre Igot says:

    bbushman: I am no AppleScript expert myself. In what format did you save your script? I saved mine in AppleScript with the file format “script” (with extension “.scpt“), and then selected that script from within the iCal event to run it every night.

    My script definitely has no “tell” command. It just contains a series of “do shell script” commands, each one with a different rsync command for a specific folder.

    Does this help at all?

  11. bbushman says:

    Pierre:
    Got it to work by putting it within another function:
    ——-
    set buildCart to “TheLoraxG4PB-HD:Users:bbushman:DEV:brandsourcedev:website:trunk:src:cart”
    set siteCart to “TheLoraxG4PB-HD:Library:WebServer:Documents:brandsourcedev”

    set mySrc to “/Volumes/TheLoraxG4PB-HD” & the POSIX path of buildCart
    set myTgt to “/Volumes/TheLoraxG4PB-HD” & the POSIX path of siteCart

    on MakeCopy()
    return do shell script “/usr/local/bin/rsync -a –eahfs ” & mySrc & myTgt & ” 2>>~/rsyncErr.txt || echo -n”
    end makeCopy
    ———————–
    HOWEVER: now the rsync portion isn’t working. It makes my error file, so it’s running it in some capacity, but nothing is getting copied. Thoughts?

  12. Pierre Igot says:

    What’s the error mentioned in the error file? Are you sure your rsync command is at the specified location (type “which rsync” in Terminal to check)?

    Also, do you really need to have the extra AppleScript code? Can you not just type out the full path of the source and the target in the do shell script command itself and have an AppleScript script with a single line of code? It would make a very long line of code, but that doesn’t really matter.

  13. bbushman says:

    Pierre:
    yeah, I checked which rsync, that part is correct.

    I started with all one big script and then did the extra applescript to get it to work. if I just run this:

    do shell script
    “rsync -a –eahfs –delete /Volumes/TheLoraxG4PB-HD/Users/bbushman/DEV/brandsourcedev/website/trunk/src/cart /Volumes/TheLoraxG4PB-HD/Library/WebServer/Documents/brandsourcedev 2>>~/rsyncErr.txt || echo -n”

    I get this applescript error, whether I run as scpt file from the menu, or from the “run” command in script editor, or as a .app file:
    “can’t make current apllication into type string.”

    As far as my rsync error file: it’s blank. :)

  14. bbushman says:

    Pierre:
    Thank you for your thoughts and info… I think I got it working! Removed the function crap, but went back to my concatonated shell script, attaching a “set result to do shell script…” And now no applescript error.

    THEN: I was getting real errors in my RSync error file, which showed that I’m missing some spaces in my concatonation. Just fixed those, and VOILA! It copies, and it syncs.

    Thanks again…

  15. Pierre Igot says:

    I see. And you say that this very same rsync command works just fine when you try it directly in Terminal, right?

    If so, I am afraid I don’t really have any further suggestions… except maybe to try experimenting with folders with shorter paths and see if the command works then within an Apple Script script. For example, I would try to put a “test” folder in your home folder and then try a script with:

    do shell script “rsync -a –eahfs –delete /Volumes/TheLoraxG4PB-HD/Users/bbushman/test /Volumes/TheLoraxG4PB-HD/Library/WebServer/Documents 2>>~/rsyncErr.txt || echo -n”

    and see if that works. And then take it from there. There must be something in the folder paths themselves that is causing the problem.

  16. Creation Robot » iPodBackup - Automatic backups to an iPod on Mac OS X says:

    […] A solution that others have found: http://www.macdevcenter.com/pub/a/mac/2005/07/22/backup.html which has evolved into the following: http://www.betalogue.com/2005/08/18/more-on-automated-backups-with-tiger-usin g-rsync/ […]

Leave a Reply

Comments are closed.