Slow copying using rsync on Mac OS X?

Thursday, 15. April 2010

Last night I’ve started the backup – pardon – rsync process – of our STORAGE volume to the newer BACKUP device. While both devices are connected via eSATA (Sonnet PCIe x1 eSATA card in the Mac Pro with Sil3132 chip), the copy process is what I would describe as very slow. A max. of 13MB/s from one eSATA drive to the other. Keep in mind, that both are RAID5 boxes with 4x 1TB HDs each! So I would suggest a higher speed.

After plugging the BACKUP device from eSATA to FW800 (thought it’s maybe a bus issue because PCIe x1 can only do 150MB/s), I started the rsync process again – with the same result: 13MB/s still. Ouch!

Next thing I tried was copying a file NOT with rsync but via drag&drop using the ordinary Finder. For my surprise, the speed to FW800 went up to 50-60MB/s. Still not what I had in mind, but a lot better than 13MB/s, isn’t it?

So I unplugged FW800 and attached eSATA gain. Now I’m copying the COMPLETE content using the Finder and pray for no interruption or any kind of mistake, because the Finder is so “dumb” and stops the complete process if this happens…

By the way – the max. peak after copying the first 40GB is about 73MB/s.

Ciao
Dennis 

[Update]
The Finder managed it to copy all stuff over the night :) Nice! Backup completed now rsync can do it’s slow job for syncing just changes. 



  1. warp42

    Tach! Maybe a problem with caching and syncing the filesystem? Using compression w/ rsync? Show us your script?



  2. Dennis Klein

    You’re right! I used the compression tag. But – even without that, copying an 8GB file results in max. 40MB/s peak.




  3. Dennis Klein

    Very interesting! Many thanks for highlighting this. :) So it sounds like it’s simply the “wrong” controller. Not that I would now say: Oh dear, what have I bought, I need another one. No – it’s not that bad ;) Maybe also interesting for you – my benchmarks on http://www.klein2.de/benchmarked-onnto-datatale-esata-port


  4. Hi, I realize this is an old thread, but wanted to point out two things:

    You should add the –cache option to rsync when doing copies on the local filesystem, I’m not sure when that was introduced by Apple, but it is there in 10.6 and 10.7. From the man page:
    –cache
    Apple specific option to enable filesystem caching of rsync file i/o
    Otherwise fcntl(F_NOCACHE) is used to limit memory growth.

    Being able to write at a high rate to the disk implies that the controller is working just fine. I have a MaxPower 6g PCIe esata card in a macrpo 1,1 connected to an external RAID5 enclosure. Using simple dd, I can write at up to 80MBytes/sec to it. Using rsync without –cache, I can only write at a peak of 25MBytes/sec. With –cache, the writes peak at around 50MBytes/sec.

    The bottleneck is of course within rsync itself and if you are worried about the total memory usage of the process, then you may want to leave off the caching.

    Cheers,
    Colby

leave a reply