Nevertheless here are other options to considerĬopying without worrying about deleting old files. If you can run an rsync on your remote server (so you get a true client-server scenario across your network) you'll gain a significant amount of traction. Is there any measure I could take to avoid this long period prior to copying the files, other than excluding folders? Any kind of "cache" I can implement so that rsync does not have to rebuild all the file list from scratch?Īs far as rsync is concerned you're copying between two local file trees, so it disables most of its optimisations (including its delta algorithm for which it is famous). All in all, a 5 second file copy ends up lasting 20 minutes! It usually happens that rsync takes about 15m to finally find something it has to copy, then takes 5 seconds to copy it, then continues checking for some other files to copy for another 5 minutes. During this period, the newtwork utilization sits at a low 200-500KB/s, while when transferring files the speed is about 40MB/s. The problem is that rsync takes quite a long time (10-20m) before starting to move any files, I guess because it has to compute file lists for a very large number of small files. The disks contain many files, most of them small. I use the following rsync command: rsync -haAXi -quiet -append-verify -delete /mnt/ROUTER_WD_2TB/* /mnt/BACKUP_HITACHI_2TB/. The other, the destination, is mounted locally (peak 110MB/s) and is /mnt/BACKUP_HITACHI_2TB/. This is the source, with a speed of 30-40MB/s maximum due to network limitations. I use rsync 3.1.1 to keep in sync two discs, one of which is on the network and mounted as a samba share on /mnt/ROUTER_WD_2TB/.
0 Comments
Leave a Reply. |