| View previous topic :: View next topic |
| Author |
Message |
kernelOfTruth
Watchman
Joined: 20 Dec 2005
Posts: 6111
Location: Vienna, Austria; Germany; hello world :)
|
 Posted: Sun Oct 20, 2013 6:47 pm Post subject: rsync 3.1.0 **very slow** during backup (even new files) |
 |
|
Hi,
after the upgrade to rsync 3.1.0 (which supposedly integrates several data-integrity related improvements from previous versions)
backup jobs are really slow:
e.g. when backing up my recoll (desktop search) directory
the xapiandb subdirectory takes several minutes or up to half an hour and more to copy (even when deleting the already existing old folder with contents on the receiver backup drive)
| Quote: | xapiandb/flintlock
0 100% 0.00kB/s 0:00:00 (xfr#12, to-chk=16/31)
xapiandb/iamchert
28 100% 0.06kB/s 0:00:00 (xfr#13, to-chk=15/31)
xapiandb/position.DB
10,171,236,352 100% 53.26MB/s 0:03:02 (xfr#14, to-chk=14/31)
xapiandb/position.baseA
155,223 100% 520.91kB/s 0:00:00 (xfr#15, to-chk=13/31)
xapiandb/position.baseB
155,223 100% 462.15kB/s 0:00:00 (xfr#16, to-chk=12/31)
xapiandb/postlist.DB
876,380,160 27% 4.13MB/s 879,001,600 27% 3.18MB/s |
postlist.DB is around 9.5 GiB large so this is pretty annoying 
can this be sped up or is it Working as intended™ ?
Similar experience on your part ?
Many thanks in advance 
edit:
/home is on btrfs filesystem
btrfs to btrfs
and
btrfs to ext4
shows this behavior
haven't tested yet with other filesystem types ...
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.9.0
https://github.com/kernelOfTruth/pulseaudio-equalizer-ladspa
Hardcore Gentoo Linux user since 2004  |
|
| Back to top |
|
 |
mv
Watchman
Joined: 20 Apr 2005
Posts: 6747
|
| Posted: Sun Oct 20, 2013 7:31 pm Post subject: |
|
|
| What are your options? For instance, are you using --inplace? (you should!) |
|
| Back to top |
|
|
kernelOfTruth
Watchman
Joined: 20 Dec 2005
Posts: 6111
Location: Vienna, Austria; Germany; hello world :)
|
| Posted: Sun Oct 20, 2013 7:42 pm Post subject: |
|
|
| mv wrote: | | What are your options? For instance, are you using --inplace? (you should!) |
aha !
wasn't aware I had forgotten to add those options
but:
| Quote: | WARNING: you should not use this option to update files that are
being accessed by others, so be careful when choosing to use
this for a copy.
This option is useful for transferring large files with
block-based changes or appended data, and also on systems that
are disk bound, not network bound. It can also help keep a
copy-on-write filesystem snapshot from diverging the entire con-
tents of a file that only has minor changes. |
thing is: oftentimes when backing up I'm using firefox or chromium so those directories / files might be inconsistent or incomplete
so I either have to live with slow speed or inconsistent files, and thus avoid working with the computer while backing up - hm ...
I'll give that option a try and see how it changes behavior
thanks mv !
more info:
http://www.manpagez.com/man/1/rsync/
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.9.0
https://github.com/kernelOfTruth/pulseaudio-equalizer-ladspa
Hardcore Gentoo Linux user since 2004 |
|
| Back to top |
|
|
mv
Watchman
Joined: 20 Apr 2005
Posts: 6747
|
| Posted: Mon Oct 21, 2013 7:12 am Post subject: |
|
|
| kernelOfTruth wrote: | | oftentimes when backing up I'm using firefox or chromium so those directories / files might be inconsistent or incomplete |
Not using --inplace will not save you from this problem either. If you want to avoid such problems you have to use the snapshot function of the filesystem (if available), but even then the consistency depends on the running application - this is why good databases take a considerable effort in performing operations "atomically". |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Other Things Gentoo
