rsync does not preserve permissions [SOLVED]

[halcon]

Hello.

The issue seems to be easy to resolve, but I can't get what is the trick.

I launch it as:

Code: Select all

/usr/bin/rsync -avhRz --delete --delete-excluded --exclude-from="/some/path/to/exclude/list" --rsync-path="sudo rsync" -e "ssh -i /home/myuser/.ssh/mykey" myuser@myipaddress:/some/path/at/source /some/path/at/destination 2>&1 | tee -a /some/path/to/log

from user "myuser" (non-root, but in group wheel)

Other details:
- both machines run Gentoo
- both machines have user "root", user "myuser", and many other users coincide
- the destination folder resides on a partition EXT4, mounted just with parameters rw,relatime - nothing mask
- the source machine has an entry in visudo myuser myhostname=(root) NOPASSWD: /usr/bin/rsync

And ALL the files and folders at the destination have permissions myuser:myuser, including those that are of root.

If I remove "2>&1", no error appears.

Why is it so?

[szatox]

That's one complicated command, is there really no way to simplify it with basic syntax?

Anyway, you're missing -p
As long as all files belong to your user and groups you are a member of (on the destination machine), it should do the trick. Anything that does not belong to you will fail, because unprivileged users can't cede ownership.

[Hu]

The shown command has -a, which per man rsync, is short for -rlptgoD, so -p is implicit.

halcon: does this work properly if root on the local machine syncs to root on the remote machine, thereby removing all use of sudo and --rsync-path from the process? Similarly, I note that OP says that the source user has sudo permission, but is silent on whether the destination user has it. Presumably, sudo should have printed an error message and exited failure if not, and OP says there are no errors shown.

[halcon]

That's one complicated command, is there really no way to simplify it with basic syntax?

Do you have suggestions?

Anyway, you're missing -p

AFAIK, I am not. The option -a includes it. From man rsync:

Code: Select all

--archive, -a            archive mode is -rlptgoD (no -A,-X,-U,-N,-H)

As long as all files belong to your user and groups you are a member of (on the destination machine), it should do the trick. Anything that does not belong to you will fail, because unprivileged users can't cede ownership.

Do you want to say that I should rsync as (destination) root?

[halcon]

halcon: does this work properly if root on the local machine syncs to root on the remote machine, thereby removing all use of sudo and --rsync-path from the process?

So far, I don't know. I didn't open SSH access for root on the source machine for security reasons.

Similarly, I note that OP says that the source user has sudo permission, but is silent on whether the destination user has it. Presumably, sudo should have printed an error message and exited failure if not, and OP says there are no errors shown.

I don't have any sudo permissions for "muyser" on the destination machine. So, should I?

[halcon]

I have added the same visudo record for myuser at the destination machine (only hostname changed), launched a new terminal (graphical) for it, repeated the same rsync command, and nothing changed. I am not sure, maybe I have to update some environment.

[Hu]

Normally, all users have outbound access unrestricted, so there is nothing to do on the source machine to let root run ssh destination-ip. Did you try that and observe it to fail locally?

If you don't have sudo on the remote machine, why would you tell the local rsync to run sudo rsync on the remote machine? That will just fail, although it should have failed in a way that produced an obvious error.

[szatox]

Right, -a includes -p, my bad.
Well, if you want to preserve permissions, running as root is the natural choice. However, you might also try --super to either get better results of or an error (hopefully) telling you what's wrong. BTW, a --fake-super option exists too, and it exploits extended attributes to store metadata it may not be allowed to write directly.

[halcon]

If you don't have sudo on the remote machine, why would you tell the local rsync to run sudo rsync on the remote machine? That will just fail, although it should have failed in a way that produced an obvious error.

On the remote machine I had it from the beginning. The remote is source. The destination is local. I am "pulling": launching rsync at my local machine for making a backup of a server. The local launcher initially was just a user "myser" in the wheel group. And now I've added a sudo record for him too - nothing changed.

Normally, all users have outbound access unrestricted, so there is nothing to do on the source machine to let root run ssh destination-ip. Did you try that and observe it to fail locally?

Outbound, if launching from the same server? Like "pushing" a backup? And I do "pulling" ^^ Or maybe I don't understand something Also, the destination-ip would be my home machine with a dynamic IP... Yes, I can use VPN, but...

[halcon]

However, you might also try --super to either get better results of or an error (hopefully) telling you what's wrong.

Aha! With the option --super I've got a bunch of errors

Code: Select all

rsync: [receiver] chown "/some/file" failed: Operation not permitted (1)

Now I have where to dig further...

[szatox]

chown failed, so is that file really yours?

Wait...

And ALL the files and folders at the destination have permissions myuser:myuser, including those that are of root.

It wsn't clear the the first glance, but do I get it right that you knew in advance that some files you're trying to copy don't belong to your user?
This is not going to work without root privileges. You can make a backup with --fake-super at the destination (and then restore with --fake-super at the source and --super at destination), but you can't make an exact copy directly without root.

[halcon]

chown failed, so is that file really yours?

Wait...

And ALL the files and folders at the destination have permissions myuser:myuser, including those that are of root.

It wsn't clear the the first glance, but do I get it right that you knew in advance that some files you're trying to copy don't belong to your user?
This is not going to work without root privileges.

Yeah, evidently, I failed to describe it correctly. I meant "And ALL the files and folders, including those root:root, become after rsyncing myuser:myuser". Is it correct now?

EDIT
But it really worked. I got all these root files, they just didn't preserve root permissions.

EDIT2
On the server:

Code: Select all

# LC_ALL=C ls -al /root/.bash_history
-rw------- 1 root root 26014 Feb 27 19:15 /root/.bash_history

On the local machine:

Code: Select all

# LC_ALL=C ls -al .bash_history 
-rw------- 1 myuser myuser 23336 Feb 27 09:06 .bash_history

[szatox]

Yeah, you're running the remote rsync via sudo, so with root access. Of course this part works.
You still can't write permissions locally as a regular user. This part doesn't go through sudo in your command, and you said you're running it as a regular user. Wheel has nothing to do with it unless you actually use that membership to run rsync via su.

If you want to have 2 usable copies, you have to run both sides a root.

[Anon-E-moose]

If I am not mistaken you can't write root permissions (owner, etc) as a user. (not even rsync can do it)

[halcon]

If I am not mistaken you can't write root permissions (owner, etc) as a user. (not even rsync can do it)

Right, rsync can't. I tried adding sudo for rsync, and for chown.

If you want to have 2 usable copies, you have to run both sides a root.

I just copied my ~/.ssh to /root, re-run rsync as root, and it is OK now
So, not necessarily both root. Local launching root + remote user with sudo = success.

Marking as solved.

Thank you all !