| View previous topic :: View next topic |
| Author |
Message |
jesnow
l33t
Joined: 26 Apr 2006
Posts: 856
|
 Posted: Sun Dec 19, 2021 9:08 pm Post subject: Samba performance questions |
 |
|
Hi everybody:
I'm trying to debug my home network setup. I have a linux samba 4.14 server (Core i7 860 w/rotating HD) and a client (AMD Ryzen 5 3600 w/NVME) both on a GBE. I feel like though my practical throughput is not what it could be. I'm *really* having throughput issues with my client at work (across the internet via ssh tunnel), but I want to be sure my local setup is good. Everything works well enough, it just seems slow. I have been googling all day but aside from turning up the MTU (which I did) found nothing interesting.
Disk performance is good, both for large and small files. On the Core i7 server Merckx:
| Code: |
jesnow@merckx ~/garb $ !469
rsync --info=progress2 -a bench.tar bencha.tar
335,220,402 100% 120.67MB/s 0:00:02 (xfr#1, to-chk=0/1)
jesnow@merckx ~/garb $ mkdir linux
jesnow@merckx ~/garb $ rsync --info=progress2 -a /usr/src/linux/ ./linux
1,237,882,653 99% 41.51MB/s 0:00:28 (xfr#74797, to-chk=0/80031)
|
Where bench.tar is just a tarred gzipped /usr/src/linux. Yes I should have used a different file extent. This looks OK to me, especially considering that /home is mounted on a rotating hard drive. /usr/src linux is on the SSD. Pretty respectable for a 10 year old machine!
On the Ryzen 5 client Bartali of course, local file tranfers are faster:
| Code: |
jesnow@bartali ~/garb $ rsync --info=progress2 -a bench.tar bencha.tar
335,220,402 100% 485.07MB/s 0:00:00 (xfr#1, to-chk=0/1)
jesnow@bartali ~/garb $ rsync --info=progress2 -a /usr/src/linux/ .
1,565,583,139 99% 135.44MB/s 0:00:11 (xfr#81222, to-chk=0/87065)
|
In this case both /home and /usr are on the NVME.
I'm a little surprised how much slower net transfers are though:
| Code: |
jesnow@bartali ~/garb $ rsync --info=progress2 -a bench.tar ~/merckx-jesnow/garb/bench.tar
335,220,402 100% 86.37MB/s 0:00:03 (xfr#1, to-chk=0/1)
jesnow@bartali ~/garb $ rsync --info=progress2 -a /usr/src/linux/ ~/merckx-jesnow/garb
16,854 0% 15.96MB/s 0:00:00 (xfr#3, ir-chk=1056/1060)rsync: [generator] symlink "/home/jesnow/merckx-jesnow/garb/Documentation/Changes" -> "process/changes.rst" failed: Operation not supported (95)
.
.
.
<a bunch of "Operation not supported (95)" errors, I think these are symlinks that don't copy well>
.
.
.
1,565,583,139 99% 4.59MB/s 0:05:25 (xfr#81222, to-chk=0/87065)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1330) [sender=3.2.3]
|
While 90MB/s is just about 700Mb/sec, and seems to almost be saturating the network, 5MB/s seems like a lot slower than it should be. Do the errors maybe cause wait states? It does correspond to Samba being a "chatty" protocol that doesn't handle a lot of small files well. Nothing on the server is saturating, not the disk, not the cpu, not the network.
Here is what mount shows:
| Code: |
bartali /home/jesnow/garb # mount | grep mnt
//merckx/jesnow on /mnt/merckx-jesnow type cifs (rw,relatime,vers=3.1.1,cache=strict,username=jesnow,uid=1001,forceuid,gid=100,forcegid,addr=192.168.1.105,file_mode=0755,dir_mode=0755,soft,nounix,mapposix,rsize=4194304,wsize=4194304,bsize=1048576,echo_interval=60,actimeo=1,_netdev)
|
So my questions:
- Is there a way to improve Samba's small-file behavior?
- Is this entirely unavoidable smb latency?
- Is there anything I can tune?
- Is a new server (with NVME) or a new file protocol (nfs?) going to help?
- Is a faster LAN, like 40GBE the solution? (I doubt this)
- What is the best performance people are getting?
Any insight would be most welcome.
Cheers,
Jon. |
|
| Back to top |
|
 |
eccerr0r
Watchman
Joined: 01 Jul 2004
Posts: 9679
Location: almost Mile High in the USA
|
| Posted: Mon Dec 20, 2021 6:04 pm Post subject: |
|
|
Small files will always suck, mainly because of the handoff between files where network latency starts consuming time. Me, if I knew I was going to deal with a ton of small files I'd tar then up and then transfer the tarball even on my home network. Because of this I rarely test small file performance, but on my NFS setup, large file performance I'm seeing 100MB/sec routinely on GbE (RAID5 on rust spinners to SSD, both ways).
On the other hand when I do computer to computer backup I use rsync. Rsync itself is terrible at small files and even when using rsyncd/sshd it's pretty bad... I still let it go, mainly for its diff/incremental copy routines. 100MB/sec for GbE is fast, but even platter speed is still faster (can get 150MB+/sec sequential reads on my RAID5).
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
| Back to top |
|
|
jesnow
l33t
Joined: 26 Apr 2006
Posts: 856
|
| Posted: Tue Dec 21, 2021 12:13 am Post subject: |
|
|
Yes anybody can tar up the files to move them, but that's not the point. I'm asking about performance as a network filesystem. NFS for example is said to be better with small files.
Cheers,
Jon. |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Networking & Security
