Bootable 64-bit RPi3 Gentoo image (OpenRC/Xfce/VC4) UPDATED

[Sakaki]

emerge --sync and then eix --update?

Yes, if you haven't run an update for a while... or just use eix-sync as a shorthand (if not using webrsync-gpg).

If eix is still out of alignment, you can force a full db refresh with "eix-sync -0" (zero) (this doesn't hit the network for updates, just re-reads everything you have locally).

[roylongbottom]

Sakaki

Per my earlier post, I have now gone ahead and created an RPi3 optimized binary kernel package

Thanks, after I have run all my benchmarks, I will try the new kernel

PS have you been able to determine whether the RPi3 B+ has any better thermal stability than the 3B, in light of its heat spreader?

I carried out a quick test via Raspbian, running four programs carrying out integer calculations. For original RPi 3 and 3B are examples of one of the four logs (that are essentially the same), with CPU MHz and temperature included. The original might have an inefficient heatsink but the 3B+ without one, in a plastic case.

The 3B+ switched to 1200 MHz, within the first 80 seconds but remained there for more than 6 minutes. After 80 seconds the original intermittently ran at 600 MHz.

Code: Select all

 Original RPi 3

 Integer Stress Test Linux/ARM A7 v1.0 Mon Jul 11 19:58:19 2016

   40 KBytes Cache or RAM Space, 80 Seconds Per Test, 12 Tests
                                                                    MHz    °C

 Write/Read                                                        1200   62.8
  1    2472 MB/sec  Pattern 00000000 Result OK    2413986 passes    735   84.9
  2    1853 MB/sec  Pattern FFFFFFFF Result OK    1809189 passes    600   83.3
  3    1792 MB/sec  Pattern A5A5A5A5 Result OK    1749631 passes    600   83.3
  4    1770 MB/sec  Pattern 55555555 Result OK    1728301 passes    600   83.3
  5    1733 MB/sec  Pattern 33333333 Result OK    1692058 passes    891   83.8
  6    1745 MB/sec  Pattern F0F0F0F0 Result OK    1704346 passes    864   84.4
 Read
  1    1788 MB/sec  Pattern 00000000 Result OK    3491600 passes    714   85.4
  2    1723 MB/sec  Pattern FFFFFFFF Result OK    3366300 passes    600   84.9
  3    1670 MB/sec  Pattern A5A5A5A5 Result OK    3261500 passes    600   83.3
  4    1661 MB/sec  Pattern 55555555 Result OK    3244700 passes    600   83.8
  5    1662 MB/sec  Pattern 33333333 Result OK    3246100 passes    744   85.4
  6    1647 MB/sec  Pattern F0F0F0F0 Result OK    3217300 passes    600   83.8

  Stand Alone
  Write/Read
  1    3099 MB/sec
 Read
  1    3220 MB/sec

 ###############################################################################

 RPi 3B+

  Integer Stress Test Linux/ARM A7 v1.0 Thu Mar 22 18:58:18 2018

   40 KBytes Cache or RAM Space, 80 Seconds Per Test, 12 Tests
                                                                    MHz    °C
 Write/Read                                                        1400
  1    3399 MB/sec  Pattern 00000000 Result OK    3319566 passes   1200   70.9
  2    3169 MB/sec  Pattern FFFFFFFF Result OK    3094903 passes   1200   75.2
  3    3169 MB/sec  Pattern A5A5A5A5 Result OK    3094857 passes   1200   77.4
  4    3170 MB/sec  Pattern 55555555 Result OK    3095448 passes   1200   79.5
  5    3151 MB/sec  Pattern 33333333 Result OK    3077499 passes   1195   80.6
  6    3051 MB/sec  Pattern F0F0F0F0 Result OK    2979893 passes   1087   80.6
 Read
  1    3140 MB/sec  Pattern 00000000 Result OK    6133000 passes   1087   80.6
  2    3049 MB/sec  Pattern FFFFFFFF Result OK    5955700 passes   1141   81.1
  3    2881 MB/sec  Pattern A5A5A5A5 Result OK    5626600 passes   1034   81.7
  4    2796 MB/sec  Pattern 55555555 Result OK    5461900 passes   1034   81.7
  5    2786 MB/sec  Pattern 33333333 Result OK    5441200 passes   1034   81.7
  6    2759 MB/sec  Pattern F0F0F0F0 Result OK    5389200 passes    980   82.2

  Stand Alone
  Write/Read
  1    3760 MB/sec
 Read
  1    3740 MB/sec

[Sakaki]

roylongbottom,

1200 not 1400 MHz peak for the RPi3B+; is that what is expected?

[roylongbottom]

Sakaki

1200 not 1400 MHz peak for the RPi3B+; is that what is expected?

In my case, the MHz reduced from 1400 to 1200 MHz after two recorded samples or 32.3 seconds, when temperature of 69.8'C was indicated. Claimed expected behaviour is provided in a graph down the page in:

https://www.raspberrypi.org/blog/raspbe ... le-now-35/

Here, MHz reduces to 1200 at 70'C. In my case, temperature would not have increased as much if I used a decent heatsink or even less in my Flirc case (if the built-in heatsingk still makes contact with the CPU).

[Sakaki]

Hello,

if you are running a 'pure' 64-bit Gentoo system on your RPi3 B / B+ (such as my gentoo-on-rpi3-64bit image), you may sometimes find yourself wishing you could just install and run a particular 32-bit package from Raspbian, if it has not yet been added to the Gentoo arm64 tree.

Well, the good news is you can, while still running your 64-bit Gentoo, since mixed-mode userland is permitted on ARMv8! I've just posted a new wiki tutorial, showing how to install a 32-bit Raspbian chroot on your gentoo-on-rpi3-64bit system, then, using this, install 32-bit apps (using apt-get) from Raspbian, and run them alongside 64-bit Gentoo apps on your (64-bit) desktop. The example app I use in the tutorial is Lazarus (but you can obviously install anything you like).

[roylongbottom]

Raspberry Pi 3B+ Memory Benchmarks

Full details of 32 bit and 64 bit memory benchmarks (and single core tests) are available at ResearchGate in Raspberry Pi 3B+ 32 Bit and 64 Bit Benchmarks and Stress Tests.pdf, from the following link (then click on down arrow to select download):

https://www.researchgate.net/publicatio ... ress_Tests

This includes 3B+ comparisons with the older Mode 3B and 64 bit versus 32 bit performance. The latter is repeated below for the newer processor (3B similar). The 3B+/3B performance is essentially proportional to respective CPU MHz speeds, where date from caches is processed, but 3B+ is often shown to be slightly slower with RAM data transfers. The benchmarks are as follows, most doubling up data size used, to cover caches, and RAM, with performance measured in MegaBytes per second. Example full results and comparisons are provided below.

MemSpeed - carries out the calculations shown in the following, the first being of the same format as the Linpack benchmark time dependent function. Maximum MFLOPS are also shown for these, plus MFLOPS/MHz ratios, these being higher that those for Linpack, mainly due to the smoother data flow and slightly using L1 cache based results. Best 64 bit performance gains were using double precision floating point but one result indicates that the older RPi was faster using RAM based data.

Code: Select all

            Memory Reading Speed Test vfpv4 32 Bit Version 1

  Memory   x[m]=x[m]+s*y[m] Int+   x[m]=x[m]+y[m]         x[m]=y[m]
  KBytes    Dble   Sngl  Int32   Dble   Sngl  Int32   Dble   Sngl  Int32
    Used    MB/S   MB/S   MB/S   MB/S   MB/S   MB/S   MB/S   MB/S   MB/S
                                                                          3B+/3B
           Raspberry Pi 3B+ CPU 1400 MHz, SDRAM ?                        Avg Gain

       8    1899   2125   4041   2783   2624   4448   3164   3693   3693  1.17 L1
      16    1901   2128   4058   2791   2628   4462   3177   3703   3707
      32    1852   2049   3817   2686   2508   4161   3186   3715   3711
      64    1796   1959   3574   2542   2367   3855   2945   3347   3347  1.16 L2
     128    1826   1989   3741   2600   2408   4031   3042   3506   3508
     256    1833   1995   3771   2617   2414   4068   2860   3616   3617
     512    1517   1618   2587   2039   1911   2687   2459   2825   2832
    1024     968   1098   1221   1172   1140   1211   1455   1144   1137  0.98 RAM 
    2048     911    980   1060   1038   1026   1062   1013    941    935
    4096     913    993   1064   1047   1038    948    992    902    903
    8192     926   1013   1077   1074   1065   1085    782    784    783

 Max MFLOPS  238    532
    Per MHz 0.17   0.38
    64 bit  0.43   0.52

 #################### Compare 64 bit / 32 bit Pi 3B+ ######################

       8    2.54   1.36   1.08   2.22   1.51   1.09   1.70   1.17   1.17
     256    2.12   1.39   1.05   1.86   1.53   1.06   1.71   1.13   1.13
    8192    0.71   1.19   1.17   1.14   1.03   1.17   1.29   1.38   1.38

 #######################################################################

NeonSpeed - executes the same functions as MemSpeed, but with all floating point calculations using single precision floating point (for compatibility with NEON). Some normal calculations are also included for comparison purposes. The NEON calculations are carried out using NEON Intrinsic Functions but the latest compilers convert these into more appropriate vector instructions. This leads to little difference between 32 bit and 64 bit speed, the former being faster in one case. For some reason, 32 bit normal calculations were faster than in MemSpeed, but maximum NEON MFLOPS per MHz were significantly faster.

Code: Select all

 NEON SP Float & Integer Benchmark RPi 3B+ 64 Bit

  Memory  Float v=v+s*v  Int v=v+v+s   Neon v=v+v    3B/3B+
  KBytes   Norm   Neon   Norm   Neon  Float    Int  Avg Gain

      16   2724   5109   3961   4841   5446   5607  1.16 L1
      32   2612   4645   3726   4450   4968   5036 
      64   2523   4247   3540   4150   4521   4519  1.16 L2
     128   2583   4363   3666   4253   4616   4635
     256   2576   4314   3674   4254   4591   4631
     512   1852   2871   2608   2466   2916   2698
    1024   1222   1207   1305   1179   1280   1216  1.08 RAM
    4096   1157   1144   1214   1109   1181   1160
   16384   1175   1245   1244   1134   1191   1180
   65536   1143   1258   1185    909   1144   1260

Max MFLOPS  681   1277
  Per MHz  0.49   0.91
  32 Bit   0.57   0.84

 #################### Compare 64 bit / 32 bit Pi 3B+ ######################

      16   0.86   1.10   0.99   0.99   1.05   1.02
     256   0.88   1.07   0.98   1.01   1.06   1.00
   65536   0.85   0.94   0.88   0.90   0.91   0.93

 #######################################################################
 

BusSpeed - is designed to identify reading data in bursts over buses and possible maximum data transfer speed from RAM (using 1 core - see MP version). The program starts by reading a word (4 bytes) with an address increment of 32 words (128 bytes) before reading another word. The increment is reduced by half on successive tests, until all data is read. Data is read using inner loops containing 64 AND statements, that appear to essentially generate the same code for 32 bit and 64 bit compilations, with only 32 bit data words being used. Surprisingly, the 64 bit version produced slow speeds on reaing all data from what should be L1 cache.

Code: Select all

                    BusSpeed 64 Bit  
                                                    
  Memory  Inc32  Inc16   Inc8   Inc4   Inc2   Read  3B+ Gain
  KBytes  Words  Words  Words  Words  Words    All  Read All

      16   3823   4251   4638   4945   5045   3854  1.15 L1
      32   1543   1677   2423   3331   4152   3680
      64    672    694   1306   2169   3300   3577  1.17 L2
     128    635    648   1211   2055   3202   3604
     256    600    615   1163   1971   3152   3612
     512    328    278    695   1272   2256   2978
    1024     94    140    281    543    960   2075  1.12 RAM
    4096     99    128    259    448   1016   1931
   16384    125    129    258    500    898   1863
   65536    125    114    257    500   1015   1898

 #################### Compare 64 bit / 32 bit Pi 3B+ ######################

      16   1.02   1.03   0.98   1.00   0.99   0.76
     256   0.96   0.97   1.00   0.96   0.99   0.90
   65536   0.99   0.88   1.02   1.02   1.01   1.10

#######################################################################

Fast Fourier Transforms - There are two FFT benchmarks, the second one benefiting from being optimised to make better use of burst data transfers, with the procedures dependent of skipped sequential access. FFT sizes vary between 1K and 1024K, covering caches and RAM. Three copies are run using both single and double precision data, the middle ones used here, as best choice due to varying millisecond running times. Because of the latter, 3B/3B+ comparisons are not as constant as for other benchmarks, this being reflected in the different 64/32 bit comparisons provided below.

With running times of the smaller FFTs being less than a millisecond, that for the first few measurements can be extended with the CPU MHz scaling governor set as on demand. A performance setting is required to produce more acceptable results. An example is shown below.

Code: Select all

                 FFT Benchmarks 
   
    Size  -------- milliseconds --------
       K  Single  Double  Single  Double

                 scaling_governor
             performance      ondemand
  
       1    0.17    0.14    0.40    0.14
       2    0.38    0.32    0.93    0.32
       4    1.07    0.77    1.97    0.75
       8    2.13    1.89    4.64    1.76
      16    4.57    5.83    4.47    5.83 



 #################### Compare 64 bit / 32 bit ######################

                RPi3           RPi3B+ 
        K  Single  Double  Single  Double 
  FFT1
   1 to 8    1.05    0.86    1.06    0.90
  16 to 128  1.17    0.83    1.14    1.06
 256 to 1M   1.26    0.88    1.58    1.13

 FFT3C
   1 to 8    1.24    0.89    1.17    0.88
  16 to 128  1.05    1.04    1.15    1.17
 256 to 1M   1.14    1.01    1.26    1.16

 #######################################################################

[Sakaki]

RPi3 B+ Ethernet Issues (RPi3 B users unaffected) and Workaround

gentoo-on-rpi3-64bit users,

Currently, there appear to be two issues with the RPi3 B+'s LAN7515 chip which can significantly affect performance (and which will need fixes in the customized lan78xx driver, most likely). As far as I can tell, these affect both 64 and 32-bit users, all distros.

The first issue occurs when connecting your RPi3 B+ to an Ethernet switch that does not have flow control turned on (as is the case in many datacentres, for example). RPF engineer comment (quotes below all from this thread on the forums):

Reports so far.

Been hitting the ethernet with a stick for the last few days, that stick being iperf3. I have been testing the onboard, and two different USB->GigE dongles.

The main conclusion we have come to - if your switch does NOT have flow control turned on, you will suffer a large drop in performance. This is due to the Pi being unable to service incoming data at gig speeds, and therefore a lot of retries are required for dropped frames.

[...]

Finally, we are talking to Microchip to see what can be done to improve performance. It's their chip and their driver. However, we are still investigating.

The second issue concerns bad Ethernet performance once a certain amount of data (around 2GB?) has been transferred. RPF engineer comment:

I have a suspicion I'm also having related network troubles on my 3B+. If I copy more than a few GB via SMB, the 3B+ network becomes unreliable and stops being able to copy files. Extremely frustrating. I did set flow control to on in my Netgear switch, and that meant that failures took longer to occur and copy speeds were a little higher (~18MB/s), but the same problem still occurs.

For now I've gone back to my 3B. Should I be returning my 3B+?

We have had a number of reports on large transfers dying when using Samba . I think its unrelated to the issue in this thread, but it is being looked in to.

I have personally experienced this (I think), where a large rsync backup of my RPi3 B+ build server choked (repeatedly) about half way (several GB) through, connected over a local LAN (I have never had any issues running the same rsync script on an RPi3 B).

Temporary Workaround

If you are using your RPi3 B+ to provide production services over Ethernet, I recommend using ethtool to reduce the advertised performance to 100Mbit (that of the old RPi3B's adaptor) pro tem. This workaround seems to prevent the problems occurring.

NB: If you only use your RPi3 B+'s Ethernet for light access (e.g. ssh), or use your RPi3 B+'s WiFi networking primarily, or use an RPi3 B, you need take no action at this stage.

To do so, open a terminal on your gentoo-on-rpi3-64bit RPi3 B+ system, and issue:

Code: Select all

demouser@pi64 ~ $ sudo emaint sync --repo rpi3
demouser@pi64 ~ $ sudo emerge -v sys-apps/ethtool
demouser@pi64 ~ $ sudo mousepad /etc/local.d/slow-down-eth0.start &>/dev/null&

The mousepad editor should open, onto the (new) file /etc/local.d/slow-down-eth0.start. Copy the following into that file:

Code: Select all

#!/bin/bash

# slow down eth0 to 100Mbit until properly fixed
# see https://www.raspberrypi.org/forums/viewtopic.php?f=63&t=208512

>&2 echo "WARNING: setting eth0 to 100Mbit max"
ethtool -s eth0 speed 100 duplex full autoneg on

Save the file, and exit mousepad. Then issue:

Code: Select all

demouser@pi64 ~ $ sudo chmod -v ugo+x /etc/local.d/slow-down-eth0.start

to make the script runnable at (each) boot time.

Finally, run the script now, so it will take effect immediately (no need to reboot):

Code: Select all

demouser@pi64 ~ $ sudo /etc/local.d/slow-down-eth0.start
WARNING: setting eth0 to 100Mbit max

You can check the change has been taken up, with:

Code: Select all

demouser@pi64 ~ $ sudo ethtool eth0 | grep "Advertised link modes:"
	Advertised link modes:  100baseT/Full 

When a fix to the kernel driver (covering both the above issues) is released, I'll post again here, at which point you can simply delete the /etc/local.d/slow-down-eth0.start file, if you are using it.

Apologies for the inconvenience ><

[orion777]

Good evening!
I was migrating to the latest version of the assembly. Now I'm getting extremelly high loads reported, however we know that RPI3 has onle 4 kernels..

Code: Select all

 Load avg: 14.1, 11.6, 7.5

How to understand it?

And one more question: if I was installing specific version of some software, will it be updated to the latest version during genup?

[roylongbottom]

LAN Speed

I have installed SAMBA in Gentoo for RPi 3B+ and would like to make use of it in testing my LanSpeed benchmark. Previously I have just edited samba.conf by copying a defined file (with possible mods). This time, there was no conf file to edit but, in vain, I tried a different one. Can anyone help.?

The LanSpeed benchmark runs properly on the RPi 3B+ via Raspbian but, at 32 bits, probably as would be expected, produces a segmentation fault in attempting to allocate 2048 MB of memory. However, it works at 2000 MB, writing three files of this size to a PC. Writing speed was 31 to 34 MB/sec (maybe speed to buffer) and reading was at 22 to 27 MB/sec (>2 times original RPi3 speed).

I wonder what will happen at 64 bits.

[Sakaki]

Good evening!
I was migrating to the latest version of the assembly. Now I'm getting extremelly high loads reported, however we know that RPI3 has onle 4 kernels..

Code: Select all

 Load avg: 14.1, 11.6, 7.5

How to understand it?

For more than you ever wanted to know on Linux load averages ^-^, see this Wikipedia article, and "Linux Load Averages: Solving the Mystery" by Brendan Gregg. From the latter (emphasis added):

I grew up with OSes where load averages meant CPU load averages, so the Linux version has always bothered me. Perhaps the real problem all along is that the words "load averages" are about as ambiguous as "I/O". Which type of I/O? Disk I/O? File system I/O? Network I/O? ... Likewise, which load averages? CPU load averages? System load averages? Clarifying it this way lets me make sense of it like this:

• On Linux, load averages are (or try to be) "system load averages", for the system as a whole, measuring the number of threads that are working and waiting to work (CPU, disk, uninterruptible locks). Put differently, it measures the number of threads that aren't completely idle. Advantage: includes demand for different resources.
• On other OSes, load averages are "CPU load averages", measuring the number of CPU running + CPU runnable threads. Advantage: can be easier to understand and reason about (for CPUs only).

Note that 'disk, uninterruptible locks' part... from the Wikipedia article (emphasis added):

An idle computer has a load number of 0 (the idle process isn't counted). Each process using or waiting for CPU (the ready queue or run queue) increments the load number by 1. Each process that terminates decrements it by 1. Most UNIX systems count only processes in the running (on CPU) or runnable (waiting for CPU) states. However, Linux also includes processes in uninterruptible sleep states (usually waiting for disk activity), which can lead to markedly different results if many processes remain blocked in I/O due to a busy or stalled I/O system.[1] This, for example, includes processes blocking due to an NFS server failure or too slow media (e.g., USB 1.x storage devices). Such circumstances can result in an elevated load average which does not reflect an actual increase in CPU use (but still gives an idea of how long users have to wait).

Translating this somewhat liberally, a load of 14.1 (the first number in your load average list) on a 4 core processor means that over the last minute (in an exponentially averaged sense) there were 14.1/4 = 3.5 runnable or I/O blocked threads per core.

The system is indeed highly loaded, but much of it probably relates to the @world update triggered by genup (see below) trying to merge large numbers of binary packages to a slow I/O subsystem (e.g. your RPi3's microSD card). Note that by default though, genup runs emerges at maximum niceness (minimum system priority), so although your machine may become sluggish, it shouldn't completely lock up when an update is happening - if one of those "ready to run" threads is e.g. due to the GUI, that will have higher priority than the emerge-related threads, and get CPU time (unless the system is swapping heavily of course, in which case the GUI-related thread may need to wait on the - presumably stalled - I/O for necessary swap pages, and then you will experience some stalling; this is one good reason to put your swap on a separate bus from your sysroot, as ericbish notes).

And one more question: if I was installing specific version of some software, will it be updated to the latest version during genup?

The process followed by genup is described in its manpage. The important line is:

• updates all packages in the @world set (using emerge --deep --with-bdeps=y --changed-use --update @world)

Now, if you manually install a particular version of a package, but take no other action, Portage will record only the generic package atom in your world file (/var/lib/portage/world).

For example, at the moment, dev-util/diffstat is available in two versions in the main Gentoo tree:

Code: Select all

pi64 ~ # eix -e diffstat
* dev-util/diffstat
     Available versions:  1.60 (~)1.61
     Homepage:            http://invisible-island.net/diffstat/
     Description:         Display a histogram of diff changes

Suppose you chose to emerge the 1.60 specifically, with:

Code: Select all

pi64 ~ # emerge -v =dev-util/diffstat-1.60

Then dev-util/diffstat-1.60 would be duly installed, but:

Code: Select all

pi64 ~ # grep diffstat /var/lib/portage/world
dev-util/diffstat

No mention of the version.

So, since there is a valid (on ~arm64) newer version in the tree, it will be upgraded (along with any other packages) when you run genup. Incidentally, eix knows this (the '' means eligible for upgrade):

Code: Select all

pi64 ~ # eix -e diffstat
[U] dev-util/diffstat
     Available versions:  1.60{tbz2} (~)1.61
     Installed versions:  1.60{tbz2}(12:30:07 AM 04/30/2018)
     Homepage:            http://invisible-island.net/diffstat/
     Description:         Display a histogram of diff changes

If you wanted to 'pin' diffstat-1.60 (to prevent an upgrade, which for manually installed packages may trigger a lengthy compile on your machine during genup, since there is no matching binary package on the binhost), you could e.g. create a file /etc/portage/package.mask/diffstat with the contents:

Code: Select all

<dev-util/diffstat-1.60
>dev-util/diffstat-1.60

(there are other ways to achieve this, of course). Now run eix again:

Code: Select all

pi64 ~ # eix -e diffstat
[I] dev-util/diffstat
     Available versions:  1.60{tbz2} [m](~)1.61
     Installed versions:  1.60{tbz2}(12:30:07 AM 04/30/2018)
     Homepage:            http://invisible-island.net/diffstat/
     Description:         Display a histogram of diff changes

That "" indicates that Portage no longer "sees" any valid upgrade candidate (because your masks exclude 1.61, as shown by the "[m]").

Hope that makes sense!

Edit: PS you can always use the "top" command to get an idea of what is loading your system.

[Sakaki]

I have installed SAMBA in Gentoo for RPi 3B+ and would like to make use of it in testing my LanSpeed benchmark. Previously I have just edited samba.conf by copying a defined file (with possible mods). This time, there was no conf file to edit but, in vain, I tried a different one. Can anyone help.?

Unfortunately I know very little about SAMBA, having never used it, but hopefully someone else on here can help point you in the right direction... In fact this may not even be the best sub-forum to post in for advice (since you have SAMBA installed, so the issue isn't really arm-related per se, just getting an appropriate smb.conf or whatever). Neddy?

[Sakaki]

I have installed SAMBA in Gentoo for RPi 3B+ and would like to make use of it in testing my LanSpeed benchmark. Previously I have just edited samba.conf by copying a defined file (with possible mods). This time, there was no conf file to edit but, in vain, I tried a different one. Can anyone help.?

Just to double-check, on your benchmarks page you state that:

In order for Windows Workgroup systems to access RPi files, samba and samba-common-bin need to be installed, along with changes to /etc/samba/smb.conf. Detailed procedures are in Treating Raspberry Pi as just another Windows machine.

The linked article suggests a smb.conf file:

Code: Select all

#======================= Global Settings =======================
[global]
workgroup = WORKGROUP
server string = mypi server
netbios name = mypi

dns proxy = no

#### Debugging/Accounting ####
log file = /var/log/samba/log.%m
max log size = 1000
syslog = 0
panic action = /usr/share/samba/panic-action %d

####### Authentication #######
security = user
map to guest = pi

#======================= Share Definitions =======================
[homes]

comment = Home Directories
browseable = yes
guest ok = yes
read only = no
create mask = 0775
directory mask = 0775
writeable = yes
guest account = pi

[public]
path = /
guest ok = yes
guest account = ftp
browseable = yes
read only = no
create mask = 0777
directory mask = 0777
writeable = yes
admin users = everyone

If this is the file you tried using, did you replace the references to the "pi" account in the smb.conf file with "demouser" (the image, as shipped, does not have a "pi" account pre-defined, as the Raspbian ones do)?

[NeddySeagoon]

Sakaki,

Samba? Me?

I've not suffered from Windows at home since I discovered Gentoo in April 2002. :)

[Sakaki]

Sakaki,

Samba? Me?

I've not suffered from Windows at home since I discovered Gentoo in April 2002.

I only meant maybe move roylongbottom's question to a different subforum, not that you had gone over to the dark side lol ^-^

[roylongbottom]

If this is the file you tried using, did you replace the references to the "pi" account in the smb.conf file with "demouser" (the image, as shipped, does not have a "pi" account pre-defined, as the Raspbian ones do)?

I tried that with "demouser" but samba failed to start after command sudo /etc/init.d/samba restart, with the following message:

Code: Select all

demouser@pi64 ~ $ sudo /etc/init.d/samba restart
 * samba -> start: smbd ...
 * start-stop-daemon: failed to start `/usr/sbin/smbd'                    [ !! ]
 * samba -> start: nmbd ...
 * start-stop-daemon: failed to start `/usr/sbin/nmbd'                    [ !! ]
 * Error: starting services (see system logs)
 * samba -> stop: smbd ...
 * start-stop-daemon: no matching processes found                         [ ok ]
 * samba -> stop: nmbd ...
 * start-stop-daemon: no matching processes found                         [ ok ]
 * ERROR: samba failed to start

I don't completely depend on Windows, Where can I find instructions to connect to Linux Ubuntu?

[Sakaki]

roylongbottom,

are any useful messages printed to /var/log/samba.log* or /var/log/messages when you try starting the SAMBA service? (try "sudo tail <logname>" to see; you could also try "sudo dmesg | tail"). There is probably something simple at issue in the configuration file.

[Sakaki]

roylongbottom,

I've managed to get samba to start at least on an RPi3 B+, with some changes to the /etc/samba/smb.conf file. I have no idea whether it'll allow you to see the directories etc., but netstat shows it is listening on the relevant ports and there are no errors reported.

Here's what I did:

Code: Select all

pi64 ~ # emerge -v net-fs/samba
pi64 ~ # nano -w /etc/samba/smb.conf

and put in that file:

Code: Select all

#======================= Global Settings =======================
[global]
workgroup = WORKGROUP
server string = mypi server
netbios name = mypi

dns proxy = no

passdb backend = smbpasswd
wins support = yes
guest account = demouser

#### Debugging/Accounting ####
log file = /var/log/samba/log.%m
max log size = 1000
panic action = /usr/share/samba/panic-action %d

####### Authentication #######
security = user

#======================= Share Definitions =======================
[homes]

comment = Home Directories
browseable = yes
guest ok = yes
read only = no
create mask = 0775
directory mask = 0775
writeable = yes

[public]
path = /
guest ok = yes
browseable = yes
read only = no
create mask = 0777
directory mask = 0777
writeable = yes
admin users = everyone

Save, and exit nano. Then:

Code: Select all

pi64 ~ # testparm
Load smb config files from /etc/samba/smb.conf
rlimit_max: increasing rlimit_max (1024) to minimum Windows limit (16384)
Processing section "[homes]"
Processing section "[public]"
Loaded services file OK.
Server role: ROLE_STANDALONE

Press enter to see a dump of your service definitions

# Global parameters
[global]
	dns proxy = No
	guest account = demouser
	log file = /var/log/samba/log.%m
	max log size = 1000
	netbios name = MYPI
	panic action = /usr/share/samba/panic-action %d
	passdb backend = smbpasswd
	security = USER
	server string = mypi server
	wins support = Yes
	idmap config * : backend = tdb


[homes]
	comment = Home Directories
	create mask = 0775
	directory mask = 0775
	guest ok = Yes
	read only = No


[public]
	admin users = everyone
	create mask = 0777
	directory mask = 0777
	guest ok = Yes
	path = /
	read only = No

Looks OK as far as it goes (although it may be wildly insecure, idk).

Next, per your linked tutorial article, I set up a password using what now appears to be a deprecated method, but it still works, so:

Code: Select all

pi64 ~ # smbpasswd -a demouser
<enter raspberrypi64 when prompted>

Run it:

Code: Select all

pi64 ~ # rc-service samba start
 * samba -> start: smbd ...                                               [ ok ]
 * samba -> start: nmbd ...                                               [ ok ]
pi64 ~ # rc-service samba status
 * status: started

yay!

Ports 139 and 445 are live:

Code: Select all

pi64 ~ # netstat -tl4n | grep "\(139\|445\)"
tcp        0      0 0.0.0.0:139             0.0.0.0:*               LISTEN     
tcp        0      0 0.0.0.0:445             0.0.0.0:*               LISTEN     

I have no idea whether this will actually serve anything, but at least it is a step nearer ^-^
Please let me know if it works your end.

[roylongbottom]

Sakaki

Thanks for that, but I was just going to tell you that I had managed to start samba, after noticing that my smb.conf for Raspbian had Authentication “map to guest = ?????” commented out. I could not access the Rpi from Windows but could mount Windows folders on the RPi to run my LAN benchmark, well not exactly run. It wrote and read three 8 MB files, at the same speed as the 32 bit benchmark, then crashed trying to write a second 16 MB file.

I tried all your suggested commands and results were the same as yours. Then I set the password (why didn’t I read my recommended file?). Then I could access RPi files from Windows. There I have a LAN.exe Windows benchmark. When clicked, it runs on the PC, transferring data to/from the RPi. This ran to completion, but at an extremely slow speed.

Then, The RPi would not power off (until I pulled the plug). I wonder if it was still trying to transfer data to the PC. I will see what happens tomorrow.

[Sakaki]

roylongbottom,

no idea what may be causing the samba issues, probably something I've misconfigured in the smb.conf file ><

However, if you have access to an Ubuntu machine on the same subnet, why not try iperf on both of them?

It is available on the RPi3 (sudo emerge net-misc/iperf) and on ubuntu (sudo apt-get install iperf).

Run it as "iperf3 -s" on your RPi3, and "iperf -c <IP address of RPi3>" from your Ubuntu box.

Will give you basic network throughput stats.

[Sakaki]

roylongbottom,

FWIW, I tried transferring some large files to and from a Win10 box today over samba from an RPi3B+, using a folder exported from the Pi. It seemed to work fine (albeit I was using WiFi, not Ethernet).

As to your question:

I don't completely depend on Windows, Where can I find instructions to connect to Linux Ubuntu?

please see my short NFS 'howto' below.

Sharing Folders between an RPi3 (B or B+) and a Ubuntu Box, Using NFS

Since you have an Ubuntu system, you could use NFS to share folders from the RPi3. Here's a simple (i.e. not very secure!) setup to achieve this (for more detailed setup instructions, see e.g. this article).

OK, on your RPI3 (B or B+), first install the NFS userspace components (prefix all commands with sudo if not working as root):

Code: Select all

pi64 ~ # emerge -av net-fs/nfs-utils

Wait for that to complete (all required packages are on the binhost, so it shouldn't take long). Then, export your whole RPi3 root filing system (I told you this was going to be insecure ^-^): issue:

Code: Select all

pi64 ~ # mousepad /etc/exports

and edit that file so it reads:

Code: Select all

# /etc/exports: NFS file systems being exported.  See exports(5).
/ *(rw,sync,no_subtree_check)

Save, and exit mousepad. Take a note of your RPi3's Ethernet IP address:

Code: Select all

pi64 ~ # ifconfig eth0 | grep 'inet '
        inet 192.168.1.100  netmask 255.255.255.0  broadcast 192.168.1.255

(the address is 192.168.1.100 here, but yours will probably differ).
Then start the NFS server:

Code: Select all

pi64 ~ # rc-service nfs start
 * Mounting nfsd filesystem in /proc ...                                  [ ok ]
 * Exporting NFS directories ...                                          [ ok ]
 * Starting NFS mountd ...                                                [ ok ]
 * Starting NFS daemon ...                                                [ ok ]
 * Starting NFS smnotify ...                                              [ ok ]

Now, on your Ubuntu machine install the NFS client utils:

Code: Select all

user@ubuntu~:$ sudo apt-get update
user@ubuntu~:$ sudo apt-get install nfs-client

Create a mountpoint, and mount the NFS share:

Code: Select all

user@ubuntu~:$ sudo mkdir -pv /mnt/piroot
user@ubuntu~:$ sudo mount -v 192.168.1.100:/ /mnt/piroot

Obviously, substitute 192.168.1.100 in the above with the IP address of your RPi3 as returned by ifconfig, earlier.

That's it! You should now be able to copy files from the Pi to your Ubuntu system, and vice versa (you need appropriate access permissions of course, but e.g. the Pi's /tmp directory, which will now be visible at /mnt/piroot/tmp on Ubuntu, should allow full read/write access).

When done, just:

Code: Select all

user@ubuntu~:$ sudo umount -v /mnt/piroot

to disconnect.

[roylongbottom]

Deleted details as accidently posted twice

[roylongbottom]

Sakaki

LAN Speed Benchmarks


At last, I have run my 64 bit LAN speed benchmark on the Raspberry Pi 3B+ with Gentoo. I was distracted by problems with SAMBA and my own stupid network, that often fails to recognise other computers. The first SAMBA issues are that it does not start automatically on booting and fails to identify connected PCs (all the time, unlike other computers).

Then I realised that SAMBA is not needed to run the benchmarks on the Pi. This is achieved by using a mount command such as “sudo mount -t cifs -o password=raspberrypi64 -o dir_mode=0777,file_mode=0777 //192.168.1.68/d /media/public” and running with “./LanSpeed64 FilePath /media/public/ray”. Details and benchmark downloads can be found by downloading the pdf document from:

https://www.researchgate.net/publicatio ... ationTitle

LanSpeed writes and reads three files at two sizes (defaults 8 and 16 MB), followed by random reading and writing of 1KB blocks out of 4. 8 and 16 MB and finally, writing and reading 200 small files, sized 4, 8 and 16 KB.

Below is an example of the log file produced for the 32 bit Raspbian version running on the older Pi 3B, then the performance results for same version running on a Pi 3B+, both using the same Windows 7 based PC. Speeds of the latter are typically three times faster on writing large files and more than twice as fast reading (but note variations). Again using the same remote PC, speeds are shown from a run using the 64 bit version via Gentoo, with similar performance (numerous runs would be required to accurately compare all the results). Finally, using the same setup are results ion using larger files (example command for the second one “./LanSpeed64 MB 1024 FilePath /media/public/ray”

Code: Select all

            Raspbian 32 bit LanSpeed Raspberry Pi 3B

            LanSpeed RasPi 1.0 Wed Apr  4 12:29:52 2018
 
 Selected File Path: 
 /media/public/ray/
 Total MB  266240, Free MB   90555, Used MB  175685 - PC Windows 7

                        MBytes/Second
  MB   Write1   Write2   Write3    Read1    Read2    Read3

   8    11.09    11.19    11.27    11.67    11.63    11.65
  16    11.30    11.42    11.23    11.68    11.65    11.69

 Random         Read                       Write
 From MB        4        8       16        4        8       16
 msecs      0.732    1.341    0.908     1.60     0.92     0.95

 200 Files      Write                      Read                  Delete
 File KB        4        8       16        4        8       16     secs
 MB/sec      0.82     1.50     2.37     1.38     1.59     2.88
 ms/file     5.01     5.47     6.91     2.97     5.14     5.69    0.764

                End of test Wed Apr  4 12:31:00 2018

 ======================================================================
         Raspbian 32 bit Raspberry Pi 3B+ to Windows 7 PC

                        MBytes/Second
  MB   Write1   Write2   Write3    Read1    Read2    Read3

   8    32.59    35.27    35.57     8.07    25.79    25.57
  16    35.64    34.43    35.79    22.76    25.60    25.76

 Random         Read                       Write
 From MB        4        8       16        4        8       16
 msecs      0.453    1.277    1.150     1.32     0.70     0.68

 200 Files      Write                      Read                  Delete
 File KB        4        8       16        4        8       16     secs
 MB/sec      1.14     2.03     3.82     1.77     3.06     5.28
 ms/file     3.58     4.04     4.29     2.31     2.68     3.10    0.735

 ======================================================================
          Gentoo 64 bit Raspberry Pi 3B+ to Windows 7 PC

                       MBytes/Second
  MB   Write1   Write2   Write3    Read1    Read2    Read3

   8    28.34    34.87    35.13    13.47    27.08    26.72
  16    34.16    35.21    35.05    25.85    24.75    25.23

 Random         Read                       Write
 From MB        4        8       16        4        8       16
 msecs      0.006    1.062    0.693     1.04     1.35     1.40

 200 Files      Write                      Read                  Delete
 File KB        4        8       16        4        8       16     secs
 MB/sec      0.87     1.96     3.15     1.67     3.49     5.90
 ms/file     4.69     4.18     5.21     2.45     2.34     2.77    0.460

 ======================================================================
  Larger Files Gentoo 64 bit Raspberry Pi 3B+ to Windows 7 PC

                        MBytes/Second
  MB   Write1   Write2   Write3    Read1    Read2    Read3

 256    38.11    38.30     1.62    24.17    27.89    27.87
 512    38.66    38.29    38.36    25.11    25.20    27.83

1024    38.76    38.87    38.89    27.63    26.43    26.11
2048    39.01    38.91    38.94    26.02    26.20    25.30

I have a PC with a 3900 MHz Core i7 CPU that can dual boot to Windows 10 and Linux Ubuntu. At this time, I could not mount the Windows system, due to a security issue, but could to Ubuntu, with results accessing large files shown below. These might be interpreted as being slightly slower than data transfers to the Windows PC. The other tests had similar variations in performance.

Following are all results from the more comprehensive 64 bit Linux version, accessing the RPi 3B+ Gentoo system. Note that reading is shown as faster than writing, or data in the same direction as RPi writing.

Code: Select all

    Gentoo 64 bit Raspberry Pi 3B+ to Linux Ubuntu

 Selected File Path: 
 /media/public/
 Total MB  446040, Free MB  369390, Used MB   76650 - PC Ubuntu

                        MBytes/Second
  MB   Write1   Write2   Write3    Read1    Read2    Read3

   8    29.01    34.05    34.53    16.75    26.10    28.52
  16    33.98    34.80    34.88    28.64    20.66    26.41

 256    35.03    34.99    34.97    22.10    18.53    19.35
 512    35.07    35.01    34.76    18.12    20.62    21.45

1024    34.80    34.85    35.09    20.59    20.85    19.72
2048    34.23    34.17    34.31    20.94    25.06    24.76

 ======================================================================
      Linux LanSpeed Bechmark 64 Bit Ubuntu to Rpi 3B+ with Gentoo
 
 Selected File Path: 
 /media/public/benchmarks/
 Total MB   28414, Free MB   20683, Used MB    7731

 Linux LAN/WiFi Speed Test 64-Bit Version 1.1, Wed May  2 16:47:41 2018

                Copyright (C) Roy Longbottom 2011

     8 MB File         1          2          3          4          5
 Writing MB/sec      19.23      21.36      14.55      21.84      21.04
 Reading MB/sec      36.43      38.98      39.08      39.26      39.17

    16 MB File         1          2          3          4          5
 Writing MB/sec      18.21      15.64      23.86      23.82      12.79
 Reading MB/sec      38.68      39.47      39.56      39.35      39.41

    32 MB File         1          2          3          4          5
 Writing MB/sec      17.33      17.23      16.61       6.88       8.53
 Reading MB/sec      39.26      14.83      39.67      39.63      38.03

 ---------------------------------------------------------------------
 8 MB Cached File      1          2          3          4          5
 Writing MB/sec      22.56      21.45      21.52      21.61      19.46
 Reading MB/sec      36.84      38.87      39.18      38.83      39.18

 ---------------------------------------------------------------------
 Bus Speed Block KB     64        128        256        512       1024
 Reading MB/sec   21056.30   20819.11   18642.71   16841.94   16906.29

 ---------------------------------------------------------------------
 1 KB Reads File MB >    2      4      8     16     32     64    128
 Random Read msecs    0.60   0.56   0.56   0.57   0.56   0.56   0.75

 ---------------------------------------------------------------------
 500 Files   Write             Read             Delete
 File KB     MB/sec  ms/File   MB/sec  ms/File  Seconds
       2       0.40     5.09     0.60     3.39    2.526
       4       0.80     5.09     1.15     3.56    2.475
       8       1.55     5.30     2.31     3.55    2.494
      16       2.94     5.57     4.38     3.74    2.525
      32       5.18     6.32     7.85     4.17    2.593
      64       8.24     7.95    12.34     5.31    2.545

              End of test Wed May  2 16:49:48 2018

SAMBA Tests

Finally are results from running a Windows/Intel benchmark, stored on the RPi 3B+/Gentoo SD card. The appropriate directory has to be opened via a Windows network selection and the program run by simply clicking on the entry.

There was more variability in measured performance and particularly slow reading speeds. On occasions, running from the AMD Phenom based PC, caused the Gentoo/Raspberry Pi 3B+ system to freeze.

Code: Select all

 Running LanSpdx86Win.exe stored ob RPi 3B+ Executed from
 Intel(R) Core(TM) i7 CPU @ 3.90GHz, Windows 10  

 Current Drive Details
 Total MB    7333, Free MB    3069, Used MB    4264

 LanSpeed Windows 32-Bit Version 1.0, Thu May 03 09:22:29 2018

                        MBytes/Second
  MB   Write1   Write2   Write3    Read1    Read2    Read3

   8     3.72     5.28    22.58    13.56     9.70     7.56
  16    23.66    22.99    16.33    10.89     8.22     6.67

 Random         Read                       Write
 From MB        4        8       16        4        8       16
 msecs      0.566    0.536    0.472    0.498    0.455    0.441

 200 Files      Write                      Read                  Delete
 File KB        4        8       16        4        8       16     secs
 MB/sec      0.72     1.42     2.05     0.49     1.25     1.94
 ms/file     5.70     5.78     7.98     8.43     6.55     8.45    1.347

              End of test Thu May 03 09:23:08 2018

 ======================================================================

   From AMD Phenom(tm) II X4 945 @ 3000 MHz, Windows 7 

                        MBytes/Second
  MB   Write1   Write2   Write3    Read1    Read2    Read3

   8     8.58    24.64    25.25    15.10    10.78     8.35
  16    16.45    22.74     6.12     5.26     4.61     4.09

Earlier Suggestions

I installed and tried executing iperf but nothing happened. I confirmed data transmission speeds via Windows and Linux performance monitors.

Thanks for you advice on sharing folders. I might try it later, if needed. For the time being, I can copy files to/from the directory mounted as above.

[steveL]

Sakaki: this is the best thread I have seen in a very long time.

Thank you so much, and to Roy Longbottom, as well as Neddy (as always :) for all the hard work, and especially the exemplary documentation (using the forums as Jah intended.)

[orion777]

Translating this somewhat liberally, a load of 14.1 (the first number in your load average list) on a 4 core processor means that over the last minute (in an exponentially averaged sense) there were 14.1/4 = 3.5 runnable or I/O blocked threads per core.

I was migrating to the bigger, but seems that slower SD card. During this load only emegre and xserver was running, so the swap file was used.. so maybe swap access was too slow.


Now I use image which was downloaded 2 weeks ago. Now I have to build my own kernel

Code: Select all

cd /usr/src
git clone --depth 1 https://github.com/raspberrypi/linux.git -b rpi-4.14.y
cd linux
make distclean
make bcmrpi3_defconfig
make menuconfig
	> CPU Power Management > CPU Frequency scaling   --- Default CPUFreq governor (powersave)  ---> Ondemand
make –j5

but I get a loooot of such errors:

Code: Select all

net/sunrpc/xdr.o:(__mcount_loc+0x148): dangerous relocation: unsupported relocation
net/sunrpc/cache.o:(.data+0x50): dangerous relocation: unsupported relocation
lib/radix-tree.o:(___ksymtab+radix_tree_tag_clear+0x0): dangerous relocation: unsupported relocation
.....
lib/win_minmax.o:(___ksymtab+minmax_running_max+0x8): dangerous relocation: unsupported relocation
drivers/firmware/efi/libstub/lib-sort.stub.o: In function `__efistub_$d':
__efistub_sort.c:(.init___ksymtab+sort+0x0): dangerous relocation: unsupported relocation
__efistub_sort.c:(.init___ksymtab+sort+0x8): dangerous relocation: unsupported relocation
ld: warning: creating a DT_TEXTREL in a shared object.
make: *** [Makefile:1021: vmlinux] Error 1

But I have to build my own kernel to implement MPTCP ))

[Sakaki]

Sakaki: this is the best thread I have seen in a very long time.

Thank you so much, and to Roy Longbottom, as well as Neddy (as always for all the hard work, and especially the exemplary documentation (using the forums as Jah intended.)

steveL - on behalf of the contributors on this thread, thanks for your kind words - much appreciated ^-^