Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Bootable 64-bit RPi3 Gentoo image (OpenRC/Xfce/VC4) UPDATED
View unanswered posts
View posts from last 24 hours

Goto page Previous  1, 2, 3 ... 14, 15, 16 ... 18, 19, 20  Next  
This topic is locked: you cannot edit posts or make replies.    Gentoo Forums Forum Index Gentoo on ARM
View previous topic :: View next topic  
Author Message
Sakaki
Guru
Guru


Joined: 21 May 2014
Posts: 409

PostPosted: Fri Jul 06, 2018 12:29 pm    Post subject: Reply with quote

orion777 wrote:
...
Now I was copying my custom kernel to the /boot
but I forgot to run make modules_install again!
...

I'm still able to access the file system via card reader of the laptop, but I don't know how to solve this problem :( If someone have any ideas, these are welcome, because reinstalling again and again.. rpi3 64 bit image, kernel implementation and custom drone software are time-expensive..


Probably the simplest thing, to get going again, would be to mount your microSD card's partition 1 and 2 on your Linux PC, download the current prebuilt bcmrpi3-kernel-bis tarball, and untar it onto the card (per the instructions here). This should be enough to get you booted, you can then install the modules again. None of this requires any cross-compiling etc.

For example, assuming your card shows up as (e.g.) /dev/mmcblk0 on your your PC, you could do (adapt as appropriate):

Mount the card partitions on your PC:
Code:
gentoo-pc ~ # mkdir -pv /mnt/piroot
gentoo-pc ~ # mount -v /dev/mmcblk0p2 /mnt/piroot
gentoo-pc ~ # mount -v /dev/mmcblk0p1 /mnt/piroot/boot


Archive your custom kernel, download latest binary kernel tarball (it is <15MiB):
Code:
gentoo-pc ~ # mv /mnt/piroot/boot/kernel8.img{,.old}
gentoo-pc ~ # wget -c https://github.com/sakaki-/bcmrpi3-kernel-bis/releases/download/4.14.52.20180703/bcmrpi3-kernel-bis-4.14.52.20180703.tar.xz


Untar the kernel and module set (take care the path after the -C is correct!), ensure written, and dismount:
Code:
gentoo-pc ~ # tar -xJf bcmrpi3-kernel-bis-4.14.52.20180703.tar.xz -C /mnt/piroot/
gentoo-pc ~ # sync
gentoo-pc ~ # umount -v /mnt/piroot/boot
gentoo-pc ~ # umount -v /mnt/piroot


Once rebooted, you can move /boot/kernel8.img.old back to /boot/kernel8.img, do a make modules_install in your kernel build directory, and reboot.

You should then have a functioning system.

orion777 wrote:
Is it possible to exclude modemmanager from the system?

Then, to remove modemmanager, the simplest thing is just to add USE="-modemmanager" to /etc/portage/make.conf (if you already have any USE flags set in there, simply append -modemmanager to the list of course, prefixed with a space), and issue:
Code:
pi64 ~ # emerge --verbose --deep --with-bdeps=y --update --changed-use @world
pi64 ~ # emerge --depclean

And it should be gone; no need for masking etc.

PS to find out what is pulling in a package, you can use the bundled equery tool, for example:
Code:
pi64 ~ # equery depends net-misc/modemmanager
 * These packages depend on net-misc/modemmanager:
gnome-extra/nm-applet-1.8.10-r1 (modemmanager ? net-misc/modemmanager)
net-misc/networkmanager-1.10.10 (modemmanager ? >=net-misc/modemmanager-0.7.991:0)

As you can see there are two packages with direct deps, but these are conditional on the modemmanager USE flag (per the <useflag> ? <dep> syntax). As such, turning this flag off (as above) and rebuilding affected packages (the emerge ... --changed-use @world) will remove any dependencies on the installed net-misc/modemmanager instance, and emerge --depclean will then purge it from your system.
_________________
Regards,

sakaki
Back to top
View user's profile Send private message
orion777
Apprentice
Apprentice


Joined: 15 Mar 2017
Posts: 207
Location: Riga, Latvia

PostPosted: Sat Jul 07, 2018 7:19 am    Post subject: Reply with quote

Quote:
Untar the kernel and module set (take care the path after the -C is correct!), ensure written, and dismount:
gentoo-pc ~ # tar -xJf bcmrpi3-kernel-bis-4.14.52.20180703.tar.xz -C /mnt/piroot/


Where the kernel8.img must be located? In the boot partition or in the root partition folder boot?
Probably in the boot partition, as welcome screen kernel version changes only if I put the kernel into the boot partition.
Whatever, nor boot partition nor root partition boot folder does not solve my problem..
Maybe there is a hint that the system become (none) instead of pi64? Where system naming is done? maybe I will find some missing files there..

Quote:
to remove modemmanager, the simplest thing is just to add USE="-modemmanager"

Sorry, I forgot about USE flags, that can be used in gentoo :oops:
Back to top
View user's profile Send private message
orion777
Apprentice
Apprentice


Joined: 15 Mar 2017
Posts: 207
Location: Riga, Latvia

PostPosted: Sat Jul 07, 2018 7:22 am    Post subject: Reply with quote

Goverp wrote:

While I've not tried on this system (only just installed it and hardly played with it yet), my (AMD) laptop is KDE-based, and uses wpa_supplication and its GUI alone to manage WiFi. Neither networkmanager nor modemmanager are installed, nor whatever else they drag in.

As I know, it is possible to run only dhcpcd and it will manage all interfaces alone (excluding WiFi network selection).
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 54098
Location: 56N 3W

PostPosted: Sat Jul 07, 2018 8:56 am    Post subject: Reply with quote

orion777,

The boot process on the Pi is as follws. The VC4 video chip runs its internal fimware, which you cannot acoess, to load bootcode.bin into RAM from the first partition on the SD card which must be vfat. The ARM CPU is held reset.
Control is passed to bootcode.bin which reads config.txt to learn how to adjust its built in defaults.
Once the kernel, initrd (if any), and dtb are all loaded the ARM CPU is released from reset.

Getting started is all down to the VC4 and bootcode.bin. The ARM CPU is kept out of the way until the hard bit is done.

bootcode.bin looks at the hardware it finds itself on and adjusts the defaults to suit.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
Sakaki
Guru
Guru


Joined: 21 May 2014
Posts: 409

PostPosted: Sat Jul 07, 2018 10:17 am    Post subject: Reply with quote

orion777 wrote:
Where the kernel8.img must be located? In the boot partition or in the root partition folder boot?
Probably in the boot partition, as welcome screen kernel version changes only if I put the kernel into the boot partition.
Whatever, nor boot partition nor root partition boot folder does not solve my problem..
Maybe there is a hint that the system become (none) instead of pi64? Where system naming is done? maybe I will find some missing files there..


Maybe I have mis-understood your issue, if so apologies.

To boot a kernel successfully, you need various firmware files on the microSD card's first partition's vfat filesystem (bootcode.bin etc.), the kernel itself (kernel8.img) and its dtbs (same filesystem), plus various config files (cmdline.txt etc) (same filesystem). Let's call this filesystem bootfs. You also need, on the microSD card's second partition's filesystem (ext4 as shipped, let's call this rootfs), under /lib/modules, a matching module set for the kernel in question.

Once the gentoo-on-rpi3-64bit image is booted, the rootfs is mounted at /, and the bootfs at /boot within it.

Since you wrote:
orion777 wrote:
Now I was copying my custom kernel to the /boot
but I forgot to run make modules_install again!
...
I'm able to login, but there is no network interfaces, none of commands like make menuconfig, make modules_install are working, returning an error read-only file system
...
Original kernel probably boot the system (as it responds on ctrl+alt+del), but the screen is empty (probably video chip is not started)


I provided you instructions to manually install a working kernel + module set from a tarball onto the microSD card using a PC.

The tarball paths by default assume that the rootfs is mounted at ./ and the bootfs is mounted at ./boot (within ./).

However, on the PC following the above instructions, the rootfs will be mounted at /mnt/piroot and the bootfs at /mnt/piroot/boot

During the untar step, the working directory is changed (via -C) to /mnt/piroot/, ensuring that the files in the archive get put in the correct place (the kernel and dtbs into the bootfs, and the matching module set into /lib/modules within the rootfs).

The point of this is to get back to a kernel + modules system that will boot without errors, without losing the any other work stored on the rootfs, requiring any cross-compilation, or download of large files (the tarball mentioned is < 15MiB).

Note that during boot, by default, the rootfs is initially mounted r/o and is later changed to r/w by an OpenRC boot service (if the initial boot has gone OK), /etc/init.d/root. Missing modules will likely cause this to fail, leaving you with a r/o filesystem.

Some other miscellaneous troubleshooting hints:

If you can get a login console on your rpi3 after booting, you can log in as root and try issuing:

Code:
# mount -n -o remount,rw /
# mount /boot



To manually remount the rootfs r/w, then mount the bootfs at /boot within it. This may be enough to allow you to make modules_install or whatever and then reboot.
Also, if your screen is blank, try commenting out the following line in config.txt on the bootfs, so it reads:
Code:
#dtoverlay=vc4-fkms-v3d,cma-256


Leave the rest of the file as-is. You can do this with the bootfs mounted on a PC etc. This will use the simpler graphics driver on boot so you should see error messages etc. earlier (you need the vc4 graphics driver module otherwise).

The hostname is set by the OpenRC service /etc/init.d/hostname, incidentally.
_________________
Regards,

sakaki
Back to top
View user's profile Send private message
orion777
Apprentice
Apprentice


Joined: 15 Mar 2017
Posts: 207
Location: Riga, Latvia

PostPosted: Mon Jul 09, 2018 6:13 pm    Post subject: Reply with quote

Thank You for the explanation!

Ohh, now I understand, this was confusing me, since I was first time worked from the card reader..
Sakaki wrote:

Once the gentoo-on-rpi3-64bit image is booted, the rootfs is mounted at /, and the bootfs at /boot within it.



I will recheck all files. Also I have another flash card with your image, so I should to be able to compare. This flash was prepared a few days ago; also genup was runned and the system is still able to operate correctly.

Quote:
Note that during boot, by default, the rootfs is initially mounted r/o and is later changed to r/w by an OpenRC boot service (if the initial boot has gone OK), /etc/init.d/root. Missing modules will likely cause this to fail, leaving you with a r/o filesystem.

Yes, it was fond by me into the internet; I will check this service too. Thank You.
Back to top
View user's profile Send private message
orion777
Apprentice
Apprentice


Joined: 15 Mar 2017
Posts: 207
Location: Riga, Latvia

PostPosted: Tue Jul 10, 2018 3:54 am    Post subject: Reply with quote

dev-python/matplotlib fails to compile
https://forums.gentoo.org/viewtopic-p-8239162.html#8239162
Back to top
View user's profile Send private message
orion777
Apprentice
Apprentice


Joined: 15 Mar 2017
Posts: 207
Location: Riga, Latvia

PostPosted: Fri Jul 13, 2018 4:46 pm    Post subject: Reply with quote

dev-python/matplotlib-2.2.2-r1 can be installed (at least to me) only with x11-libs/wxGTK 3.0.3(3.0){tbz2}(20:26:31 15/01/18)(X libnotify opengl tiff -aqua -debug -doc -gstreamer -sdl ABI_MIPS="-n32 -n64 -o32" ABI_PPC="-32 -64" ABI_S390="-32 -64" ABI_X86="-32 -64 -x32")
If x11-libs/wxGTK 3.0.4 is installed, then matplotlib-2.2.2-r1 compillator will return
Code:
Fatal Error: Mismatch between the program and library build versions detected.
The library used 3.0 (wchar_t,compiler with C++ ABI 1010,wx containers,compatible with 2.8),
and wxPython used 3.0 (wchar_t,compiler with C++ ABI 1011,wx containers,compatible with 2.8).
Back to top
View user's profile Send private message
orion777
Apprentice
Apprentice


Joined: 15 Mar 2017
Posts: 207
Location: Riga, Latvia

PostPosted: Sun Jul 15, 2018 7:16 am    Post subject: Reply with quote

Good day. I have expected some problems with custom overlay https://forums.gentoo.org/viewtopic-p-8240964.html#8240964 Does the instructions https://wiki.gentoo.org/wiki/Custom_repository and https://wiki.gentoo.org/wiki/Handbook:AMD64/Portage/CustomTree#Defining_a_custom_repository are suitable for this image? I'm asking because instructions says to specify mkdir -p /usr/local/portage/{metadata,profiles} but we have this files in the sakaki-tools.. Maybe they will conflict?
Back to top
View user's profile Send private message
Sakaki
Guru
Guru


Joined: 21 May 2014
Posts: 409

PostPosted: Sun Jul 15, 2018 10:47 am    Post subject: Reply with quote

orion777,

on the image, /usr/local/portage is used as a top-level directory to hold multiple local ebuild repositories (aka overlays), so please don't create an ebuild tree at the top level in there. Instead, you just need to follow the instructions you linked above to create "localrepo", but change the path to use a subdirectory (e.g. to /usr/local/portage/localrepo or similar; make sure to reflect this path in /etc/portage/repos.conf/localrepo.conf, and you'd do e.g. "mkdir -p /usr/local/portage/localrepo/{metadata,profiles}" etc). You can use thin manifests in your own repo to avoid the need to create manifest entries for the ebuilds themselves (you do this in /usr/local/portage/localrepo/metadata/layout.conf - see /usr/local/portage/sakaki-tools/metadata/layout.conf for an example).
_________________
Regards,

sakaki
Back to top
View user's profile Send private message
orion777
Apprentice
Apprentice


Joined: 15 Mar 2017
Posts: 207
Location: Riga, Latvia

PostPosted: Tue Jul 17, 2018 7:40 am    Post subject: Reply with quote

I don't know what was happening here, but I'm still get an error :/
Code:

mkdir -p /usr/local/portage/localrepo/{metadata,profiles}
chown -R portage:portage /usr/local/portage/localrepo

cat /usr/local/portage/localrepo/profiles/repo_name
localrepo

cat /usr/local/portage/localrepo/metadata/layout.conf
repo-rame = localrepo
masters = gentoo
auto-sync = false

cat /etc/portage/repos.conf/localrepo.conf
[localrepo]
location = /usr/local/portage/localrepo

mkdir -p /usr/local/portage/localrepo/ros-meta/mavproxy
cp ~root/mavproxy-9999.ebuild /usr/local/portage/localrepo/ros-meta/mavproxy/mavproxy-9999.ebuild
chown -R portage:portage /usr/local/portage/localrepo    ## Again? But Ok..
pushd /usr/local/portage/localrepo/ros-meta/mavproxy
pi64 /usr/local/portage/localrepo/ros-meta/mavproxy # repoman manifest
 * ERROR: ros-meta/mavproxy-9999::localrepo failed (depend phase):
 *   External commands disallowed while sourcing ebuild:
 *
 * Call stack:
 *              ebuild.sh, line 635:  Called source '/usr/local/portage/localrepo/ros-meta/mavproxy/mavproxy-9999.ebuild'
'*   mavproxy-9999.ebuild, line   3:  Called command_not_found_handle '
 *              ebuild.sh, line  88:  Called die
 * The specific snippet of code:
 *              die "External commands disallowed while sourcing ebuild: ${*}"
 *
 * If you need support, post the output of `emerge --info '=ros-meta/mavproxy-9999::localrepo'`,
 * the complete build log and the output of `emerge -pqv '=ros-meta/mavproxy-9999::localrepo'`.
 * Working directory: '/usr/lib64/python3.5/site-packages'
 * S: '/var/tmp/portage/ros-meta/mavproxy-9999/work/mavproxy-9999'
!!! getFetchMap(): aux_get() error reading ros-meta/mavproxy-9999; aborting.
Unable to generate manifest.


Code:
pi64 /usr/local/portage/localrepo/ros-meta/mavproxy # cat /usr/local/portage/localrepo/ros-meta/mavproxy/mavproxy-9999.ebuild
# Copyright 1999-2017 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2

EAPI=6

PYTHON_COMPAT=( python2_7 )
inherit distutils-r1 git-r3

DESCRIPTION="A UAV ground station software package for MAVLink based systems"
HOMEPAGE="http://ardupilot.github.io/MAVProxy"
EGIT_REPO_URI="https://github.com/ArduPilot/MAVProxy.git"

LICENSE="GPL-3+"
SLOT="0"
# KEYWORDS="~amd64 ~x86"

REQUIRED_USE="${PYTHON_REQUIRED_USE}"

RDEPEND="${PYTHON_DEPS}
    media-libs/opencv[${PYTHON_USEDEP}]
    dev-python/wxpython:2.8[${PYTHON_USEDEP}]
    dev-python/matplotlib[${PYTHON_USEDEP}]
    dev-python/numpy[${PYTHON_USEDEP}]
    dev-python/pyserial[${PYTHON_USEDEP}]
    dev-python/pillow[${PYTHON_USEDEP}]
"

DEPEND="${RDEPEND}
    virtual/pkgconfig"
Back to top
View user's profile Send private message
Sakaki
Guru
Guru


Joined: 21 May 2014
Posts: 409

PostPosted: Tue Jul 17, 2018 4:21 pm    Post subject: Reply with quote

orion777,

I just tried setting up 'localrepo' (containing mavproxy-9999.ebuild) on a fresh copy of the image, following as exactly as I could the steps you describe above. After installing app-portage/repoman, I was able to run "repoman manifest" without error (an alternative, incidentally, would be to run "ebuild mavproxy-9999.ebuild manifest").

Perhaps you have altered some other part of the config of your system that is leading to this issue?
_________________
Regards,

sakaki
Back to top
View user's profile Send private message
orion777
Apprentice
Apprentice


Joined: 15 Mar 2017
Posts: 207
Location: Riga, Latvia

PostPosted: Tue Jul 17, 2018 7:47 pm    Post subject: Reply with quote

Okay.. I have this error with different ebuild and its folder names..
So.. how did You create the mavproxy-9999.ebuild file in your system?
-------------
UPDATE - It is fixed!
So: I was copying ebuild text from https://forums.gentoo.org/viewtopic-p-8241910.html#8241910 and paste it to the windows notepad; then it was uploaded to the linux and...
cat shows that everything is good; mc - view (F3) also shows that everything is perfect; but if you will open the ebuild file (created in windows) via mc - edit (F4), then you will see this:
https://ibb.co/hvDBmy Just clean this unnecessary stuff and feel lucky ))
Back to top
View user's profile Send private message
orion777
Apprentice
Apprentice


Joined: 15 Mar 2017
Posts: 207
Location: Riga, Latvia

PostPosted: Sat Jul 21, 2018 8:47 am    Post subject: Reply with quote

Wish to say Thanks to Sakaki for the help in ebuild testing! Everything is working.
Back to top
View user's profile Send private message
orion777
Apprentice
Apprentice


Joined: 15 Mar 2017
Posts: 207
Location: Riga, Latvia

PostPosted: Tue Jul 24, 2018 7:35 am    Post subject: Reply with quote

Good day! Is it possible to install net-wireless/hostapd ? The problem is that it forses -bindist flag for openssl, dev-embedded/rpi3-64bit-meta-1.2.2-r6 and so...
Back to top
View user's profile Send private message
roylongbottom
n00b
n00b


Joined: 13 Feb 2017
Posts: 64
Location: Essex, UK

PostPosted: Thu Jul 26, 2018 10:45 am    Post subject: Raspberry Pi 3B+ Multithreading Benchmarks Reply with quote

Raspberry Pi 3B+ Multithreading Benchmarks

This is a summary of 32 bit and 64 bit results on Rpi 3B and 3B+, run via Raspbian Stretch and the latest version of 64 bit Gentoo. Full details are available at ResearchGate in:

https://www.researchgate.net/publication/324694380_Raspberry_Pi_3B_32_Bit_and_64_Bit_Benchmarks_and_Stress_Tests

Except for the OpenMP benchmarks, all run using 1, 2, 4 and 8 threads, executing common code. One [/b]and four thread results are provided here.

MP-Whetstone Benchmark

This has an overall rating in MWIPS, mainly dependent on floating point calculations shown under MFLOPS, Cos MOPS and Exp MOPs, with the latter MOPS currently having the most impact. Each thread used dedicated variables to provide best case multithreading speeds.

Based on MWIPS, in all cases, 4 thread performance was around four times single thread performance and 3B+ speed about 17% faster than that for 3B, proportional to CPU MHz ratio (1400/1200). Test functions are too simple to be helped much by advanced architecture, leading to 64 bit benchmark average performance being 6% faster than the 32 bit compilation.

Code:
          MWIPS MFLOPS MFLOPS MFLOPS   Cos   Exp     Fixpt      If  Equal
                     1      2      3  MOPS  MOPS      MOPS    MOPS   MOPS
 32 Bit
 3B  1T   924.2  335.9  276.8  298.5  18.5  10.4    5817.2  1035.3  719.4
     4T  3718.4 1286.3 1303.9 1193.5  74.3  41.5   19961.4  4698.4 2862.7

 3B+ 1T  1084.2  391.0  384.9  348.6  21.7  12.1    6967.0  1013.1  822.3
     4T  4343.8 1540.9 1558.3 1389.5  86.6  48.4   27529.5  5549.8 3338.5

 64 Bit
 3B  1T   979.8  330.4  322.9  281.5  20.0  10.8 1368033.3  2335.5 1177.1
     4T  3914.5 1206.2 1295.8 1122.2  78.3  44.3 3007162.3  9230.2 4636.6
                                                 #########
 3B+ 1T  1151.6  383.0  382.7  327.6  23.2  13.0 1717931.5  2720.5 1364.5
     4T  4579.6 1505.5 1525.7 1304.4  92.0  51.6 4647842.5 10777.1 5448.5

   ######### over optimised but little effect on MWIPS rating


MP-Dhrystone Benchmark

This runs multiple copies of the whole program at the same time. Dedicated data arrays are used for each thread but there are numerous other variables that are shared. The latter can reduce performance gains via multiple threads and, in some cases, these can be slower than using a single thread (not in this case).

Comparisons are shown below, as expected 3B+/3B, MP performance not good, 64 bit faster than 32 bit - additional registers provide more options for optimisation (or over optimisation for which this benchmark is notorious).

Code:
Rating in VAX MIPS aka DMIPS

Threads        1        2        4        8    4T/1T
32 Bit
3B          2338     3956     5730     5780     2.45
3B+         2694     4720     6716     6729     2.49
3B+/3B      1.15     1.19     1.17     1.16

64 Bit
3B          3622     5752     6290     6359     1.74
3B+         4198     6158     7354     7411     1.75
3B+/3B      1.16     1.07     1.17     1.17

64/32 bit            
3B          1.55     1.45     1.10     1.10
3B+         1.56     1.30     1.09     1.10


MP-Linpack Benchmark

The original Linpack benchmark, operates on double precision floating point 100x100 matrices (N = 100). This version uses NEON intrinsic functions with single precision calculations. It is run run on 100x100, 500x500 and 1000x1000 matrices and includes a test without threading overheads. The benchmark was produced to demonstrate that the original Linpack 100x100 code could not be converted (by me) to show increased performance using multiple threads. The official line is that users are allowed to use their own linear equation solver for this purpose. These Raspbian tests were carried out under the later Stretch release.

Performance can vary somewhat with this benchmark but reflects the usual 3B+ speed gains, at least on averaging all results. On the same basis, average 64 bit speeds are suggested as being the same as those at 32 bits, but some indicate slower performance. Similar performance could be expected as the compiled code is derived from high level NEON SIMD vector functions.

The poor performance, even using a single thread, is due to the frequent starting and stopping of threads to execute the critical calculations. Consistent threaded speed indicates shared data write back to RAM dependency. This probably increases with larger matrices as more calculations are carried out during a threaded function call.

Code:
Threads      None        1        2        4

32 bit 3B
 N  100     542.22    61.00    60.67    60.74
 N  500     480.55   311.06   316.00   303.48
 N 1000     364.07   272.49   231.10   232.07

32 bit 3B+
 N  100     633.11    70.82    70.13    70.20
 N  500     505.37   323.24   326.81   327.73
 N 1000     378.29   337.34   337.01   337.80

64 bit 3B
 N  100     551.48    87.43    81.66    82.68
 N  500     359.51   258.43   242.92   255.61
 N 1000     296.11   281.75   279.20   282.71

64 bit 3B+
 N  100     639.82   100.30    95.24    95.25
 N  500     430.41   292.80   291.12   290.04
 N 1000     349.47   313.59   312.38   313.40


MP-BusSpd Benchmark

This runs integer read only tests using caches and RAM, each thread accessing the same data sequentially, in this case, each thread starts reading from different addresses. To start with, data is read with large address increments to demonstrate burst data transfers. Below are results for reading all data.

Most striking is the poor relative performance of the 64 bit version, with data from L1 caches. this is due to the compile options used, with the 32 bit program using four way multiple data vector instructions, but with the 64 bit compiler producing single data scalar. Using the later gcc 7 compiler made no difference.
The usual 3B+/3B gains can be assumed to apply, along with 4 core / 1 core throughput increases from cache based data, with those from RAM indicting some improvement.

Code:
MB/Second Reading All Data, 1, 2, 4, 8 Threads

 KB and         32 Bit          64 Bit
 Theads       3B     3B+      3B     3B+     

 12.3  1T   4223    4928    1492    1731
       2T   8159    9648    2928    3404
       4T  15720   18252    5346    5524
       8T  13789   16504    5016    5780
 122.9 1T   3934    4565    1470    1694
       2T   7686    8962    2932    3408
       4T  13860   15935    5261    6781
       8T  13125   15826    5271    6250
 12288 1T   1906    2085    1024    1048
       2T   1978    2126    1590    2025
       4T   2135    2338    1546    1623
       8T   2158    2488    1821    2018


MP-RandMem Benchmark

The benchmark has cache and RAM read only and read/write tests using sequential and random access, each thread accessing the same data but starting at different points. Although performance via the L1 cache, L2 cache and RAM can be different, it is normally consistent, in each of these areas, during read/write tests.

On all tests, 3B+/3B performance gains were as expected for cache based results, with averages between 1.6 and 1.7, with RAM performance being similar. Read only MP gains were mainly greater than 3.5 times for cache tests, except on random access to L2, at around 2.4 times, understandably lower using a shared cache. There were also some MP increased throughput using RAM. Raspbian based results indicate slightly improved performance over those using 64 bit Gentoo.

Code:

                MB/Second Using 1, 2, 4 and 8 Threads

            3B 32 Bit                         3B+ 32 Bit

      KB    SerRD   SerRW   RndRD   RndRW     SerRD   SerRW   RndRD   RndRW

 12.3 1T     4078    3814    4018    3798      4747    4447    4776    4435
      2T     8045    3768    8043    3777      9253    4362    9378    4362
      4T    15622    3724   15625    3730     18114    4343   18080    4322
      8T    15208    3723   15020    3724     17813    4345   17788    4321
122.9 1T     3289    3393     827     891      3871    3893     948    1016
      2T     6556    3379    1512     880      7612    3954    1742    1021
      4T    12125    3364    2078     886     14399    3929    2383    1025
      8T    12309    3364    2042     886     14089    3935    2367    1023
12288 1T     1669     878      65      64      1850     860      67      68
      2T     3485     872     121      65      3670     867     126      67
      4T     4296     876     146      65      4097     874     146      68
      8T     2435     878     147      65      2919     873     148      68

            3B 64 Bit                         3B+ 64 Bit

 12.3 1T     4260    3071    4261    3081      4939    3573    4941    3574
      2T     7500    3054    7496    3059      8730    3553    8704    3545
      4T    15092    3018   14794    3019     17121    3499   17197    3498
      8T    14315    2977   14544    2989     16685    3454   17097    3471
122.9 1T     3385    2861     867     837      3936    3347    1014     975
      2T     6323    2653    1543     838      7334    3344    1794     976
      4T    10638    2873    2009     835     12475    3333    2281     973
      8T    10810    2841    1947     834     12261    3314    2323     974
12288 1T     1607     746      71      60      1921     793      77      63
      2T     1605     696     123      59      1896     784     113      58
      4T     1939     766     129      58      1621     707     126      58
      8T     1682     681     141      58      1302     764     106      57


MP-MFLOPS Benchmarks

The benchmarks carry out calculations using 2 and 32 floating point operations per data word read. The latter is of the form form x[i] = (x[i] + a) * b - (x[i] + c) * d + (x[i] + e) * f + ............), to take advantage of linked multiply and add instructions that, with SIMD, could produce 32 single precision MFLOPS per MHz (4 words at a time x 4 cores x 2 for linked instructions). The same benchmark, compiled with gcc Linux, obtains 23 MFLOPS/MHz on an Intel Core i7 four core processor, using SSE instructions, but it is not clear what theoretical sustained rate that could be obtained on a Raspberry Pi 3.

There are three each of 32 bit and 64 bit programs provided, Single Precision (SP), Double Precision (SP) and one using NEON Intrinsic Functions (SP). A fourth variation was produced using OpenMP, that also includes tests using 8 operations per word. Each thread uses the same calculations but accessing different segments of the data. The program checks for consistent numeric results, primarily to show that all calculations are carried out and can be run. The standard format is shown below.

Code:

           MP-MFLOPS Gentoo RPi 3B+ 64 Bit

   MP-MFLOPS armv8 64Bit Wed Apr 25 10:22:43 2018

   FPU Add & Multiply using 1, 2, 4 and 8 Threads

        2 Ops/Word              32 Ops/Word
 KB     12.8     128   12800    12.8     128   12800
 MFLOPS
 1T      824     763     380    1793    1795    1704
 2T     1620    1577     404    3584    3588    3324
 4T     2093    2933     387    6981    6843    4189
 8T     2481    2263     409    6851    6391    2944
 Results x 100000
 1T    76406   97075   99969   66015   95363   99951
 2T    76406   97075   99969   66015   95363   99951
 4T    76406   97075   99969   66015   95363   99951
 8T    76406   97075   99969   66015   95363   99951

         End of test Wed Apr 25 10:22:48 2018


Calculations show that the usual 3B+ performance gains were produced on all benchmarks and that maximum performace was always using 32 operations per word. Indications of maximum performance from the various benchmarks is provided below, where most MP gains are reasonable. Best case was using NEON Intrinsic Functions at 8.26 MFLOPS/MHz (a long way from 32). Note the 64/32 bit speed gains with the original benchmarks. These are typical for all tests, excluding those wit RAM based data.

The original 32 bit versions converted the 32 operations into 22 scalar instructions, with 10 fused multiply accumulate type. The 64 bit variations unrolled the execution loop to include 128 operations, comprising 22 four way vector instructions, but speed of the original versions were degraded through using too few registers with too many register loads. A new 64 bit program is now provided, compiled with gcc 7, and this provides the same order of performance as the NEON benchmark, using 25 vector registers (4 way SP) with out of loop loads.

Code:
       Raspberry Pi 3B+ MFLOPS at 32 Operations Per Data Word   

                                         NEON               64 bit OpenMP
              32 bit        64 bit      64 bit  64 bit gcc7  gcc6   gcc7
              SP     DP     SP     DP     SP     SP     DP     SP     SP
                              
 1 Thread    813    798   1793   1405   2999   2800   1403   1692   2781-@
 4 Threads  3189   3109   6981   4398  11563  10608   4492   6469  10007
 4T/1T      3.92   3.90   3.89   3.13   3.86   3.79   3.20   3.82   3.60


OpenMP Benchmarks

These benchmarks use OpenMP complier directives and programming options to automatically spread the load over multiple CPUs or Cores. Two benchmarks are provided, one named notOpenMP-name, that is compiled from the same code without using OpenMP options, to provide single core performance.

OpenMP-MFLOPS Benchmark

This benchmark carries out the same calculations as the MP-MFLOPS Benchmarks but, in addition, calculations with eight operations per data word. As indicated in the above table, recompiling with gcc 7 produced similar performance as the best C version. Note, these benchmarks are from the same code as the Linux PC versions, with higher memory demands than MP-MFLOPS, starting at 400 KB, L2 cache size. Below are single and multi-core 64 bit 3B+ results, indicating that MP performance was much slower when RAM data was involved.

Comparisons are also provided, indicating the usual 3B+/3B performance gains, at least on using a single core. Respectable MP gains were obtained at 64 bits but less so at 32 bits. The latter leads to indications that 64 bit working is faster on multi-tasking, but the detail shows that 32 bit speeds are faster using a single core.

Code:

            notOpenMP MFLOPS64 GCC7 Mon Jul  9 11:28:16 2018

  Test             4 Byte  Ops/   Repeat    Seconds   MFLOPS       First   All
                    Words  Word   Passes                         Results  Same

 Data in & out     100000     2     2500   0.646220      774    0.929538   Yes
 Data in & out    1000000     2      250   1.199940      417    0.992550   Yes
 Data in & out   10000000     2       25   1.158499      432    0.999250   Yes

 Data in & out     100000     8     2500   1.049060     1906    0.957117   Yes
 Data in & out    1000000     8      250   1.403440     1425    0.995518   Yes
 Data in & out   10000000     8       25   1.375663     1454    0.999549   Yes

 Data in & out     100000    32     2500   2.876464     2781    0.890215   Yes
 Data in & out    1000000    32      250   3.176167     2519    0.988088   Yes
 Data in & out   10000000    32       25   3.171387     2523    0.998796   Yes

                End of test Mon Jul  9 11:28:33 2018

           OpenMP MFLOPS64 GCC7 Mon Jul  9 11:25:46 2018


 Data in & out     100000     2     2500   0.250953     1992    0.929538   Yes
 Data in & out    1000000     2      250   1.185075      422    0.992550   Yes
 Data in & out   10000000     2       25   1.194995      418    0.999250   Yes

 Data in & out     100000     8     2500   0.332016     6024    0.957117   Yes
 Data in & out    1000000     8      250   1.182464     1691    0.995518   Yes
 Data in & out   10000000     8       25   1.157733     1728    0.999549   Yes

 Data in & out     100000    32     2500   0.799423    10007    0.890215   Yes
 Data in & out    1000000    32      250   1.260969     6344    0.988088   Yes
 Data in & out   10000000    32       25   1.196507     6686    0.998796   Yes

                End of test Mon Jul  9 11:25:55 2018

                                Comparisons

              Raspbian 32 Bit      Gentoo 64 Bit        Gentoo 64 Bit gcc 7      
                3B    3B+  3B+/3B    3B    3B+  3B+/3B    3B    3B+  3B+/3B
 8 Ops/Wd
 1 Core       1723   2005   1.16   1623   1892   1.17   1654   1906   1.15
 4 Cores      4485   5071   1.13   5713   6504   1.14   5893   6024   1.02
 4/1 Cores    2.60   2.53          3.52   3.44          3.56   3.16   

 32 Ops/Wd
 1 Core       1592   1857   1.17   1461   1692   1.16   2395   2781   1.16
 4 Cores      1858   2420   1.30   5493   6469   1.18   8722  10007   1.15
 4/1 Cores    1.17   1.30          3.76   3.82          3.64   3.60


OpenMP-MemSpeed Benchmark

This is the same as the single core Memory Speed Benchmark, with similar results to NotOpenMP varieties, but with measurements extending to test more memory, with 16 sets of results between 4 KB and 128 MB. Floating point and integer calculations are carried out as indicated in the table headings. Samples of results are provided below, covering L1 cache, L2 cache and RAM based data.

Comparing characteristics provides wide ranges of variations. The 3B+/3B ratio was almost entirely in the expected 1.6 to 1.7 ratio for L1 cache data, but with some significant differences via L2 cache and RAM. On comparing MP performance, the most noticeable were many at half speed with integer arithmetic. Then, 32 bit Raspbian produced more than three times increase on double precision calculations and twice at single precision. The 64 bit Gentoo floating MP gains were around 1.5 times. These strange results were highlighted in 64/32 bit comparisons, where the former tended to be faster standalone but slower in MP mode. Alternative compilations might be of interest.

Code:
  Memory   x[m]=x[m]+s*y[m] Int+   x[m]=x[m]+y[m]         x[m]=y[m]
  KBytes    Dble   Sngl  Int32   Dble   Sngl  Int32   Dble   Sngl  Int32
    Used    MB/S   MB/S   MB/S   MB/S   MB/S   MB/S   MB/S   MB/S   MB/S

                                 Raspbian 32 Bit
Pi 3B Not
       8    1594   2547   3811   2388   3469   3812   2717   2716   2716
      16    1595   2553   3825   2393   3478   3825   2728   2728   2728
      64    1508   2300   3304   2177   3065   3303   2542   2485   2485
     128    1515   2305   3353   2183   3108   3356   2644   2573   2574
   32768     928   1166   1206   1119   1224   1206    760    746    746
   65536     970   1171   1210   1140   1225   1212    811    810    808
Pi 3B OMP
       8    6068   3107   1382  10109   5056   1486  16438   8104   1258
      16    5739   3119   1317  10193   5114   1393  16624   7862   1220
      64    5416   3055   1303   8618   4928   1403  12254   8045   1218
     128    5396   3050   1359   9101   4932   1379   9496   8089   1249
   32768    3351   2499   1080   2089   3216   1437   1005   1001    794
   65536    3820    614   1026   3901   3078   1209   1006   1008    954

Pi 3B+ Not
       8    1860   2973   4449   2787   4047   4447   3169   3169   3170
      16    1862   2978   4463   2791   4058   4462   3137   3183   3182
      64    1753   2664   3806   2526   3541   3805   3016   3028   3046
     128    1776   2721   3968   2574   3683   3965   3073   2989   2989
   32768     939   1218   1205   1192   1219   1213    909    864    864
   65536    1052   1221   1216   1197   1214   1213    905    870    869

Pi 3B+ OMP
       8    6142   3447   1613  11721   5857   1721  19080   9581   1465
      16    6315   3608   1568  11687   5911   1699  19295   9528   1468
      64    5795   3522   1456  10640   5718   1627  14169   9392   1397
     128    5657   3542   1485  10485   5713   1588  11330   9181   1398
   32768    1388   2107   1090   4044   3099   1355    941    933   1358
   65536    2576   1750   1159   2296   3825   1661    943    971   1509

                               Gentoo 64 Bit
Pi 3B Not
       8    3935   2507   4155   5339   3399   4168   4646   3730   3732
      16    3857   2506   4171   5358   3417   4155   4680   3744   3751
      64    3074   2242   3416   3956   2940   3438   4432   3635   3618
     128    3042   2274   3495   3933   3008   3492   4209   3488   3498
   32768     529   1059    939   1106    975    898    695    701    495
   65536    1095    986   1017    990   1104    694    682    686    625
Pi 3B OMP
       8    6064   3205   1690   9310   4858   1824  10003   5704   2052
      16    3584   3228   1634   8855   4902   1742  10164   5598   1856
      64    3697   2665   1563   8723   4798   1655  10674   5331   1758
     128    2947   3206   1672   8900   4841   1618   9060   5456   1768
   32768     605    792   1562   1143   2709   1019    899    861    591
   65536    1312    555   1163   3974    803   1181    908    833    780

Pi 3B+ Not
       8    4625   2906   4828   6167   3957   4828   5384   4307   4318
      16    4624   2902   4840   6203   3954   4838   5422   4347   4332
      64    3857   2704   4232   4873   3589   4202   5002   4121   4122
     128    3899   2760   4371   4906   3720   4373   5030   4079   4111
   32768     498   1107   1064   1197   1202   1135    799    649    763
   65536    1176   1060   1165   1177   1203   1230    588    959    963
Pi 3B+ OMP
       8    7385   3728   1976   8941   5652   2107  13290   6637   2392
      16    7099   3769   1945  11092   5751   2105  11593   2798    993
      64    1432   1264    801   4343   2345    852   3740   2531    914
     128    4476   3711   1812   9677   5654   2070   4091   2665    846
   32768     777   1994   1471   1223   1269   1800    958    945    790
   65536    1736   1275   1271   1006   1249   1787    869    980   1293

_________________
Regards

Roy
Back to top
View user's profile Send private message
orion777
Apprentice
Apprentice


Joined: 15 Mar 2017
Posts: 207
Location: Riga, Latvia

PostPosted: Sun Jul 29, 2018 8:02 am    Post subject: Reply with quote

orion777 wrote:
Good day! Is it possible to install net-wireless/hostapd ? The problem is that it forses -bindist flag for openssl, dev-embedded/rpi3-64bit-meta-1.2.2-r6 and so...

So, I was creating an overlay based on hostapd version 2.6-r6 and call it 2.6-r10. In this overlay I delete [-bindist] requirement from openssl :oops: At least for now it is working, but will see..
Back to top
View user's profile Send private message
orion777
Apprentice
Apprentice


Joined: 15 Mar 2017
Posts: 207
Location: Riga, Latvia

PostPosted: Sun Jul 29, 2018 8:06 am    Post subject: Reply with quote

Maybe someone know, how to send shutdown command to the rpi3 if there are no ssh and attached usb keyboard? Because I have one raspberry as a NAT server, like "plug and it will boot", but how to stop them?
Back to top
View user's profile Send private message
Sakaki
Guru
Guru


Joined: 21 May 2014
Posts: 409

PostPosted: Mon Jul 30, 2018 6:56 pm    Post subject: Reply with quote

All,

Apologies, I am currently travelling with only very intermittent internet access; I'll be able to respond to support questions again upon my return (last week of August).
_________________
Regards,

sakaki
Back to top
View user's profile Send private message
Goverp
Veteran
Veteran


Joined: 07 Mar 2007
Posts: 1972

PostPosted: Sun Aug 12, 2018 7:31 pm    Post subject: Reply with quote

orion777 wrote:
Maybe someone know, how to send shutdown command to the rpi3 if there are no ssh and attached usb keyboard? Because I have one raspberry as a NAT server, like "plug and it will boot", but how to stop them?

Depending on what you're trying to do, you can Google for using the GPIO pins and a push button switch to provide a shutdown button. And indeed a start button that will boot from the shutdown state. Or 3-position toggle to do both. It's traditional on IBM mainframes for the power-off button to be huge and red and never used.
_________________
Greybeard
Back to top
View user's profile Send private message
Goverp
Veteran
Veteran


Joined: 07 Mar 2007
Posts: 1972

PostPosted: Mon Aug 13, 2018 8:07 am    Post subject: [EDIT] F2FS problem - or maybe not! Reply with quote

[Edit]
Having reformatted the SD card to use ext4, it's showing the same errors, so it looks like the card is dead, rather then the filesystem. Ho hum; that'll teach me not to buy the cheapest cards.
[/Edit]

I've no-one to blame but myself, but, as f2fs worked OK for me on 32-bit Arch Pi3, when it came to 64bit Gentoo, I modified the installation to use it for the root partition. Generally seems to work, but an "emerge --update", something nasty happens after about 20 packages. Kernel messages and a semi-hang (can move among TTYs, but can't reboot or run any commands), specifically. I've updated to the latest kernel tarball; still happens. Just thought I'd warn people.

Anyway, many thanks for making all this available Sakaki.
_________________
Greybeard
Back to top
View user's profile Send private message
Goverp
Veteran
Veteran


Joined: 07 Mar 2007
Posts: 1972

PostPosted: Wed Aug 15, 2018 11:29 am    Post subject: Reply with quote

Sakaki, a suggestion:
Would it be easy to build a squashfs image of /usr/portage as part of your weekly process? Emerge --sync updates a vast number of small files, exactly the wrong thing for an SD card. Downloading a new squashfs image would probably be quicker, and surely better for the card. And it would save lots of space, as the tree compresses really well, and it might reduce load on your rsync server.

To make it work sensibly, it would need moving the packages and distfiles directories out to somewhere in /var (possibly /var/cache), but they should have been there anyway if the early portage designers had thought for a bit.
_________________
Greybeard
Back to top
View user's profile Send private message
Sakaki
Guru
Guru


Joined: 21 May 2014
Posts: 409

PostPosted: Tue Aug 21, 2018 5:45 pm    Post subject: Reply with quote

Goverp wrote:
Sakaki, a suggestion:
Would it be easy to build a squashfs image of /usr/portage as part of your weekly process? Emerge --sync updates a vast number of small files, exactly the wrong thing for an SD card. Downloading a new squashfs image would probably be quicker, and surely better for the card. And it would save lots of space, as the tree compresses really well, and it might reduce load on your rsync server.

To make it work sensibly, it would need moving the packages and distfiles directories out to somewhere in /var (possibly /var/cache), but they should have been there anyway if the early portage designers had thought for a bit.

Thanks for the suggestion, I'll have a look into it. The problems are more likely to be making it reliable on the client (rather than the server) end I guess, as you'd need something to download and signature verify the image on a weekly sync, then dismount the old image, mount the new one (there'd be no point just using unsquashfs or similar, since that'd still have the small file writes), bind mount packages and distfiles into it and then clean up (optionally) past images that were too old. Do you know by any chance if anyone has written something similar for Portage in the past?
_________________
Regards,

sakaki
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 54098
Location: 56N 3W

PostPosted: Tue Aug 21, 2018 7:10 pm    Post subject: Reply with quote

Sakaki,

Would it help to start here for your weekly updates?

Move ./packages and ./distfiles out of /usr/portage, so you don't have the bind mount.
Then its throw one file away and install another.

Unless you did some auto updates to make.conf you would need a symlink pointing to the snapshot of the week, like linux points to a kernel whose name keeps changing.

eselect snapshot perhaps. That would allow for keeping several snapshots around at no extra charge.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
antonlacon
Apprentice
Apprentice


Joined: 27 Jun 2004
Posts: 257

PostPosted: Wed Aug 22, 2018 1:02 am    Post subject: Reply with quote

There's a post sync hook you can use to build the squashfs tree by putting a script in portage/repo.postsync.d/ I have a sample here I run on my shared server back when I was serving up the portage tree to some clients here: https://github.com/antonlacon/gentoo/blob/master/portage/repo.postsync.d/squashfs-portage.sh Ignore most of the timestamp stuff (the timestamp comparison was to prevent building the same package when no sync actually took place, but had a few gaps in whether its test had the correct result), but the general approach might help you along.

There was also an old thread out of f.g.o's tips and tricks section about a squashfs portage tree and a unionfs overlay to enable syncing on top of it. I believe this was the initial thread: https://forums.gentoo.org/viewtopic-t-465367.html I had my own take on that approach here: https://github.com/antonlacon/gentoo/blob/master/initscripts/ (the squash-portage files in each directory). I'll offer that I haven't used said scripts in years (maybe a decade), and that looking at them now makes me think it's some fairly complicated and ugly shell, but maybe it's helpful anyway. I don't recall if a union fs ever got merged into the kernel.
Back to top
View user's profile Send private message
Display posts from previous:   
This topic is locked: you cannot edit posts or make replies.    Gentoo Forums Forum Index Gentoo on ARM All times are GMT
Goto page Previous  1, 2, 3 ... 14, 15, 16 ... 18, 19, 20  Next
Page 15 of 20

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum