Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Gentoo for Amlogic S9xx (TV box S905\S905X\S912)
View unanswered posts
View posts from last 24 hours

Goto page Previous  1, 2, 3, 4, 5, 6, 7, 8, 9, 10  Next  
Reply to topic    Gentoo Forums Forum Index Gentoo on ARM
View previous topic :: View next topic  
Author Message
erm67
l33t
l33t


Joined: 01 Nov 2005
Posts: 653
Location: EU

PostPosted: Wed Feb 01, 2017 6:51 am    Post subject: Reply with quote

mDup wrote:
erm67 wrote:
[...]
for linux 4.10 I just followed the recipe at http://linux-meson.com/doku.php; make defconfig will enable all amlogic drivers, the problem is that the dtb are different and the ones we're using do not work (probably), also you really need a serial terminal to boot it, otherwise there is only a black screen, the linux-meson guys are working already on hdmi support.
I am thinking about merging the existing dtb with the new ones shipped with linux 4.10, mostly to partition the emmc drive with the android partitioning scheme, at least until linux supports the board fully.

Thanks for pointing to http://linux-meson.com/
just for all clarity
1) this is just unaltered linux 4.x kernel code right (y/n)?
2) does HDMI work on none or only not on your box?
[I would prefer to wait for hdmi support ...]


vanilla 4.10 code, all the work is being done on mainline kernel.
what they have working is the gxbb_200 amlogic reference board that is mounted in a lot of S905 TV boxes & hardkernel Odroid
the HDMI driver does not exist yet for any reference board

If you notice in the recipe to compile the kernel at http://linux-meson.com/ they don't do a make modules && make modules_install, I guess it's because in the defconfig all amlogic modules are compiled in kernel and enough to boot and they do not expec users to do a lot more than booting it at this point :-)
_________________
Ok boomer
True ignorance is not the absence of knowledge, but the refusal to acquire it.
Ab esse ad posse valet, a posse ad esse non valet consequentia

My fediverse account: @erm67@erm67.dynu.net
Back to top
View user's profile Send private message
mDup
Apprentice
Apprentice


Joined: 14 Apr 2006
Posts: 212

PostPosted: Wed Feb 01, 2017 1:21 pm    Post subject: Reply with quote

erm67 wrote:
[...]
I mount all write intensive dirs over iscsi to spare the internal ssd, the eth is really fast for such a small board, only slightly slower than the eMMC ... and full duplex unlike the USB2 ports.
[...]

also thanks for suggesting network drive.
I tried NFS and CIFS but had some issues. Next I found gentoo iSCSI Initiator for client (but no iSCSI Target for my gentoo file servers).
So now I use NBD network block device which gives fast results
Code:
minimx-g ~ # hdparm -t /dev/nbd0
/dev/nbd0:
 Timing buffered disk reads: 264 MB in  3.05 seconds =  86.59 MB/sec
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 54216
Location: 56N 3W

PostPosted: Wed Feb 01, 2017 1:31 pm    Post subject: Reply with quote

mDup,

Its -T that matters.

Try with
Code:
hdparm -tT /dev/nbd0

_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
mDup
Apprentice
Apprentice


Joined: 14 Apr 2006
Posts: 212

PostPosted: Wed Feb 01, 2017 1:57 pm    Post subject: Reply with quote

NeddySeagoon wrote:
mDup,
Its -T that matters.
Try with
Code:
hdparm -tT /dev/nbd0

this is while system is emerging gcc (which I do not want to interrupt) with -j3 makeflag so perhaps not best benchmark figures
Code:
minimx-g ~ # hdparm -tT /dev/nbd0
/dev/nbd0:
 Timing cached reads:   1420 MB in  2.00 seconds = 710.01 MB/sec
 Timing buffered disk reads: 222 MB in  3.00 seconds =  73.97 MB/sec
minimx-g ~ # hdparm -tT /dev/nbd0
/dev/nbd0:
 Timing cached reads:   1492 MB in  2.00 seconds = 745.55 MB/sec
 Timing buffered disk reads: 214 MB in  3.01 seconds =  71.16 MB/sec
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 54216
Location: 56N 3W

PostPosted: Wed Feb 01, 2017 1:59 pm    Post subject: Reply with quote

mDup,

That's seriously good. I wish I could get that on a Raspberry Pi.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
mDup
Apprentice
Apprentice


Joined: 14 Apr 2006
Posts: 212

PostPosted: Wed Feb 01, 2017 2:04 pm    Post subject: Reply with quote

NeddySeagoon wrote:
mDup,
That's seriously good. I wish I could get that on a Raspberry Pi.

It is GigEth switched network, and minimx-g has GigEth.


Last edited by mDup on Wed Feb 01, 2017 2:28 pm; edited 1 time in total
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 54216
Location: 56N 3W

PostPosted: Wed Feb 01, 2017 2:16 pm    Post subject: Reply with quote

mDup,

That's fairly close to the head/platter data rate on 'rotating rust' hard drives.
That's the limit on sustained reads/writes. Transfers smaller than the cache size avoid that limit.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
erm67
l33t
l33t


Joined: 01 Nov 2005
Posts: 653
Location: EU

PostPosted: Wed Feb 01, 2017 5:43 pm    Post subject: Reply with quote

NeddySeagoon wrote:
mDup,

That's fairly close to the head/platter data rate on 'rotating rust' hard drives.
That's the limit on sustained reads/writes. Transfers smaller than the cache size avoid that limit.


My 2016 1Tb rotating rust hdd makes >100Mb/s over sata ......

@mDup
To increase iscsi & network speed you'll need irqbalanced running to spread the interrupts more evenly

Code:
erm67 ~ # ethtool -k eth0|grep -v fixed
Features for eth0:
rx-checksumming: on
tx-checksumming: on
   tx-checksum-ipv4: on
   tx-checksum-ipv6: on
scatter-gather: on
   tx-scatter-gather: on
tcp-segmentation-offload: off
generic-segmentation-offload: on
generic-receive-offload: on
tx-nocache-copy: off

The network card is doing some HW acceleration as you can see, I was not able to turn on TSO but it's probaby not terribly useful

Also coalsce needs some tweaking for my taste:
Code:
erm67 ~ # ethtool -c eth0
Coalesce parameters for eth0:
Adaptive RX: off  TX: off
stats-block-usecs: 0
sample-interval: 0
pkt-rate-low: 0
pkt-rate-high: 0

rx-usecs: 393
rx-frames: 0
rx-usecs-irq: 0
rx-frames-irq: 0

tx-usecs: 40000
tx-frames: 64
tx-usecs-irq: 0


Adaptive is not supported, but you can tweak rx-usecs/frames tx-usecs/frames to smaller values that will make the card more 'responsive' (less buffering in the card)
There are probably a few other tcp/ip parameters that can be tweaked for gigabit iscsi.
_________________
Ok boomer
True ignorance is not the absence of knowledge, but the refusal to acquire it.
Ab esse ad posse valet, a posse ad esse non valet consequentia

My fediverse account: @erm67@erm67.dynu.net


Last edited by erm67 on Wed Feb 01, 2017 5:53 pm; edited 2 times in total
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 54216
Location: 56N 3W

PostPosted: Wed Feb 01, 2017 5:51 pm    Post subject: Reply with quote

erm67,

My 2016 2TB HDD, just fitted as a raid replacement does >100Mb/sec at the outside and struggles to get 40MB/sec on the inner tracks.
That's zoning for you.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
erm67
l33t
l33t


Joined: 01 Nov 2005
Posts: 653
Location: EU

PostPosted: Wed Feb 01, 2017 5:56 pm    Post subject: Reply with quote

NeddySeagoon wrote:
erm67,

My 2016 2TB HDD, just fitted as a raid replacement does >100Mb/sec at the outside and struggles to get 40MB/sec on the inner tracks.
That's zoning for you.

that's why serious disks are 2.5" now :-)

Quote:
[root@goflex ~]# hdparm -t /dev/mapper/nas-erm67

/dev/mapper/nas-erm67:
Timing buffered disk reads: 268 MB in 3.00 seconds = 89.28 MB/sec
[root@nas ~]# hdparm -T /dev/mapper/nas-erm67


That's very likely inner track since it's the slowest partition.
_________________
Ok boomer
True ignorance is not the absence of knowledge, but the refusal to acquire it.
Ab esse ad posse valet, a posse ad esse non valet consequentia

My fediverse account: @erm67@erm67.dynu.net
Back to top
View user's profile Send private message
mDup
Apprentice
Apprentice


Joined: 14 Apr 2006
Posts: 212

PostPosted: Wed Feb 01, 2017 7:37 pm    Post subject: Reply with quote

erm67 wrote:

@mDup
To increase iscsi & network speed you'll need irqbalanced running to spread the interrupts more evenly
[...]
There are probably a few other tcp/ip parameters that can be tweaked for gigabit iscsi.

Thanks, is irqbalance supposed to be run on client or server or both?
I ask because sys-apps/irqbalance seems to have emerge build errors on arm64 (which can be fixed easily).
Also: I run nbd not iscsi.
I'll experiment with ethtool, but right now I am already happy with speed (and sparing SD cards)
Back to top
View user's profile Send private message
erm67
l33t
l33t


Joined: 01 Nov 2005
Posts: 653
Location: EU

PostPosted: Thu Feb 02, 2017 11:14 am    Post subject: Reply with quote

mDup wrote:
erm67 wrote:

@mDup
To increase iscsi & network speed you'll need irqbalanced running to spread the interrupts more evenly
[...]
There are probably a few other tcp/ip parameters that can be tweaked for gigabit iscsi.

Thanks, is irqbalance supposed to be run on client or server or both?
I ask because sys-apps/irqbalance seems to have emerge build errors on arm64 (which can be fixed easily).
Also: I run nbd not iscsi.
I'll experiment with ethtool, but right now I am already happy with speed (and sparing SD cards)


on the minimx (and the server if it has multiple cpu), it's because without it cpu0 gets all eth interrupts, with it the load will be spread among multiple CPUs.
Read this if you want to play a bit with gigabit networking:
https://access.redhat.com/sites/default/files/attachments/20150325_network_performance_tuning.pdf

yes, nbd is a good alternative, ata over ethernet unfortunately is in a bad bad shape. iscsi on linux is probably the fastest since it's used by several companies, that provides patches.


Can you post the results of two ethtool commands on your system?

Quote:
tx-usecs: 40000
tx-frames: 64

This coalesce values doesn't makes sense, or maybe makes sense on a device that is supposed to play network streams but not transmit anything, beelink/Amlogic engineers probably set those values optimized for a streaming device directy in the device driver, but since we are using it for other purposes probably need some tuning.

With the default settings there's a moderate packet loss during high network activity, that improves noticeably tweaking the card, read the guide.
_________________
Ok boomer
True ignorance is not the absence of knowledge, but the refusal to acquire it.
Ab esse ad posse valet, a posse ad esse non valet consequentia

My fediverse account: @erm67@erm67.dynu.net
Back to top
View user's profile Send private message
mDup
Apprentice
Apprentice


Joined: 14 Apr 2006
Posts: 212

PostPosted: Thu Feb 02, 2017 4:53 pm    Post subject: Reply with quote

erm67 wrote:
mDup wrote:
erm67 wrote:

@mDup
To increase iscsi & network speed you'll need irqbalanced running to spread the interrupts more evenly
[...]
There are probably a few other tcp/ip parameters that can be tweaked for gigabit iscsi.

Thanks, is irqbalance supposed to be run on client or server or both?
I ask because sys-apps/irqbalance seems to have emerge build errors on arm64 (which can be fixed easily).
Also: I run nbd not iscsi.
I'll experiment with ethtool, but right now I am already happy with speed (and sparing SD cards)


on the minimx (and the server if it has multiple cpu), it's because without it cpu0 gets all eth interrupts, with it the load will be spread among multiple CPUs.
Read this if you want to play a bit with gigabit networking:
https://access.redhat.com/sites/default/files/attachments/20150325_network_performance_tuning.pdf

yes, nbd is a good alternative, ata over ethernet unfortunately is in a bad bad shape. iscsi on linux is probably the fastest since it's used by several companies, that provides patches.


Can you post the results of two ethtool commands on your system?

Quote:
tx-usecs: 40000
tx-frames: 64

This coalesce values doesn't makes sense, or maybe makes sense on a device that is supposed to play network streams but not transmit anything, beelink/Amlogic engineers probably set those values optimized for a streaming device directy in the device driver, but since we are using it for other purposes probably need some tuning.

With the default settings there's a moderate packet loss during high network activity, that improves noticeably tweaking the card, read the guide.


thanks for kind help

here are some command outputs
Code:
minimx-g ~ # ethtool -k eth0|grep -v fixed
Features for eth0:
rx-checksumming: on
tx-checksumming: on
        tx-checksum-ipv4: on
        tx-checksum-ipv6: on
scatter-gather: on
        tx-scatter-gather: on
tcp-segmentation-offload: off
generic-segmentation-offload: on
generic-receive-offload: on
tx-nocache-copy: off
minimx-g ~ # ethtool -c eth0
Coalesce parameters for eth0:
Adaptive RX: off  TX: off
stats-block-usecs: 0
sample-interval: 0
pkt-rate-low: 0
pkt-rate-high: 0

rx-usecs: 393
rx-frames: 0
rx-usecs-irq: 0
rx-frames-irq: 0

tx-usecs: 40000
tx-frames: 64
tx-usecs-irq: 0
tx-frames-irq: 0

rx-usecs-low: 0
rx-frame-low: 0
tx-usecs-low: 0
tx-frame-low: 0

rx-usecs-high: 0
rx-frame-high: 0
tx-usecs-high: 0
tx-frame-high: 0

minimx-g ~ # rc-status | grep irqbalance
 irqbalance                                                        [  started  ]
minimx-g ~ # egrep "(eth|CPU)" /proc/interrupts
           CPU0       CPU1       CPU2       CPU3       
 40:     206511          0          0          0       GIC  40  eth0

note that while irqbalance runs, still only CPU0 gets eth0 interrupts.
Perhaps a bug.
Back to top
View user's profile Send private message
erm67
l33t
l33t


Joined: 01 Nov 2005
Posts: 653
Location: EU

PostPosted: Thu Feb 02, 2017 6:25 pm    Post subject: Reply with quote

mDup wrote:


minimx-g ~ # rc-status | grep irqbalance
irqbalance [ started ]
minimx-g ~ # egrep "(eth|CPU)" /proc/interrupts
CPU0 CPU1 CPU2 CPU3
40: 206511 0 0 0 GIC 40 eth0
[/code]
note that while irqbalance runs, still only CPU0 gets eth0 interrupts.
Perhaps a bug.


Yeah, I followed the (very interesting and well written) guide a few of weeks ago apparently only irqbalance 1.1.0 has some ARM SoC support ... let me check again, for me was just an habit havint it running on all multicore machines :-)
I found this script that parses /proc/net/softnet_stat and prints the stats in a readable form:
https://github.com/netoptimizer/network-testing/blob/master/bin/softnet_stat.pl

and now after some tweaking under heavy load I see that a second CPU is used, but still 90% of the load goes to CPU0 don't remember exactly maybe it wasn't irqbalance .... but something else from that guide for sure.

Also the netstat -s improved a lot, but TBH drops and retransmission increase only using bonnie over iscsi (extreme load), under normal load it's fine ...... and tuning the coalesce values to around 300 usecs both for read and write was the most effective tweak. It also makes the web server a lot more responsive, but probably it is not something you are looking for :-)

I use it as a mail and web server mainly ......

Really impressive this little rig anyway, until a few years ago most of those features were only available on high end hardware ....
_________________
Ok boomer
True ignorance is not the absence of knowledge, but the refusal to acquire it.
Ab esse ad posse valet, a posse ad esse non valet consequentia

My fediverse account: @erm67@erm67.dynu.net
Back to top
View user's profile Send private message
mDup
Apprentice
Apprentice


Joined: 14 Apr 2006
Posts: 212

PostPosted: Sun Feb 05, 2017 2:01 am    Post subject: Re: gentoo s905x image Reply with quote

mDup wrote:
https://yadi.sk/d/8WUMwurb38CYTi
Gentoo_8gb_20170128.img.xz
md5sum da1c74abca86f07e866dac6cced01f7c

This gentoo system image and kernel are 100% natively built on minimx-g s905 itself.
It has been verified to run on s905x too.

The gentoo is bleeding edge in that it accepts all keywords,
also: gcc-6.3.0 and glibc-2.24 ebuilds are hacked (KEYWORDS un-commented),
and: arm64 use.mask profile is less restrictive.
A few packages are added in top of system.

You must set date manually or emerge will complain. In root there is a README file.
To change resolution, etc.. follow same procedure as outlined by Author balbes150 in OP.

In fact: all kind help comes from him and all credits go to him.

Kernel is 150balbes github branch S905X. It is natively built with gcc-4.9.4.

Starting from this image I have built a LxQt centric Desktop without much pain.
You can use it to build any other gentoo machine you feel like building.

Kernel is work in progress. There is another branch (ordroid) in 150balbes github which may become the next kernel. But ordroid kernel is s905 -only for now.

https://yadi.sk/d/8WUMwurb38CYTi
updated with Gentoo_8gb_20170205.img.xz (md5sum 4ebf3de58b5d701148e00b51fda81545)
fix: using balbes kernel kernel includes for userland
- sound works (aplay, paplay)
- latest balbes aml_autoscript s905_autoscript
- latest balbes kernel
* added /proc/config.gz
* added nbd module
Back to top
View user's profile Send private message
erm67
l33t
l33t


Joined: 01 Nov 2005
Posts: 653
Location: EU

PostPosted: Sun Feb 05, 2017 8:18 am    Post subject: Re: gentoo s905x image Reply with quote

mDup wrote:

fix: using balbes kernel kernel includes for userland


There are always discussions on this point, and IIRC it not such a great idea, at least for systems using glibc:

Quote:
The headers from the most recent Linux kernel should be used. The
headers used while compiling the GNU C library and the kernel
binary used when using the library do not need to match. The GNU
C library runs without problems on kernels that are older than
the kernel headers used. The other way round (compiling the GNU C
library with old kernel headers and running on a recent kernel)
does not necessarily work as expected. For example you can't use
new kernel features if you used old kernel headers to compile the
GNU C library.

Even if you are using an older kernel on your machine, we
recommend you compile GNU libc with the most current kernel
headers. That way you won't have to recompile libc if you ever
upgrade to a newer kernel.


One the various things that makes glibc bloated is that it is able to detect missing features in the kernel.
_________________
Ok boomer
True ignorance is not the absence of knowledge, but the refusal to acquire it.
Ab esse ad posse valet, a posse ad esse non valet consequentia

My fediverse account: @erm67@erm67.dynu.net
Back to top
View user's profile Send private message
mDup
Apprentice
Apprentice


Joined: 14 Apr 2006
Posts: 212

PostPosted: Sun Feb 05, 2017 2:25 pm    Post subject: Re: gentoo s905x image Reply with quote

erm67 wrote:
mDup wrote:

fix: using balbes kernel kernel includes for userland


There are always discussions on this point, and IIRC it not such a great idea, at least for systems using glibc:

Quote:
The headers from the most recent Linux kernel should be used. The
headers used while compiling the GNU C library and the kernel
binary used when using the library do not need to match. The GNU
C library runs without problems on kernels that are older than
the kernel headers used. The other way round (compiling the GNU C
library with old kernel headers and running on a recent kernel)
does not necessarily work as expected. For example you can't use
new kernel features if you used old kernel headers to compile the
GNU C library.

Even if you are using an older kernel on your machine, we
recommend you compile GNU libc with the most current kernel
headers. That way you won't have to recompile libc if you ever
upgrade to a newer kernel.


One the various things that makes glibc bloated is that it is able to detect missing features in the kernel.


Thanks for advice!
In fact I did not rebuild glibc with older headers and will use recent headers when glibc needs update.
I actually only updated alsa and pulseaudio with older headers and that fixed the missing memfd syscall problem at hand.
Back to top
View user's profile Send private message
erm67
l33t
l33t


Joined: 01 Nov 2005
Posts: 653
Location: EU

PostPosted: Sun Feb 05, 2017 5:49 pm    Post subject: Reply with quote

Apparently there is a backport of the syscall for older ARM kernel .....

https://lists.debian.org/debian-kernel/2014/09/msg00193.html

I am still playing with irqbalance, the git version almost works :-)
_________________
Ok boomer
True ignorance is not the absence of knowledge, but the refusal to acquire it.
Ab esse ad posse valet, a posse ad esse non valet consequentia

My fediverse account: @erm67@erm67.dynu.net
Back to top
View user's profile Send private message
mDup
Apprentice
Apprentice


Joined: 14 Apr 2006
Posts: 212

PostPosted: Mon Feb 06, 2017 5:11 am    Post subject: Reply with quote

erm67 wrote:
Apparently there is a backport of the syscall for older ARM kernel .....
https://lists.debian.org/debian-kernel/2014/09/msg00193.html
I am still playing with irqbalance, the git version almost works :-)

It would be cool if irqbalance would work.
I have not tried (big) memfd syscall backport, I wonder which aml kernel currently did (armbian? odroid?).
I hope 4.x kernel train one day would be amlogic sufficient.
It seems the Raspberry-Pi-3 managed to do so (jump on to 4.x kernel)
Back to top
View user's profile Send private message
mDup
Apprentice
Apprentice


Joined: 14 Apr 2006
Posts: 212

PostPosted: Mon Feb 27, 2017 11:58 pm    Post subject: Reply with quote

just FYI
I got wayland weston-launch working on gentoo arm64 using fbdev backend.
Back to top
View user's profile Send private message
mDup
Apprentice
Apprentice


Joined: 14 Apr 2006
Posts: 212

PostPosted: Sat Mar 04, 2017 2:50 am    Post subject: Reply with quote

erm67 wrote:
[...]
I am still playing with irqbalance, the git version almost works :-)

I upgraded to irqbalance-1.2.0 but do not see ethernet IRQ usage balanced.
The emerge mentions about no PCI_MSI config but I do not think my aarch64 has PCI...
Back to top
View user's profile Send private message
erm67
l33t
l33t


Joined: 01 Nov 2005
Posts: 653
Location: EU

PostPosted: Sat Mar 04, 2017 10:21 am    Post subject: Reply with quote

mDup wrote:
erm67 wrote:
[...]
I am still playing with irqbalance, the git version almost works :-)

I upgraded to irqbalance-1.2.0 but do not see ethernet IRQ usage balanced.
The emerge mentions about no PCI_MSI config but I do not think my aarch64 has PCI...


Automatic irq balancing doesn't work on most arm64 SoC, but manually balancing does, I use this script:
Code:

#!/bin/sh
for i in $(gawk -F":" '/emmc/ {print $1}' </proc/interrupts); do echo 2 >/proc/irq/$i/smp_affinity; done
for i in $(gawk -F":" '/dwc_otg/ {print $1}' </proc/interrupts); do echo 4 >/proc/irq/$i/smp_affinity; done
for i in $(gawk -F":" '/eth0/ {print $1}' </proc/interrupts); do echo 8 >/proc/irq/$i/smp_affinity; done


All 3 eMMC irq will go to CPU1
All USB irq (for both ports) go to CPU2 (my dvb-t adapter generates an insane amount of interrupts)
all eth interrupts will go to CPU3
CPU0 gets all other.


It's not perfect and doesn't adapt dynamically but better than nothing ....... and iscsi is faster. Hopefully we'll get mainline kernel soon. I am reading the mailing list and they are doing a lot of progresses, beside hdmi also opengl is being ported, it looks like it will be usable soon.

Also make sure the nbd device is not using the cfq scheduler because it sucks really. deadline or noop are best, an udev rule like this could help:
Code:

ACTION=="add|change", KERNEL=='nbd[0-9]', ATTR{queue/scheduler}="deadline", ATTR{device/queue_depth}="64", ATTR{queue/nr_requests}="256", ATTR{queue/max_sectors_kb}="4096"


This will set sane interrups coalescing defaults for eth:
Code:
ACTION=="add", SUBSYSTEM=="net", KERNEL=="eth0", RUN+="/bin/ifconfig eth0 txqueuelen 2000"
ACTION=="add", SUBSYSTEM=="net", KERNEL=="eth0", RUN+="/usr/sbin/ethtool -C eth0 rx-usecs 300 tx-usecs 300 rx-frames 0 tx-frames 0"

_________________
Ok boomer
True ignorance is not the absence of knowledge, but the refusal to acquire it.
Ab esse ad posse valet, a posse ad esse non valet consequentia

My fediverse account: @erm67@erm67.dynu.net
Back to top
View user's profile Send private message
mDup
Apprentice
Apprentice


Joined: 14 Apr 2006
Posts: 212

PostPosted: Sat Mar 04, 2017 4:57 pm    Post subject: Reply with quote

erm67 wrote:
mDup wrote:
erm67 wrote:
[...]
I am still playing with irqbalance, the git version almost works :-)

I upgraded to irqbalance-1.2.0 but do not see ethernet IRQ usage balanced.
The emerge mentions about no PCI_MSI config but I do not think my aarch64 has PCI...


Automatic irq balancing doesn't work on most arm64 SoC, but manually balancing does, I use this script:
Code:

#!/bin/sh
for i in $(gawk -F":" '/emmc/ {print $1}' </proc/interrupts); do echo 2 >/proc/irq/$i/smp_affinity; done
for i in $(gawk -F":" '/dwc_otg/ {print $1}' </proc/interrupts); do echo 4 >/proc/irq/$i/smp_affinity; done
for i in $(gawk -F":" '/eth0/ {print $1}' </proc/interrupts); do echo 8 >/proc/irq/$i/smp_affinity; done


All 3 eMMC irq will go to CPU1
All USB irq (for both ports) go to CPU2 (my dvb-t adapter generates an insane amount of interrupts)
all eth interrupts will go to CPU3
CPU0 gets all other.


It's not perfect and doesn't adapt dynamically but better than nothing ....... and iscsi is faster. Hopefully we'll get mainline kernel soon. I am reading the mailing list and they are doing a lot of progresses, beside hdmi also opengl is being ported, it looks like it will be usable soon.

Also make sure the nbd device is not using the cfq scheduler because it sucks really. deadline or noop are best, an udev rule like this could help:
Code:

ACTION=="add|change", KERNEL=='nbd[0-9]', ATTR{queue/scheduler}="deadline", ATTR{device/queue_depth}="64", ATTR{queue/nr_requests}="256", ATTR{queue/max_sectors_kb}="4096"


This will set sane interrups coalescing defaults for eth:
Code:
ACTION=="add", SUBSYSTEM=="net", KERNEL=="eth0", RUN+="/bin/ifconfig eth0 txqueuelen 2000"
ACTION=="add", SUBSYSTEM=="net", KERNEL=="eth0", RUN+="/usr/sbin/ethtool -C eth0 rx-usecs 300 tx-usecs 300 rx-frames 0 tx-frames 0"


@erm67

Thanks, as always, for various hints. The irq statically spreads as you describe with your script.
I use noop for nbd io scheduler.
I had tried mainline kernel 4.10.0, but, for some reason, it does not pass slow steady teddy bear logo loop on my mini-mx-g.
I have no serial USB console port soldered so I do not know what may be wrong.
Do you have serial console? Did you try 4.10.1 already?
I had
Code:
tx-usecs: 40000
tx-frames: 64
and you set
Code:
tx-usecs: 300
tx-frames: 0
can you explain a bit for example how 300 is better than 40000?
Back to top
View user's profile Send private message
erm67
l33t
l33t


Joined: 01 Nov 2005
Posts: 653
Location: EU

PostPosted: Sat Mar 04, 2017 6:08 pm    Post subject: Reply with quote

mDup wrote:


Thanks, as always, for various hints. The irq statically spreads as you describe with your script.

irqbalanced is the user space component but it requires a kernel component that is not working on aarch64 3.x kernels .....
Adjust the script for your needs.


mDup wrote:

I had tried mainline kernel 4.10.0, but, for some reason, it does not pass slow steady teddy bear logo loop on my mini-mx-g.
I have no serial USB console port soldered so I do not know what may be wrong.
Do you have serial console? Did you try 4.10.1 already?


Nope, wait for the 4.11:
https://www.phoronix.com/scan.php?page=news_item&px=WeTek-Linux-4.11-Device-Support

Support for the wetek play (identical to my box) and for the kadash vip (identical to yours) has been submitted. It is very likely that hdmi will work for 4.11.

I have a serial soldered, it is very likely that you used the wrong dtb, you have to use the dtbs supplied with the kernel and only 4.11 will ship one for your box.
mDup wrote:

I had
Code:
tx-usecs: 40000
tx-frames: 64
and you set
Code:
tx-usecs: 300
tx-frames: 0
can you explain a bit for example how 300 is better than 40000?


Interrupt coalescing means that the network card will buffer packets to reduce the number of interrupts;
tx-usecs: 40000 and tx-frames: 64 means that it will wait 40000 useconds or until there are 64 ip frames in the buffer before sending anything, that is a 'long' time and might slow synchronous protocols down with the delayed answers.

using tx-usec=0 and rx-usecs=0 will process incoming and outgoing packets imediately but it is not a good idea because it will respond instantly but use a lot of CPU, try various values from 0 to 1000 to find the best for you. I also use it as a owncloud server so I like it to be responsive :-) Basically smaller values = more responsive & more cpu usage, larger values = more latency and less CPU usage. In practice using rx-usec=0 and tx-usec=0 it is impossible to get gigabit speed.

Also the queue length used by the linux kernel are a bit too small, try
iostat -xmt 1
and keep an eye on avgrq-sz and avgqu-sz you will see that during emerge --sync, for example, the queues will grow a lot bigger than the default 32.
_________________
Ok boomer
True ignorance is not the absence of knowledge, but the refusal to acquire it.
Ab esse ad posse valet, a posse ad esse non valet consequentia

My fediverse account: @erm67@erm67.dynu.net
Back to top
View user's profile Send private message
mDup
Apprentice
Apprentice


Joined: 14 Apr 2006
Posts: 212

PostPosted: Sun Mar 05, 2017 9:43 pm    Post subject: Gentoo_8gb_20170305 Reply with quote

updated https://yadi.sk/d/8WUMwurb38CYTi 'bleeding edge' minimal (say stage3) gentoo arm64 system s905(x) image
- latest ebuilds (glibc-2.25 gcc-6.3.0 ...)
- latest balbes 20170303 kernel (with thermal trip crash fix)
md5sum 07f70399609e982f1e068f0472f0a79d
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Gentoo on ARM All times are GMT
Goto page Previous  1, 2, 3, 4, 5, 6, 7, 8, 9, 10  Next
Page 4 of 10

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum