Waydroid support - full kernel config

Message

an7h · Post by **an7h** » Sat Oct 28, 2023 3:15 pm

I was messing around for a quite a bit to get this piece of unsupported software running under Gentoo Linux (which I emerged from the guru overlay). The biggest problem I encountered was dealing with setting up my custom kernel config correctly so that it supports functionality Waydroid expects from it. As an a easy solution you can just get the sys-kernel/gentoo-kernel or gentoo-kernel-bin package and be done with it, but I'm one of those people that likes to optimize things as much as possible with a custom kernel configuration (hence I use Gentoo). So, for those who need this piece of software, here's a list of things that should be enabled/disabled within the kernel to get Waydroid working with full functionality. Hope this will be useful to someone out there, so that they won't spend time on looking for info on what needs to be enabled/disabled to get it to work, because the official documentation doesn't mention any of this (except for like one or two options).

>=sys-kernel/gentoo-sources-6.5.7 :

Code: Select all

MEMCG [=y]
VETH [=y]
MACVLAN [=y]
VLAN_8021Q [=y]
BRIDGE [=y]

NF_CONNTRACK [=y]
IP_NF_TARGET_MASQUERADE [=y]

IP6_NF_TARGET_MASQUERADE [=y]
NETFILTER_XT_TARGET_CHECKSUM [=y]
NETFILTER_XT_MATCH_COMMENT [=y]
CHECKPOINT_RESTORE [=y] 
UNIX_DIAG [=y]
INET_DIAG [=y]
PACKET_DIAG [=y]
NETLINK_DIAG [=y]

ANDROID_BINDER_IPC [=y]
ANDROID_BINDERFS [=y]
ANDROID_BINDER_DEVICES [=binder,hwbinder,vndbinder]
ANDROID_BINDER_IPC_SELFTEST [=n] 

BLK_DEV_LOOP [=y]
BPF_SYSCALL [=y]
BRIDGE_VLAN_FILTERING [=y]
CGROUP_BPF [=y]

PSI [=y]
PSI_DEFAULT_DISABLED [=n]

SQUASHFS [=y]
SQUASHFS_XATTR [=y]
SQUASHFS_XZ [=y]

TUN [=y]

BRIDGE_NETFILTER [=y]
NETFILTER_XT_MATCH_ADDRTYPE [=y]
NETFILTER_XT_MATCH_CONNTRACK [=y]
NETFILTER_XT_MATCH_IPVS [=y]
IP_VS [=y]
NETFILTER_XT_MARK [=y]
DUMMY [=y]
BLK_DEV_THROTTLING [=y]
RT_GROUP_SCHED [=y]
IP_NF_TARGET_REDIRECT [=y]
IP_VS_NFCT [=y]
IP_VS_PROTO_TCP [=y]
IP_VS_PROTO_UDP [=y]
IP_VS_RR [=y]
CFS_BANDWIDTH [=y]
DM_THIN_PROVISIONING [=y]
VXLAN [=y]
OVERLAY_FS [=y]
IPVLAN [=y]

CONFIG_NF_TABLES [=n]
CONFIG_NFT_COMPAT [=n]

XFRM_INTERFACE [=y]

INET6_TUNNEL [=y]
INET6_XFRM_TUNNEL [=y]
INET6_IPCOMP [=y]

VSOCKETS [=y]
VIRTIO_VSOCKETS [=y]

VLAN_8021Q_GVRP [=y]
VLAN_8021Q_MVRP [=y]

BPFILTER [=y]
NETFILTER_XT_MATCH_STATE [=y]

IPV6_SUBTREES [=y]
IPV6_SEG6_LWTUNNEL [=y]
IPV6_SEG6_HMAC [=y]
IPV6_RPL_LWTUNNEL [=y]

The settings were enabled/disabled for gentoo-sources-6.5.7 and I believe it'll work for newer versions as well. I should also mention that this is without support for apparmor. The use flag was disabled and therefore nothing for it was enabled in the kernel itself.

papu · Post by **papu** » Tue Nov 14, 2023 2:14 am

hi,
I like to mark M if is eligible,
it is a must the Y to get working waydroid?,

today I was trying to use waydroid but i can't,

do you use openrc? me yes,
config_cgroup_bpf depens on a kernel systemd option

Selected by [n]: - GENTOO_LINUX_INIT_SYSTEMD [=n] && GENTOO_LINUX [=y] && GENTOO_LINUX_UDEV [=y]

thanks.

Post by Hu » Tue Nov 14, 2023 3:26 am

papu wrote:I like to mark M if is eligible,
it is a must the Y to get working waydroid?,

Why would you make it a module if you will always load it and use it?

papu wrote:config_cgroup_bpf depens on a kernel systemd option

Selected by [n]: - GENTOO_LINUX_INIT_SYSTEMD [=n] && GENTOO_LINUX [=y] && GENTOO_LINUX_UDEV [=y]

You are reading this backward. Setting systemd will force this option to be enabled, presumably because systemd does not work properly without it. If you choose systemd=n, then you can set this option to enabled or disabled at your preference.

an7h · Post by **an7h** » Tue Nov 14, 2023 8:10 am

papu wrote:hi,
I like to mark M if is eligible,
it is a must the Y to get working waydroid?,

You can set most as modules, but I'm not sure what's the point? it works either way, but getting most of these as modules is going to be more of a waste of time really. I would suggest to just build 'em into the kernel or use sys-kernel/gentoo-kernel-bin if you need waydroid for "one time uses", but that'll mean you would have to load a different kernel every time you need waydroid for something.

papu wrote:do you use openrc? me yes,

Yes, I'm using OpenRC. Don't forget that Waydroid has a service as well.
You can enable it with:
rc-update add waydroid default

-- edit --
Btw, just because waydroid requires systemd kernel support doesn't mean you need to switch to systemd. I've got elogind as a global USE flag and everything is fine that way under OpenRC. Some options are dependent on other options, otherwise they stay hidden and you need to press Z to see them within the list and check via HELP dependencies that should be enabled first, before you can enable an option you're looking for.

Website · Post by **Zucca** » Tue Nov 14, 2023 6:01 pm

If this information isn't yet on wiki, please consider adding it. Thank you.

papu · Post by **papu** » Tue Nov 14, 2023 10:50 pm

Hu wrote:
papu wrote:I like to mark M if is eligible,
it is a must the Y to get working waydroid?,
Why would you make it a module if you will always load it and use it?
papu wrote:config_cgroup_bpf depens on a kernel systemd option

Selected by [n]: - GENTOO_LINUX_INIT_SYSTEMD [=n] && GENTOO_LINUX [=y] && GENTOO_LINUX_UDEV [=y]
You are reading this backward. Setting systemd will force this option to be enabled, presumably because systemd does not work properly without it. If you choose systemd=n, then you can set this option to enabled or disabled at your preference.

yep thanks

papu · Post by **papu** » Tue Nov 14, 2023 10:57 pm

an7h wrote:
papu wrote:hi,
I like to mark M if is eligible,
it is a must the Y to get working waydroid?,
You can set most as modules, but I'm not sure what's the point? it works either way, but getting most of these as modules is going to be more of a waste of time really. I would suggest to just build 'em into the kernel or use sys-kernel/gentoo-kernel-bin if you need waydroid for "one time uses", but that'll mean you would have to load a different kernel every time you need waydroid for something.

i prefer that way for better control of myself , since i am used to configured amdgpu drivers in kernel like M, and bzimage also is small

p.d: well at the end it not works i just gets and screen with animation from right o left all time but nothing more happens

https://i.imgur.com/cHe50gO.jpg

thanks

an7h · Post by **an7h** » Wed Nov 15, 2023 1:26 pm

@papu
That's something already. When I was figuring this stuff out, I couldn't get passed some sort of netfilter script trying to start for a long time, before I got at least the window of Waydroid popping up.

Yet, something is still missing in your kernel by the looks of it, but I can't really tell hat exactly. Try doing a test with sys-kernel/gentoo-kernel-bin just to confirm this is an issue with the kernel. Also I'd like to point out that I've got all those modules required by waydroid as built into the kernel itself and it works fine. I haven't really tested with mostly modules... theoretically it shouldn't be a problem, really... but you might want to try building them into the kernel instead.

You can also check the log with the command waydroid log and post it here. It could help pinpoint what's wrong.

If it'll work with sys-kernel/gentoo-kernel-bin, but not with your custom kernel and built-in modules, then I'll recheck the full list of required options that should be enabled/disabled and make sure nothing above is missing here.The stuff I posted above is just what should be enabled for the gentoo kernel.

papu · Post by **papu** » Thu Nov 16, 2023 11:45 am

an7h wrote:@papu
That's something already. When I was figuring this stuff out, I couldn't get passed some sort of netfilter script trying to start for a long time, before I got at least the window of Waydroid popping up.

Yet, something is still missing in your kernel by the looks of it, but I can't really tell hat exactly. Try doing a test with sys-kernel/gentoo-kernel-bin just to confirm this is an issue with the kernel. Also I'd like to point out that I've got all those modules required by waydroid as built into the kernel itself and it works fine. I haven't really tested with mostly modules... theoretically it shouldn't be a problem, really... but you might want to try building them into the kernel instead.

You can also check the log with the command waydroid log and post it here. It could help pinpoint what's wrong.

If it'll work with sys-kernel/gentoo-kernel-bin, but not with your custom kernel and built-in modules, then I'll recheck the full list of required options that should be enabled/disabled and make sure nothing above is missing here.The stuff I posted above is just what should be enabled for the gentoo kernel.

with kernel-bin seems it works, but i cant install barely nothing from fdroid and neither firefox apt(from apt mirror), the container works very fast.
to find out what kernel-bin option(s) are missing to my kernel sounds too much hard work

and I am already using a qemu-androidx86
why do you disable nftables? It is needed for waydroid according to post messages of emerge
https://i.imgur.com/AnK8Ino.png

have you tried that thing? https://github.com/casualsnek/waydroid_script

an7h · Post by **an7h** » Thu Nov 16, 2023 1:39 pm

papu wrote:why do you disable nftables? It is needed for waydroid according to post messages of emerge
https://i.imgur.com/AnK8Ino.png

I didn't mean you have to disable all netfilter options in the kernel. I was just mentioning that due to improper configuration I would often get a netfilter error in the waydroid log. Something about a script... You only have to disable CONFIG_NF_TABLES [=n] and CONFIG_NFT_COMPAT [=n] because for some reason those two break waydroid if enabled (I tested it and it does).

papu wrote:but i cant install barely nothing from fdroid and neither firefox apt(from apt mirror)

I don't know much about that. I just need waydroid for like two apps... I didn't really test much really with the apps, but I am aware that many games work under waydroid, which is a pretty cool achievement for the devs.

papu wrote: the container works very fast.

Yes, which is why I spent a lot of time to get this thing to work on my system.

papu wrote: to find out what kernel-bin option(s) are missing to my kernel sounds too much hard work

Well, I can give the full and filtered list of required options, not only those that for the gentoo kernel. I have everything saved anyway, just need to organize it and post. I just thought it would redundant to mention options that are already enabled by default within the gentoo kernel. Of course after that you would have to figure out the configuration by yourself. I'll post it a bit later when I have time. Perhaps it would be of use to someone else. Btw, most of the list is taken from PostmarketOS (made for Linux phones and stuff like that).

I should really just post the links here:
https://gitlab.com/postmarketOS/pmboots ... y#L458-572
https://gitlab.com/postmarketOS/pmboots ... y#L349-393

Some options I had to find out from other sources, though...

papu wrote:have you tried that thing? https://github.com/casualsnek/waydroid_script

Yes. It works until you update android itself (at least with the smart dock thing). It seems alright, but I remember having android break two times because of it and I had to manually delete everything waydroid related and re-emerge the whole thing, and do waydroid init as well to redownload the image and everything. The clean-up command from the waydroid troubleshooting doc doesn't really delete everything and might not help if something is broken badly, which this script is somehow capable of doing.

papu · Post by **papu** » Fri Nov 17, 2023 2:07 am

finally I maked it works , there was a problem with the net , I had no internet iinside waydroid, i figure out the options for that, and now i have internet:

Code: Select all

[*]     CONFIG_IP_ADVANCED_ROUTER
     [*]     CONFIG_IP_MULTIPLE_TABLES
     [*]     CONFIG_IP_ROUTE_MULTIPATH
     [*]     CONFIG_IP_ROUTER_VERBOSE

now i have to test waydroid to see how it works and waydroid script..I don't know if it will be the subsitute of my curren qemu+androidx86 that works like a charm right now.

On the other hand the config of gentoo-kernel-bin it is very instructive and will help me a lot.

an7h · Post by **an7h** » Sat Nov 18, 2023 8:35 am

@papu
I checked and those options are enabled by default in the latest sys-kernel/gentoo-sources (6.6.1). Looks like this is something that you disabled yourself earlier.

papu · Post by **papu** » Sat Nov 18, 2023 11:47 am

an7h wrote:@papu
I checked and those options are enabled by default in the latest sys-kernel/gentoo-sources (6.6.1). Looks like this is something that you disabled yourself earlier.

yes I know, I have been using my .config, as cleaner as posible, for a long time and sometimes is needed o add some stuff

however, I've decided to keep using the qemu androidx86 while of waydroid have a great potential has problems righ now, at least on my gentoo.
Moreover, the wayland in kde plasma is not mature enough, and it giving to me some extra problems.

On the other hand figuring out inside the kernel net stuff has helped me optimizate my .config a bit more.

an7h · Post by **an7h** » Sat Nov 18, 2023 4:20 pm

@papu
Yeah, I hear ya. I disable as much as possible as well to make the kernel smaller. Glad you figured it out, and yes, I agree, it seems waydroid needs more time to mature. At least it's good enough for my use case. It actually makes me want to try out a linux phone again. It's something that is pretty major for such a device, considering the lack of mobile optimized software on those things.

CaptainBlood · Post by **CaptainBlood** » Sat Nov 18, 2023 6:54 pm

Nice thread.
I have a basic kernel setting that tries to mach hardware feature.
When a new feature is required I template the previous with another CONFIG_LOCALVERSION.

Despite it has drawbacks I favor <M> when possible.

Will likely seed another kernel for waydroid sooner or later, as far as I can tell.

Thks 4 ur attention, interest & support.

mojyack · Post by **mojyack** » Wed Jan 03, 2024 8:42 am

I got Waydroid working on a custom minimal kernel, so I'll share the information.
I think these kernel options are required:

Code: Select all

*(required)
?(may be required)

* XFRM_INTERFACE
? IP_MULTICAST
? IP_ADVANCED_ROUTER
? IP_MULTIPLE_TABLES
? IP_MROUTE
? IP_MROUTE_MULTIPLE_TABLES
* INET_DIAG
? IPV6_MULTIPLE_TABLES
? IPV6_SUBTREES
? IPV6_MROUTE
? IPV6_MROUTE_MULTIPLE_TABLES
? IPV6_SEG6_LWTUNNEL
? IPV6_SEG6_HMAC
? IPV6_RPL_LWTUNNEL
? NETFILTER
? NETFILTER_ADVANCED
? NETFILTER_INGRESS
? NETFILTER_EGRESS
* NETFILTER_NETLINK_LOG
* NF_CONNTRACK
? NF_CT_PROTO_DCCP
? NF_CT_PROTO_SCTP
? NF_CT_PROTO_UDPLITE
* NF_NAT
* NETFILTER_XTABLES
? NETFILTER_XTABLES_COMPAT
* NETFILTER_XT_TARGET_CHECKSUM
* NETFILTER_XT_NAT
* NETFILTER_XT_TARGET_MASQUERADE
* IP_NF_IPTABLES
* IP_NF_FILTER
* IP_NF_NAT
* IP_NF_MANGLE
* BPFILTER
* BPFILTER_UMH
* BRIDGE
? BRIDGE_IGMP_SNOOPING
* VSOCKETS
* VSOCKETS_LOOPBACK
? NET_L3_MASTER_DEV
* MD
* BLK_DEV_DM
* NET_CORE
* VETH
* ANDROID_BINDER_IPC
* ANDROID_BINDERFS
* ANDROID_BINDER_DEVICES="binder,hwbinder,vndbinder"

The original file can be found here https://github.com/mojyack/gentoo-note/ ... roid/kconf
For reference, the complete kernel config is here https://github.com/mojyack/kconfig/blob ... rface_go_2

Website · Post by **Zucca** » Sat Jan 06, 2024 9:23 pm

I guess I could drop that in /etc/kernel/config.d. Needs a little sed/awk to be compatible, but that's not a problem.

Fitap · Post by **Fitap** » Mon Aug 19, 2024 2:07 am

Hi, I follow this thread however I was unable to run waydroid.

Code: Select all

(006024) [22:37:40] RuntimeError: Command failed: % /usr/lib/waydroid/data/scripts/waydroid-net.sh start
(008889) [23:01:57] Run waydroid -h for usage information.
(008942) [23:02:22] Starting waydroid session
(005986) [23:02:22] % /usr/lib/waydroid/data/scripts/waydroid-net.sh start
awk: fatal: no se puede abrir el fichero «/var/lib/waydroid/lxc/waydroid/config» para lectura: No existe el fichero o el directorio
vnic is waydroid0
Error: No such file or directory; did you mean chain ‘postrouting’ in table ip ‘lxc’?
;
                                    ^^^
Error: Could not process rule: No such file or directory
;
                ^^^^^^^^^^^
Failed to setup waydroid-net.
(005986) [23:02:23] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(005986) [23:02:23] NOTE: The failed command's output is above the ^^^ line in the log file: /var/lib/waydroid/waydroid.log
(008942) [23:02:23] org.freedesktop.DBus.Python.RuntimeError: Traceback (most recent call last):
  File "/usr/lib/python3.12/site-packages/dbus/service.py", line 712, in _message_cb
    retval = candidate_method(self, *args, **keywords)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/waydroid/tools/actions/container_manager.py", line 34, in Start
    do_start(self.args, session)
  File "/usr/lib/waydroid/tools/actions/container_manager.py", line 138, in do_start
    tools.helpers.run.user(args, command)
  File "/usr/lib/waydroid/tools/helpers/run.py", line 58, in user
    return tools.helpers.run_core.core(args, msg, cmd, working_dir, output,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/waydroid/tools/helpers/run_core.py", line 343, in core
    check_return_code(args, code, log_message)
  File "/usr/lib/waydroid/tools/helpers/run_core.py", line 219, in check_return_code
    raise RuntimeError("Command failed: " + log_message)
RuntimeError: Command failed: % /usr/lib/waydroid/data/scripts/waydroid-net.sh start

(008942) [23:02:23] RuntimeError: Command failed: % /usr/lib/waydroid/data/scripts/waydroid-net.sh start

Run over openrc and weston instead.
What else do I need to get it?
Thanks you.