| View previous topic :: View next topic |
| Author |
Message |
grant123
Veteran
Joined: 23 Mar 2005
Posts: 1080
|
 Posted: Sun Apr 27, 2014 9:42 pm Post subject: [solved] Some services don't always start automatically |
 |
|
Is anyone else finding that some services which should start automatically at boot don't always do so? I just restarted my laptop and my LAN network interface wasn't up along with all of the services which depend on it such as shorewall. I did an 'rc' and it came right up followed by all dependencies. I have a remote system which I sometimes can't get into after rebooting and I suspect it's due to this issue.
Last edited by grant123 on Sat May 10, 2014 9:51 pm; edited 1 time in total |
|
| Back to top |
|
 |
grant123
Veteran
Joined: 23 Mar 2005
Posts: 1080
|
| Posted: Wed Apr 30, 2014 4:27 pm Post subject: |
|
|
| What could cause this? |
|
| Back to top |
|
|
Chiitoo
Administrator
Joined: 28 Feb 2010
Posts: 2575
Location: Here and Away Again
|
| Posted: Wed Apr 30, 2014 4:46 pm Post subject: ><)))°€ |
|
|
Teegrins, grant123!
I don't really have any guesses to share (yet), but do you actually see them failing during boot?
If you boot straight into a login-manager, or you don't have --noclear set for the terminal in /etc/inittab, it may be difficult to see any of the failures. There is, however, the /var/log/rc.log which can be examined later on.
One needs to have something like this in /etc/rc.conf for the file to appear:
| Code: | # rc_logger launches a logging daemon to log the entire rc process to
# /var/log/rc.log
# NOTE: Linux systems require the devfs service to be started before
# logging can take place and as such cannot log the sysinit runlevel.
rc_logger="YES" |
I know this isn't much, but I do hope it helps!
_________________
Kindest of regardses. |
|
| Back to top |
|
|
grant123
Veteran
Joined: 23 Mar 2005
Posts: 1080
|
| Posted: Thu May 01, 2014 3:01 pm Post subject: |
|
|
| Thank you, I've just implemented that and I will report back with any findings. |
|
| Back to top |
|
|
steveL
Watchman
Joined: 13 Sep 2006
Posts: 5153
Location: The Peanut Gallery
|
| Posted: Thu May 01, 2014 4:23 pm Post subject: |
|
|
| grant123 wrote: | | What could cause this? |
It sounds a lot like udev "persistently-changing" network names. Have you disabled those; if so, you know they changed the filename that disables it? If not, you might want to, unless you have multiple ethernet cards. The kernel parm works, but I currently have both /etc/udev/rules.d/80-net-name-slot.rules and /etc/udev/rules.d/80-net-setup-link.rules (I just hardlinked the latter to the former with ln, and haven't upgraded yet.)
| Code: | $ cat /etc/udev/rules.d/80-net-setup-link.rules
# dummy empty file to keep eth* and wlan*
|
Don't do this unless you're sure, as it means you have to change your net setup scripts back to eth0, and I don't want to be blamed for your remote system. ;) I mention it in case you've already done this in 80-net-name-slot.rules, and as I think it's better, in general; your specific case might differ. |
|
| Back to top |
|
|
grant123
Veteran
Joined: 23 Mar 2005
Posts: 1080
|
| Posted: Sat May 03, 2014 12:45 am Post subject: |
|
|
I have this from /var/log/rc.log:
* Bringing up interface enp0s20u2u1
* ERROR: interface enp0s20u2u1 does not exist
* Ensure that you have loaded the correct kernel module for your hardware
* ERROR: net.enp0s20u2u1 failed to start
* Bringing up interface enp0s20u2u2
* ERROR: interface enp0s20u2u2 does not exist
* Ensure that you have loaded the correct kernel module for your hardware
* ERROR: net.enp0s20u2u2 failed to start
Both interfaces are USB connected to my laptop and the drivers are built into the kernel. A subsequent reboot starts both interfaces fine. I checked /var/log/everything/current (metalog) but nothing jumps out at me except this later in the boot sequence:
[/etc/init.d/net.enp0s20u2u1] net.enp0s20u2u1: not allowed to be hotplugged
I'll try rc_hotplug="net.*" but that's not fixing the real problem. |
|
| Back to top |
|
|
666threesixes666
Veteran
Joined: 31 May 2011
Posts: 1248
Location: 42.68n 85.41w
|
|
| Back to top |
|
|
grant123
Veteran
Joined: 23 Mar 2005
Posts: 1080
|
| Posted: Sun May 04, 2014 3:36 am Post subject: |
|
|
rc_hotplug="net.*" seems to start the network interfaces reliably but I'd like to get to the root of the problem and disable hotplug. I found this in the logs:
systemd-udevd[1249]: renamed network interface eth0 to enp0s20u2u1
Could udev be renaming my interfaces after the initscripts are trying to start them? I have net.enp0s20u2u1 and net.enp0s20u2u2 in the default runlevel. |
|
| Back to top |
|
|
khayyam
Watchman
Joined: 07 Jun 2012
Posts: 6227
Location: Room 101
|
| Posted: Sun May 04, 2014 12:01 pm Post subject: |
|
|
| grant123 wrote: | | Could udev be renaming my interfaces after the initscripts are trying to start them? I have net.enp0s20u2u1 and net.enp0s20u2u2 in the default runlevel. |
grant123 ... no, udev should be in the boot runlevel and so have re-named the devices prior to them being started. Note that if you re-plug the USB into another slot these "(non-)persistent names" will be changed.
What do you have set for 'rc_depend_strict' (/etc/rc.conf)? Also, rc_parallel="YES" isn't set?
BTW, is there a reason both interfaces are brought up ... are they providing separate subnets?
best ... khay |
|
| Back to top |
|
|
grant123
Veteran
Joined: 23 Mar 2005
Posts: 1080
|
| Posted: Sun May 04, 2014 2:30 pm Post subject: |
|
|
| Quote: | | udev should be in the boot runlevel and so have re-named the devices prior to them being started |
In this chunk from the logs, "net.enp0s20u2u1 would not start" comes before "renamed network interface eth0 to enp0s20u2u1":
| Code: | [/etc/init.d/shorewall] ERROR: cannot start shorewall as net.enp0s20u2u1 would not start
[sshd] Server listening on 0.0.0.0 port 22.
[lightdm] PAM unable to dlopen(/lib64/security/pam_systemd.so): /lib64/security/pam_systemd.so: cannot open shared object file: No such file or directory
[lightdm] PAM adding faulty module: /lib64/security/pam_systemd.so
[kernel] [ 4.139828] ax88179_178a 3-2.1:1.0 eth0: register 'ax88179_178a' at usb-0000:00:14.0-2.1, ASIX AX88179 USB 3.0 Gigabit Ethernet
[dbus] [system] Activating service name='org.freedesktop.Accounts' (using servicehelper)
[mtp-probe] checking bus 3, device 8: "/sys/devices/pci0000:00/0000:00:14.0/usb3/3-2/3-2.1"_
[mtp-probe] bus: 3, device: 8 was not an MTP device_
[kernel] [ 4.154662] systemd-udevd[1395]: renamed network interface eth0 to enp0s20u2u1 |
I have udev and udev-mount in the sysinit runlevel:
| Code: | # rc-update -s
alsasound | boot
bootmisc | boot
cupsd | default
dbus | default
devfs | sysinit
device-mapper | boot
dmesg | sysinit
fsck | boot
hostname | boot
hwclock | boot
keymaps | boot
killprocs | shutdown
kmod-static-nodes | sysinit
local | default
localmount | boot
loopback | boot
lvm | boot
metalog | default
modules | boot
mount-ro | shutdown
mtab | boot
net.enp0s20u2u1 | default
net.enp0s20u2u2 | default
net.lo | boot
netmount | default
ntpd | default
procfs | boot
root | boot
savecache | shutdown
shorewall | default
sshd | default
swap | boot
swapfiles | boot
sysctl | boot
sysfs | sysinit
termencoding | boot
tmpfiles.dev | sysinit
tmpfiles.setup | boot
udev | sysinit
udev-mount | sysinit
unbound | default
urandom | boot
vixie-cron | default
xdm | default |
| Quote: | | What do you have set for 'rc_depend_strict' (/etc/rc.conf)? |
It's unset so it's default.
| Quote: | | Also, rc_parallel="YES" isn't set? |
Unset there too.
| Quote: | | is there a reason both interfaces are brought up ... are they providing separate subnets? |
enp0s20u2u1 is the WAN and enp0s20u2u2 is the LAN. The laptop has no built-in ethernet adapters. |
|
| Back to top |
|
|
khayyam
Watchman
Joined: 07 Jun 2012
Posts: 6227
Location: Room 101
|
| Posted: Sun May 04, 2014 7:05 pm Post subject: |
|
|
| grant123 wrote: | | Code: | [lightdm] PAM unable to dlopen(/lib64/security/pam_systemd.so): /lib64/security/pam_systemd.so: cannot open shared object file: No such file or directory
[lightdm] PAM adding faulty module: /lib64/security/pam_systemd.so |
|
grant123 ... overlooking the fact that obviously this is getting messed up by systemd-udev not completing its tasks before the default runlevel is reached ... why on earth do you have components built with the systemd useflag? Or is this some new "udev needs it" shananigans.
Anyhow, I can't say why systemd-udev is screwing the process up as I don't use udev, perhaps the "aggressive parallelization" of systemd is now default for all components, or the above is first signs that "standalone udev" properly requires systemd for two network devices to be activated before the rest of the system is brought up. Well, as I don't support systemd (or systemd users) I might as well include systemd-udev in the list ... as the old adage goes "when it breaks you get to keep all the peices".
| grant123 wrote: | | Quote: | | What do you have set for 'rc_depend_strict' (/etc/rc.conf)? |
It's unset so it's default. |
I believe that is rc_depend_strict="NO" (at least as I remember I changed this myself), anyhow, you should read the description of what this does (in /etc/rc.conf).
| grant123 wrote: | | Quote: | | Also, rc_parallel="YES" isn't set? |
Unset there too. |
So, "NO" ... ok.
| grant123 wrote: | | Quote: | | is there a reason both interfaces are brought up ... are they providing separate subnets? |
enp0s20u2u1 is the WAN and enp0s20u2u2 is the LAN. The laptop has no built-in ethernet adapters. |
Huh? This laptop is the gateway?
best ... khay |
|
| Back to top |
|
|
grant123
Veteran
Joined: 23 Mar 2005
Posts: 1080
|
| Posted: Mon May 05, 2014 9:30 pm Post subject: |
|
|
| Quote: | | overlooking the fact that obviously this is getting messed up by systemd-udev not completing its tasks before the default runlevel is reached ... why on earth do you have components built with the systemd useflag? Or is this some new "udev needs it" shananigans |
I actually don't have anything built with USE=systemd:
# USE=-systemd emerge -pvDuN world
These are the packages that would be merged, in order:
Calculating dependencies... done!
Total: 0 packages, Size of downloads: 0 kB
#
| Quote: | | Anyhow, I can't say why systemd-udev is screwing the process up as I don't use udev, perhaps the "aggressive parallelization" of systemd is now default for all components, or the above is first signs that "standalone udev" properly requires systemd for two network devices to be activated before the rest of the system is brought up. Well, as I don't support systemd (or systemd users) I might as well include systemd-udev in the list ... as the old adage goes "when it breaks you get to keep all the peices". |
Should I try emerging world with USE="systemd"?
| Quote: | | I believe that is rc_depend_strict="NO" (at least as I remember I changed this myself), anyhow, you should read the description of what this does (in /etc/rc.conf). |
Here's what I have in /etc/rc.conf:
# Do we allow any started service in the runlevel to satisfy the dependency
# or do we want all of them regardless of state? For example, if net.eth0
# and net.eth1 are in the default runlevel then with rc_depend_strict="NO"
# both will be started, but services that depend on 'net' will work if either
# one comes up. With rc_depend_strict="YES" we would require them both to
# come up.
#rc_depend_strict="YES"
So I think it defaults to YES. It sounds like this only matters if both interfaces should not come up, but I do want them both up.
It looks like rc_parallel defaults to NO.
| Quote: | | Huh? This laptop is the gateway? |
Yes, it's connected to a modem/router which forwards all ports to the laptop. |
|
| Back to top |
|
|
khayyam
Watchman
Joined: 07 Jun 2012
Posts: 6227
Location: Room 101
|
| Posted: Tue May 06, 2014 12:12 am Post subject: |
|
|
| grant123 wrote: | | I actually don't have anything built with USE=systemd |
grant123 ... ok, but then why is lightdm looking for /lib64/security/pam_systemd.so, that doesn't seem normal as it (or something) has to be linked to pam_systemd.so. Sorry, I wasn't suggesting you'd screwed up somehow, but having the useflag enabled seemed the most obvious reason for this.
| grant123 wrote: | | Should I try emerging world with USE="systemd"? |
No, definitely not. Your best to try and debug udev and see why its taking so long. Oh, my memory ... ummm ... should be /etc/udev/udev.conf ... set the debug level and enable 'udev_monitor' and 'udev_debug' ... you should then get debug info in dmesg and syslog ... udev also logs somewhere but I can't remember where right now (perhaps under /var/log or /var/run/udev/).
| grant123 wrote: | | Quote: | | Huh? This laptop is the gateway? |
Yes, it's connected to a modem/router which forwards all ports to the laptop. |
I see ... that'll eat up quite a lot of power, I'm sure if you went to a saturday market, boot sale, thrift store, or similar you'd find something that could replace it.
best ... khay |
|
| Back to top |
|
|
grant123
Veteran
Joined: 23 Mar 2005
Posts: 1080
|
| Posted: Tue May 06, 2014 10:19 pm Post subject: |
|
|
| Quote: | | why is lightdm looking for /lib64/security/pam_systemd.so, that doesn't seem normal as it (or something) has to be linked to pam_systemd.so. |
I think that's because of these lines in these default files:
/etc/pam.d/lightdm:
| Code: | | session optional pam_systemd.so |
/etc/pam.d/lightdm-greeter:
| Code: | | session optional pam_systemd.so |
I tried commenting them and rebooting and the errors do go away but the problem remains.
| Quote: | | Your best to try and debug udev and see why its taking so long. Oh, my memory ... ummm ... should be /etc/udev/udev.conf ... set the debug level and enable 'udev_monitor' and 'udev_debug' ... you should then get debug info in dmesg and syslog ... udev also logs somewhere but I can't remember where right now (perhaps under /var/log or /var/run/udev/) |
I tried that but couldn't find anything enlightening. Do we know that udev is taking too long? At this point I don't get any messages in /var/log/everything/current about any services failing to start because the enp* interfaces failed to start, but I do still get this in rc.log:
| Code: | * Bringing up interface enp0s20u2u1
* ERROR: interface enp0s20u2u1 does not exist
* Ensure that you have loaded the correct kernel module for your hardware
* ERROR: net.enp0s20u2u1 failed to start
* Bringing up interface enp0s20u2u2
Cannot find device "enp0s20u2u2"
* ERROR: interface enp0s20u2u2 does not exist
* Ensure that you have loaded the correct kernel module for your hardware
* ERROR: net.enp0s20u2u2 failed to start |
Once the system is fully booted, 'rc' does nothing so I think hotplug is making sure everything comes up but I'd prefer the interfaces to start via their initscripts so I could disable hotplug.
| Quote: | | Huh? This laptop is the gateway? |
| Quote: | | Yes, it's connected to a modem/router which forwards all ports to the laptop. |
| Quote: | | I see ... that'll eat up quite a lot of power, I'm sure if you went to a saturday market, boot sale, thrift store, or similar you'd find something that could replace it. |
It's a workstation too though. It does double-duty. |
|
| Back to top |
|
|
grant123
Veteran
Joined: 23 Mar 2005
Posts: 1080
|
| Posted: Sat May 10, 2014 9:52 pm Post subject: |
|
|
| I needed to build xhci_hcd into the kernel instead of as a module. Thank you for all your help. You definitely helped me reach the solution. |
|
| Back to top |
|
|
khayyam
Watchman
Joined: 07 Jun 2012
Posts: 6227
Location: Room 101
|
| Posted: Sun May 11, 2014 12:29 am Post subject: |
|
|
| grant123 wrote: | | I needed to build xhci_hcd into the kernel instead of as a module. Thank you for all your help. You definitely helped me reach the solution. |
grant123 ... you're welcome. Occassionally a topic reply notification either doesn't reach me or I overlook it for some reason ... this is one such case, I hadn't noticed your reply until now. Anyhow, pleased to hear your issue was resolved ... I'd look into the reason for the pam_systemd.so is being defined though.
best ... khay |
|
| Back to top |
|
|
|
Portage & Programming
