View previous topic :: View next topic |
Author |
Message |
barrymac Tux's lil' helper
Joined: 15 Jul 2004 Posts: 87
|
Posted: Wed Jan 16, 2013 3:34 pm Post subject: can't boot , read only root filesystem |
|
|
Hi all,
I'm afraid the last update on a server I run has made it not able to run init scripts after Code: | Booting (initramfs) | The kernel was upgraded to 3.5.0 at the time the damage was done, I've since upgraded to 3.6.0 from a livecd but it wasn't repaired by that.
It's an LVM JBOD setup with a non lvm boot partition. I pursued grub issues for a while and managed to remove some file not found errors. I get an mdadm error saying missing arrays but I never set up anyway.
In rescue mode the system gets as far as starting init scripts and then returns the error cannot execute /etc/init.d/rcS
enter root password for maintenance. None of the init sripts have run, no network or anything. when I try and run any init scripts after that then for any of them it says Code: | warning [service] is already starting |
From a live Cd I've removed some old kernels and reinstalled openrc and udev for the hell of it to no avail.
From the live cd I chrooted with the boot partition mounted in the right place before any kernel or grub changes.
fsck was clean on the volumes
Root filesystem is being mounted readonly, I don't know why this is, any ideas? |
|
Back to top |
|
|
DONAHUE Watchman
Joined: 09 Dec 2006 Posts: 7651 Location: Goose Creek SC
|
Posted: Wed Jan 16, 2013 7:25 pm Post subject: |
|
|
boot cd run fsck on all partitions, especially / ,then boot system Code: | mount -o remount,rw /
rc | should give you access Kernel may need one or more of following in menuconfig and a recompile: Quote: | General Setup --->
(Make sure the following item is *not* enabled)
[ ] enable deprecated sysfs features to support old userspace tools
Device Drivers │
Generic Driver Options
[*] Maintain a devtmpfs filesystem to mount at /dev
[*] Automount devtmpfs at /dev, after the kernel mounted the rootfs
File Systems --->
[*] Dnotify support
[*] Inotify support for userspace |
_________________ Defund the FCC. |
|
Back to top |
|
|
barrymac Tux's lil' helper
Joined: 15 Jul 2004 Posts: 87
|
Posted: Thu Jan 17, 2013 8:04 pm Post subject: |
|
|
Thanks for the reply Donahue.
I've already tried fscking and all volumes returned clean. I've remounted the root partition rw and it just returns to ro next reboot.
I didn't know about running rc just like that though, it runs but as the system is at either level 2 or 6 depending on whether I use failsafe boot option or not it doesn't get very far.
When I try switching runlevels using init 3 for example then it always returns Code: | INIT: cannot execute "/etc/init.d/rc" | which doesn't exist. I'm using openrc |
|
Back to top |
|
|
barrymac Tux's lil' helper
Joined: 15 Jul 2004 Posts: 87
|
Posted: Thu Jan 17, 2013 8:16 pm Post subject: |
|
|
Another very strange thing when I run df -h I get
Code: | /dev/mapper/vg_lv_root 50G 39G 8.6G 82% /
/dev/mapper/vg_lv_home 50G 39G 8.6G 82% /home
|
although the home partition should be a much bigger one , it's clearly pointing to the root volume here. What's more when I try to unmount home then it tells me it's not mounted! |
|
Back to top |
|
|
DONAHUE Watchman
Joined: 09 Dec 2006 Posts: 7651 Location: Goose Creek SC
|
Posted: Thu Jan 17, 2013 10:32 pm Post subject: |
|
|
Looking at these items tells you?
Quote: | General Setup --->
(Make sure the following item is *not* enabled)
[ ] enable deprecated sysfs features to support old userspace tools
Device Drivers │
Generic Driver Options
[*] Maintain a devtmpfs filesystem to mount at /dev
[*] Automount devtmpfs at /dev, after the kernel mounted the rootfs
File Systems --->
[*] Dnotify support
[*] Inotify support for userspace |
Quote: | emerge -s openrc udev | shows which versions? _________________ Defund the FCC. |
|
Back to top |
|
|
DONAHUE Watchman
Joined: 09 Dec 2006 Posts: 7651 Location: Goose Creek SC
|
Posted: Fri Jan 18, 2013 12:32 am Post subject: |
|
|
Is /etc/fstab still as you want it to be? _________________ Defund the FCC. |
|
Back to top |
|
|
barrymac Tux's lil' helper
Joined: 15 Jul 2004 Posts: 87
|
Posted: Fri Jan 18, 2013 10:03 pm Post subject: |
|
|
Yes my fstab seems fine. I can mount things manually later, and once I remount the root partition rw then I can start the services manually, including udev.
All those kernel options are correct apart from the Automount devtmpfs , so I might try that option out.
My openrc version is 0.10.5 and udev is 189
Thanks again! |
|
Back to top |
|
|
DONAHUE Watchman
Joined: 09 Dec 2006 Posts: 7651 Location: Goose Creek SC
|
Posted: Sat Jan 19, 2013 12:31 am Post subject: |
|
|
complete? list for prefork udev: Quote: | CONFIG_DEVTMPFS=y
CONFIG_DEVTMPFS_MOUNT=y
CONFIG_HOTPLUG=y
CONFIG_FSNOTIFY=y
CONFIG_DNOTIFY=y
CONFIG_INOTIFY_USER=y
CONFIG_NET=y
CONFIG_PROC_FS=y
CONFIG_SIGNALFD=y
CONFIG_SYSFS=y
CONFIG_SYSFS_DEPRECATED is not set
CONFIG_UEVENT_HELPER_PATH=""
CONFIG_BLK_DEV_BSG=y
CONFIG_TMPFS=y
CONFIG_TMPFS_POSIX_ACL=y
CONFIG_TMPFS_XATTR=y | some may not be needed but I think none are harmful. My udev is prefork with udev-171-r9 openrc-0.11.8 here.
uncommenting in /etc/rc.conf might provide some information at /var/log/rc.log _________________ Defund the FCC. |
|
Back to top |
|
|
barrymac Tux's lil' helper
Joined: 15 Jul 2004 Posts: 87
|
Posted: Sat Jan 19, 2013 3:03 am Post subject: |
|
|
once I've booted into maintenance mode I get the following output from rc-status
udev does appear to have not started. I can start all these services manually though and gradually bring the system up.
Code: | Runlevel: sysinit
udev-mount [ started ]
dmesg [ started ]
udev [ crashed ]
devfs [ started ]
Dynamic Runlevel: hotplugged
Dynamic Runlevel: needed
rpcbind [ crashed ]
rpc.statd [ crashed ]
sysfs [ started ]
syslog-ng [ crashed ]
rpc.idmapd [ started ]
Dynamic Runlevel: manual
local [ started ]
network [ started ]
net.eth0 [ started ]
samba [ crashed ]
sshd [ crashed ]
nfs [ started ]
gkrellmd [ crashed ]
|
I'll try checking those further kernel options and see. Much appreciated! |
|
Back to top |
|
|
|