Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
[SOLVED] Gentoo & ESXi 5.0-Problems with newest vmware-tools
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Other Things Gentoo
View previous topic :: View next topic  
Author Message
spiffywiffy
n00b
n00b


Joined: 08 May 2007
Posts: 21

PostPosted: Fri Jan 20, 2012 4:23 pm    Post subject: [SOLVED] Gentoo & ESXi 5.0-Problems with newest vmware-t Reply with quote

Hello:

I get the impression that Gentoo is not widely used in the Enterprise and particularly not widely used as a Guest OS for vSphere/ESXi in the Enterprise. But, my hope is that someone out there is in our boat and can possibly help with this strange issue we are having. Or perhaps someone can help in some way anyway.

Our environment:
We currently run 4 ESXi hosts on Dell M610/M1000e blade hardware with shared EqualLogic storage. All are running the latest build of ESXi 5.0.
We are running about 40-50 Hardened Gentoo Linux VM's. Mixture of 32 and 64-bit kernels. Kernel versions ranging from 2.6.29-hardened to 3.1.5-hardened.

The story:
We used to run ESXi 4.0. We never had any issues compiling and running official VMWare Tools in Gentoo, which would gracefully handle restarts and shutdowns (amongst other features it obviously provides) from the vSphere client no problem.
Since migrating our VM's to ESXi 5.0 and subsequently upgrading out of date VMWare Tools to the latest version (8.6.0.6868 (build-446312)), we are experiencing a strange issue. In order to gracefully restart or shutdown a Gentoo guest from the vSphere client, we must issue the restart or shutdown _twice_ from the vSphere client. Nothing happens at all the first time. On the second attempt, the shutdown or restart process begins normally. If anyone is familiar with the vSphere workings related to HA and Update Manager for example, you can see why this would be a problem in automating tasks.

At this point I should note, that yes, we have tested open-vm-tools. They work fine and do not exhibit the above issue. However, not using VMWare distributed tools limits our ability to manage the automated updating of tools for example via VMWare Update Manager.
Also, we DID test officially supported distros (like CentOS), with the same version of tools we are installing in Gentoo. They do NOT experience the issue noted above.
We also did not experience the issue on VM's migrated from ESXi 4.0 to 5.0 before we updated tools to newest version.

So, this appears to be Gentoo and newest VMWare tools related. Not sure which is really at fault as logs do not help. The command to shutdown or restart just appears to go into a blackhole until the 2nd attempt.

It is not the end of the world, it is just annoying and inconvenient. VMWare support will not help us because Gentoo is not "officially" supported as a guest OS. This is fine - we are well aware that Gentoo is not officially supported, however, we have always found ways to make it work well for us until now. Our use of hardened Gentoo stems back to pre-VM days as a corporate policy and will likely not change anytime soon, so we would prefer to attempt to solve this issue as using VMWare distributed tools is the best case scenario for us.

Thanks for any help. Let me know if I can provide more information!


Last edited by spiffywiffy on Mon Feb 06, 2012 2:25 pm; edited 1 time in total
Back to top
View user's profile Send private message
Kaso_da_Zmok
Apprentice
Apprentice


Joined: 19 Aug 2004
Posts: 189
Location: Zurich, Switzerland

PostPosted: Mon Jan 30, 2012 11:54 am    Post subject: i can confirm the same behavior on ESXi 5.0.0 515841 Reply with quote

i can confirm the same behavior on ESXi 5.0.0 515841
just finished funtoo with gentoo-sources 3.2.1-gentoo-r2
VMwareTools-8.6.0-515842.tar.gz
I have to issue the restart or shutdown guest twice from vSphere client
And also i cannot start the vmware-tools service with start command
Code:
/etc/init.d/vmware-tools start

but have to use
Code:
/etc/init.d/vmware-tools restart

to start the vmware tools.

so adding it to default runlevel does not start the vmware tools and i have to start it per hand with the restart of the service.
maybe related to my kernel config.
Back to top
View user's profile Send private message
ToeiRei
Veteran
Veteran


Joined: 03 Jan 2005
Posts: 1191
Location: Austria

PostPosted: Mon Jan 30, 2012 12:17 pm    Post subject: Reply with quote

Switched to open-vm-tools for that reason, keeping bumped ebuilds in my overlay to work wirh recent kernels.
_________________
Please stand by - The mailer daemon is busy burning your messages in hell...
Back to top
View user's profile Send private message
Kaso_da_Zmok
Apprentice
Apprentice


Joined: 19 Aug 2004
Posts: 189
Location: Zurich, Switzerland

PostPosted: Mon Jan 30, 2012 1:33 pm    Post subject: looking at the esx logs Reply with quote

here is the first attempt to restart guest from esx5 side

Code:
tail -f /var/log/hostd.log

2012-01-30T13:24:13.169Z [32EC2B90 info 'TaskManager' opID=34BF6CCF-00000620-56] Task Created : haTask-60-vim.VirtualMachine.rebootGuest-98092487
2012-01-30T13:24:13.170Z [32EC2B90 verbose 'vm:/vmfs/volumes/4e5d5ebc-9870d0cb-eadc-d8d3855ff8b4/rz-funtoo/rz-funtoo.vmx' opID=34BF6CCF-00000620-56] Rebooting guest
2012-01-30T13:24:13.170Z [32EC2B90 info 'vm:/vmfs/volumes/4e5d5ebc-9870d0cb-eadc-d8d3855ff8b4/rz-funtoo/rz-funtoo.vmx' opID=34BF6CCF-00000620-56] State Transition (VM_STATE_ON -> VM_STATE_REBOOTING)

2012-01-30T13:24:13.263Z [3319CB90 verbose 'vm:/vmfs/volumes/4e5d5ebc-9870d0cb-eadc-d8d3855ff8b4/rz-funtoo/rz-funtoo.vmx'] Handling message _vmxa: The request to Reset this virtual machine failed because the corresponding VMware Tools script did not run successfully. If you have configured a custom reset script in this virtual machine, make sure that it contains no errors. Attempting the operation again will ignore the script failure. You can also submit a support request to report this issue.
-->


and vmware.log
Code:
2012-01-30T13:24:13.175Z| vmx| TOOLS received reboot state change request; adding delayed poweron request
2012-01-30T13:24:13.175Z| vmx| TOOLS sending 'OS_Reboot' (state=2 callback=2B2E3A20 clientData=2C06D700) state change request
2012-01-30T13:24:13.175Z| vmx| Vix: [1888031 vmxCommands.c:677]: VMAutomation_ResetImpl: SoftReboot succeeded.
2012-01-30T13:24:13.261Z| vcpu-1| TOOLS state change 2 returned status 0
2012-01-30T13:24:13.261Z| vcpu-1| Msg_Post: Error
2012-01-30T13:24:13.261Z| vcpu-1| [msg.tools.rebootFailed] The request to Reset this virtual machine failed because the corresponding VMware Tools script did not run successfully. If you have configured a custom reset script in this virtual machine, make sure that it contains no errors. Attempting the operation again will ignore the script failure. You can also submit a support request to report this issue.
2012-01-30T13:24:13.261Z| vcpu-1|
2012-01-30T13:24:13.261Z| vcpu-1| ----------------------------------------





So that explains why the second attempt works
Back to top
View user's profile Send private message
Kaso_da_Zmok
Apprentice
Apprentice


Joined: 19 Aug 2004
Posts: 189
Location: Zurich, Switzerland

PostPosted: Mon Jan 30, 2012 2:24 pm    Post subject: sorted here Reply with quote

Ok i have got it fixed here.
It fails on the script
Code:
/etc/vmware-tools/poweroff-vm-default

which fails because it calls this
Code:
/etc/vmware-tools/scripts/vmware/network

and this fails because it looks for init script called network or networking and also expects command ifup which is not in search path.
so the workaround/fix ( i created link from dhcpcd but you do link to whatever you use to start network)
Code:
rz-funtoo ~ # ln -s /etc/init.d/dhcpcd /etc/init.d/network

And create /sbin/ifup ifdown
Code:
rz-funtoo ~ # nano -w /sbin/ifup
/sbin/ifconfig $1 up
rz-funtoo ~ # chmod 744 /sbin/ifup

Code:
rz-funtoo ~ # nano -w /sbin/ifdown
/sbin/ifconfig $1 down
rz-funtoo ~ # chmod 744 /sbin/ifdown

this makes the script happy and VM listens to the first restart/shutdown request


Last edited by Kaso_da_Zmok on Mon Feb 06, 2012 2:52 pm; edited 1 time in total
Back to top
View user's profile Send private message
spiffywiffy
n00b
n00b


Joined: 08 May 2007
Posts: 21

PostPosted: Mon Jan 30, 2012 2:47 pm    Post subject: Reply with quote

Wow... well done! We haven't tested this yet, but it looks good. Just wanted to jump in and say thanks for the help - will confirm today hopefully and change to solved.
Back to top
View user's profile Send private message
Kaso_da_Zmok
Apprentice
Apprentice


Joined: 19 Aug 2004
Posts: 189
Location: Zurich, Switzerland

PostPosted: Sat Feb 04, 2012 2:13 pm    Post subject: did it work? Reply with quote

did it work for you?
Back to top
View user's profile Send private message
spiffywiffy
n00b
n00b


Joined: 08 May 2007
Posts: 21

PostPosted: Mon Feb 06, 2012 2:25 pm    Post subject: Reply with quote

Yep! Worked like a charm! Thanks.

Think there is anything we can do to make these changes part of Gentoo or at least the vmware-tools ebuild that is in portage? I doubt VMWare will do anything based on my conversations with support.
Back to top
View user's profile Send private message
libertytrek
Apprentice
Apprentice


Joined: 18 Jul 2007
Posts: 258

PostPosted: Fri Apr 13, 2012 1:29 pm    Post subject: Reply with quote

Did you ever post a bug/enhancement request for this? We are getting ready to convert our servers to ESXi, and I'll be running my one gentoo mail/groupware server on it, and would love to not have to kludge anything on a long term ongoing basis...
Back to top
View user's profile Send private message
spiffywiffy
n00b
n00b


Joined: 08 May 2007
Posts: 21

PostPosted: Fri Apr 13, 2012 1:47 pm    Post subject: Reply with quote

No, I didn't actually, but I shall do so right now.

Btw.. since you are going ESXi, we currently can't get any official version of VMWare tools to run on a v3 hardened kernel. Not sure if vanilla kernel is affected too. We have to run open-vm-tools on those, but this is not ideal at all. Just so you know going in. :)
Back to top
View user's profile Send private message
libertytrek
Apprentice
Apprentice


Joined: 18 Jul 2007
Posts: 258

PostPosted: Fri Apr 13, 2012 2:00 pm    Post subject: Reply with quote

Ok, thanks for the heads up... are the drawbacks documented anywhere? I looked around on their website but didn't see anything comparing the two...

The main things I'm worried about are time syncing (looks like the open tools do that), snapshots, and automated safe shutdown/startup of the VMs via the hypervisor (ie, if the UPS issues a shutdown command)...
Back to top
View user's profile Send private message
spiffywiffy
n00b
n00b


Joined: 08 May 2007
Posts: 21

PostPosted: Fri Apr 13, 2012 3:03 pm    Post subject: Reply with quote

Here's some detail:
http://open-vm-tools.sourceforge.net/faq.php

Basically, all of the Open Source components in the commercial VMware Tools are included in open-vm-tools. vmxnet3 and vmmemctl (balloon driver) are now included upstream, so those are less of an issue for folks. In reality, there aren't many drawbacks.

In our experience, the drawbacks for us are:
- Inability to snapshot virtual machine using VMware Data Recovery. Haven't investigated much yet - its a bit of a mystery considering that vmsync module is part of open-vm-tools and snapshot with quiesce enabled from vSphere client is successful.
- Inability to manage tools with VMware Update Manager. Any VM running open-vm-tools shows tools status as '3rd-party/Independant' in vSphere client and cannot be managed for updates to tools.

You should have no issues with graceful shutdown, restart, or time sync, etc.
Back to top
View user's profile Send private message
libertytrek
Apprentice
Apprentice


Joined: 18 Jul 2007
Posts: 258

PostPosted: Fri Apr 13, 2012 3:53 pm    Post subject: Reply with quote

spiffywiffy wrote:
You should have no issues with graceful shutdown, restart, or time sync, etc.


Ok thanks for confirming... but as to snapshots (I'm new to ESXi, and virtualization in general, so pardon my ignorance)...

With the open-vm-tools, will I be able to *automate* snapshots (for backup purposes)?
Back to top
View user's profile Send private message
spiffywiffy
n00b
n00b


Joined: 08 May 2007
Posts: 21

PostPosted: Fri Apr 13, 2012 4:04 pm    Post subject: Reply with quote

Technically, yes. But, snapshot success is dependent on other factors. Sometimes snapshots fail if the filesystem cannot be quiesced, which is not necessarily the fault of vSphere, tools, etc. Frankly, we mostly only see automated snapshot problems with Windows systems.
Back to top
View user's profile Send private message
libertytrek
Apprentice
Apprentice


Joined: 18 Jul 2007
Posts: 258

PostPosted: Fri Apr 13, 2012 5:59 pm    Post subject: Reply with quote

Ok, many thanks for the tips...

Oh - one more thing - when automating snapshots, do these tools allow me to define an email address (and/or SMS number) so I can get email and/or text notifications of success (but more importantly, failure)? If so, that would be sufficient... if not, I guess I'll have to be sure to perform manual snapshots regularly (ugh)...

Thanks again...
Back to top
View user's profile Send private message
spiffywiffy
n00b
n00b


Joined: 08 May 2007
Posts: 21

PostPosted: Mon Apr 16, 2012 1:58 pm    Post subject: Reply with quote

The scheduled tasks feature in vSphere client has notification options.

I'd recommend using VMware Data Recovery - its free, has dedupe, and will offer you more overall flexibility. Plus, you won't suffer any of the possible performance issues having multiple snapshots on the VM object itself.
Back to top
View user's profile Send private message
honp
Guru
Guru


Joined: 25 Sep 2006
Posts: 355
Location: Good old Prague, Czech rep.

PostPosted: Fri Oct 19, 2012 12:41 pm    Post subject: Reply with quote

It is old post but this didn`t solve my problem, but use of /et/local.d did. I added there script with /etc/init.d/vmware-tools restart... I don`t know if it is right, but it works.. :)

H.
Back to top
View user's profile Send private message
honp
Guru
Guru


Joined: 25 Sep 2006
Posts: 355
Location: Good old Prague, Czech rep.

PostPosted: Fri Oct 19, 2012 12:42 pm    Post subject: Reply with quote

This is old post but this didn`t solve my problem, but use of /et/local.d did. I added there script with /etc/init.d/vmware-tools restart... I don`t know if it is right, but it works.. :)

H.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Other Things Gentoo All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum