RVL VMware HOWTO
Jump to topics in this page:
-
The host OS is copacabana.ecn.purdue.edu
(the top of two large rack-mount servers).
-
The configuration is stored under /etc/vmware/
-
The guests are:
copavm1.ecn.purdue.edu 128.46.144.211
copavm2.ecn.purdue.edu 128.46.144.212
copavm3.ecn.purdue.edu 128.46.144.213
copavm4.ecn.purdue.edu 128.46.144.214
copavm5.ecn.purdue.edu 128.46.144.215
copavm6.ecn.purdue.edu 128.46.144.216
copavm7.ecn.purdue.edu 128.46.144.217
copavm8.ecn.purdue.edu 128.46.144.218
The images of the guest drives are stored under
/var/lib/vmware/Virtual\ Machines/
VMware is already installed!
However, as a reference:
-
Software is kept in /etc/vmware/software-archive/
-
Install the RPM:
# rpm -Uvh VMware-gsx-3.1.0-9089.i386.rpm
-
Configure VMware:
# vmware-config.pl
-
Extract the tar file:
# cd /tmp
# tar xvfz VMware-mui-3.1.0-9089.tar.gz
-
Configure the web-based administrative interface:
# cd vmware-mui-distrib
# ./vmware-install.pl
-
That last step should have had you run
vmware-config-mui.pl
but if it did not, do so now.
Note that we have GSX Version 3.1 Server, and there are three components:
-
VMware itself, what simulates the virtual hardware platforms.
-
vmware-mui, a graphical monitoring and administration
interface.
-
vmware-console, what lets you attach a virtual console
to a running virtual machine via the graphical interface.
-
Configuration files are in /etc/vmware*/
-
Virtual machine disk images and individual configuration files
are in
/var/lib/vmware/Virtual\ Machines/<name>
-
Helper programs are in /usr/lib/vmware*/
-
The main programs are in /usr/bin/vmware*
Note -- not working yet!
-
First, verify on the host OS that the web-based interface
is listening for you:
# lsof -i tcp:8333
-
Start a Mozilla browser (it seems to have some problems with
Konqueror) and direct it to:
https://copacabana:8333/
-
Authenticate there as root
-
The guests mount the host's /home as their /home
so that a user's environment will be identical across all guests
and the host.
-
The guests also mount the host's /usr/local as their
/usr/local,
so that any software installed in that area on the guest will
be available to all guests without duplication of disk storage.
-
I have looked into using LDAPS for remote authentication, so there
could be a single LDAPS authentication service running on the host OS.
So far, I have not been successful in setting up an LDAP/LDAPS
server, OpenLDAP seems to assume a lot of knowledge of X.500.
Until then, users will need to run passwd on all guests
and the host.
The following is a brute-force method, which takes advantage of
the assumption that an existing machine is the product of a lot
of careful configuration work.
It assumes that two machines have been created, and we are making
the third.
And of course, all this must be done on the host OS!
First, clone the filesystem images and do some reconfiguration:
-
Create a new directory:
# cd /var/lib/vmware/Virtual\ Machines
# mkdir "Virtual 03"
-
Copy the files over.
The following scripting trickery changes filename components
as needed.
Change the "01" and "03" as needed, and watch the direction
of those quote marks!
# for F in Virtual\ 01/*
> do
> cp -av "$F" "`echo $F | sed 's/Virtual 01/Virtual 03/g'`"
> done
Be patient, this will take a while...
-
Change the first of two files that specify the disk images:
# cd Virtual\ 03
# vi Virtual\ 03.vmdk
Globally change "Virtual 01" to "Virtual 03" by typing:
:%s/Virtual 01/Virtual 03/g
Save your changes and exit the editor.
-
Now edit the other file:
# vi Virtual\ 03.vmx
Globally change "Virtual 01" to "Virtual 03" by typing:
:%s/Virtual 01/Virtual 03/g
-
Delete the write-lock file, if it exists:
# rm Virtual\ 03.vmdk.WRITELOCK
-
Edit the file /etc/vmware/vm-list to add the
obviously needed line:
config "/var/lib/vmware/Virtual Machines/Virtual 03/Virtual 03.vmx"
Now, bring up the new machine to single-user mode only:
-
Go to the console of the host OS.
If it is not in graphical mode, first login as root and run:
# init 5
-
Login under X using your personal account.
-
Become root:
$ su
-
Run VMware as root.
Be patient, it will take several seconds for the graphical interface
to appear!
# vmware &
-
When you get the "Connect to host" window, make sure that
the "Local host" button is selected, then click "OK".
-
You will see the virtual machine you just created at the
end of the list in the inventory in the left-most pane
(click the "Show/Hide Inventory" button if needed).
It will have a name that is a duplicate of a name above.
Right-click that last entry, select "Rename", and give it
the appropriate name.
-
Click Start this virtual machine,
and agree to "Create a new identifier".
-
When the VM starts to boot, you must quickly
click within the black console window (to get focus inside
that window), and then be ready to hit the down and up arrow
keys when the LILO screen appears, so the system does not boot
in the normal way.
-
Tell LILO to boot into single-user mode:
linux S
-
It will boot into single-user mode, so you are root on the console.
You will be editing a couple of files, ignore the warning about
being unable to write to $HOME.
-
Edit /etc/sysconfig/network and change this line:
HOSTNAME=oldname.ecn.purdue.edu
-
Edit /etc/sysconfig/network-scripts/ifcfg-eth0
and change this line:
IPADDR=128.46.144.123
-
Shut it down cleanly with halt so all file system
changes are flushed to the virtual disk.
It will be ready for starting from the web-based administrative
interface.
Starting the new machine
-
You should be able to start the new machine with
the graphical interface, see above
-
Alternatively, careful reading of
/etc/init.d/vmware
revealed that if you correctly made the change to
/etc/vmware/vm-list,
then the new machine should be started by the
vmware-serverd daemon.
Send it a kill signal, forcing it to restart:
# killall vmware-serverd
Have patience, it may take up to three minutes to start
the process of booting each guest. You could investigate
what's running with this:
# ps axuw | grep vmware-vmx
-
Finally, here is a really brute-force way of getting a
virtual machine up and running.
Again, change the "03" as needed:
# /usr/lib/vmware/bin/vmware-vmx \
-C /var/lib/vmware/Virtual\ Machines/Virtual\ 03/Virtual\ 03.vmx -@ "" &
When Mandrake boots, it re-creates /boot/kernel.h,
which normally is a very good thing.
However, it makes VMware believe that it has not yet been configured under
this kernel, even though it is likely the very same kernel that was
running before the last shutdown.
This means that the guest machines probably will not come up after a
reboot of the host OS without manual intervention --
probably not that big of a deal because the host OS should not need
frequent rebooting.
To work around this issue:
-
First verify that no guest machines are running -- if you do see
the expected set of guest OS processes, do nothing!
# ps axuw | egrep 'PID|vmware-vmx'
-
Make sure that the VMware software is really shut down.
First, stop the VMware services with the boot script:
# /etc/init.d/vmware stop
-
You will likely find that some process survives the shutdown,
probably vmware-serverd -- shut down any surviving
processing, changing vmware-serverd as needed in
the following:
# ps axuw | egrep 'PID|vmware'
# pkill -TERM vmware-serverd
# pkill -KILL vmware-serverd
-
Reconfigure VMware for operation under the "new" kernel.
You should be able to simply agree with all the choices,
but do make sure that it builds and loads the kernel modules
as needed:
# vmware-config.pl
-
At this point you should find that VMware has been started,
but check with the following command and use the boot script
if needed:
# ps axuw | egrep 'PID|vmware-vmx'
-
The guest machines will come up one at a time, a new one starting
every three minutes in the reverse of the order they're listed
in /etc/vmware/vm-list.
While you're waiting, you could type the following at a shell
prompt to monitor their process:
# while true
> do
> clear
> uptime
> ps axuw | egrep 'PID|vmware-vmx | grep -v grep
> sleep 10
> done
One alternative is to re-start one machine with the above command.
But that's not really the best way if it's a general die-off....
To really shutdown down all of them cleanly and restart, at least
until I get the vmware-mui interface running correctly:
-
Shut down all the running machines:
# for I in 1 2 3 4 5 6 7 8
> do
> ssh 128.46.144.21${I} halt
> done
-
Make sure the VMware daemons are shut down:
# /etc/init.d/vmware stop
-
Note that the above will not kill off all the VMware processes,
I think this is due to differences between the expected
boot scripts and those found on Mandrake 10.X.
So make sure:
# pkill -TERM vmware-serverd
# pkill -KILL vmware-serverd
# ps axuw | grep vm
You should see nothing here.
-
There may be a bunch of lock files left laying around,
get rid of them now if needed.
Be careful with the last two commands!
# cd /var/run/vmware
# rm %2f*
# rm serverd-rendezvous
# rm vmware%2dserverd
# rm -r root/*
# rm -r httpd/*
-
OK, now you should be ready to restart the VMware daemons:
# /etc/init.d/vmware start
-
Now you need to be very patient.
There should be a process vmware-serverd
which will have read the file /etc/vmware/vm-list
and will be starting the specified virtual machines.
However, it will only start one every three minutes, to keep
from overwhelming the host OS by trying to start all the daemons
and all the guest operating systems simultaneously.
Give it about five minutes, and see if any are running:
# ps axuw | grep vmx
-
Now wait another three to five minutes and look again:
# ps axuw | grep vmx
-
You should see the virtual machines starting,
in the reverse of the order that they're listed in
/etc/vmware/vm-list.
If so, good, just continue to be patient!
If not, well, go back to the first step...
Another trick for general troubleshooting is to use top
to watch the running virtual machine emulators, the processes running
the command vmware-vmx.
Here's a useful trick:
-
Find the list of numeric PID's:
# ps axuw | grep vmware-vmx
-
Now ask for just those PID's, changing the numeric PID's as
needed in the following:
# top -p 22824,22865,22907,22948,23010,23049,23093,23102
-
Not as bad as you might think.
Each running virtual machine seems to take 5-7% of the available
CPU cycles when it's mostly idle.
2-3 times that when it's running fsck (at which point
the processing is probably blocked waiting on I/O).
-
Virtual machines just sitting waiting for a login seem to take
about 4% of the available RAM.
-
But remember, anything that pegs the CPU on one guest will peg
the one real CPU.
All the guests, plus the host OS, must share the RAM and CPU cycles.
-
With eight virtual machines, expect a load average around 10-12!