I have a customer out in Ohio who has had a horrible year. His software
vendor convinced him to buy a $6,000 server to run their new Windows
version of their software; he had all kinds of Windows configuration
and hardware problems; his users hated the new software; he finally gave up and went back to the ancient, unsupported SCO Unix version.
What to do with the $6,000.00 server? I suggested putting Linux
and VMware on it - heck, it's a big, powerful box, it seems a shame
to have it go to waste (and we both thought that running Windows 2003 Server
was definitely a "waste").
So he did. And immediately ran into all kinds of networking problems.
Unfortunately, most of times when he called me about it, I was on the
road or otherwise tied up and couldn't help him much. Add to that is
that I am no VMware expert, so he was getting nowhere. He basically
had everything working, but the VMware Windows instance couldn't talk
to the Linux host that ran the VMware. It could talk to everything
else, but not its own host. He tried fixing it, but made things worse
and when he called me Saturday afternoon, Windows wasn't talking
to much of anything.
My poor wife.. we have our daughter and son-in-law coming Monday morning
to stay for the week; we are pretty much ready but there are still
a few things she needs me to do, and now I'm tied up on the phone. She
knows customers have to come first but sheesh - I'd already been out
in the morning to help a neighbor and now this? She sighed..
I sshed to the box while keeping my customer on the
phone. As I said, I'm no VMware expert, but I saw a few things that
bothered me. First of all, he had 4 NIC's in the machine. Given
the size of his business and the network traffic, I saw no reason
for that and I had already realized that he was confused as to which
card was which. We decided to cut it back to two cards: one for the
inside, private IP, one for its public interface. That would make
it much easier to figure out where cables needed to go.
The second thing I realized as I looked things over was that
eth0 was configured to 192.168.1.4 and that eth1 was 192.168.1.3. As
he was going to disable eth1, I would need to add an alias for
.3, so I added "ifconfig eth0:1 192.168.1.3" to rc.local. However,
there was something more that I had not caught on to in our
phone conversations: the Windows machine was configured to be 1.4 also.
VMware works in "bridged" mode - you use the 192.168.1.4 card by attaching a virtual network device to it, but
you put a free address on the Windows virtual card - say 1.5 or 1.6.
That's why Windows couldn't talk to 1.4 or 1.3, it had to go through eth0
to do that. So Windows at 1.4 is trying to pass packets through the host
that is also using 1.4. Frankly, I'm surprised it could talk to any other IP on the 192.168.1 network. Of course for those, it didn't have to pass through the host's
1.4 NIC, but still I expected that the IP conflict would
have confused things.
I know that's confusing. Look at it this way:
The Windows virtual machine uses a virtual network adaptor configured
by VMware to use the real physical hardware "eth0". The Linux host
had 192.168.1.4 assigned to eth0. He had configured the virtual card
in Windows to use 1.4 also. Obviously 1.4 on either machine (real or virtual) can't talk to 1.4 on the other. Less obvious is that the Windows 1.4 can't talk to the Linux 1.3 either because its only path to that is through the Linux eth0 card (even before I made 1.3 an alias).
Bridged mode lets the virtual machine share the host's Ethernet connection, while appearing as a separate machine with its own MAC and TCP/IP address.
NAT mode nats through the hosts NIC, much like the 192.168.1.x machines
nat through your router to the internet (yes, yes, I know what
the deep networking geeks are about to complain about: nat vs. masquerading).
The VMware server assigns DHCP addresses to the virtual network cards, and the NAT system takes it from there through the real NIC (though that in turn is
obviously natting or masquerading itself).
Finally, Host mode
only allows the virtual machine to talk to the hosting machine and other virtual machines configured the same way, but nothing else.
After making the changes and resetting Windows to be .5, everything
worked as it should. If he wants to add back the other NICS, he can, but
at least he's starting from a known working configuration.