VMware Networks, Bridged vs. Nat vs. Host

I have a customer out in Ohio who has had a horrible year. His software vendor convinced him to buy a $6,000 server to run their new Windows version of their software; he had all kinds of Windows configuration and hardware problems; his users hated the new software; he finally gave up and went back to the ancient, unsupported SCO Unix version.

What to do with the $6,000.00 server? I suggested putting Linux and VMware on it - heck, it's a big, powerful box, it seems a shame to have it go to waste (and we both thought that running Windows 2003 Server was definitely a "waste").

So he did. And immediately ran into all kinds of networking problems. Unfortunately, most of times when he called me about it, I was on the road or otherwise tied up and couldn't help him much. Add to that is that I am no VMware expert, so he was getting nowhere. He basically had everything working, but the VMware Windows instance couldn't talk to the Linux host that ran the VMware. It could talk to everything else, but not its own host. He tried fixing it, but made things worse and when he called me Saturday afternoon, Windows wasn't talking to much of anything.

My poor wife.. we have our daughter and son-in-law coming Monday morning to stay for the week; we are pretty much ready but there are still a few things she needs me to do, and now I'm tied up on the phone. She knows customers have to come first but sheesh - I'd already been out in the morning to help a neighbor and now this? She sighed..

I sshed to the box while keeping my customer on the phone. As I said, I'm no VMware expert, but I saw a few things that bothered me. First of all, he had 4 NIC's in the machine. Given the size of his business and the network traffic, I saw no reason for that and I had already realized that he was confused as to which card was which. We decided to cut it back to two cards: one for the inside, private IP, one for its public interface. That would make it much easier to figure out where cables needed to go.

The second thing I realized as I looked things over was that eth0 was configured to 192.168.1.4 and that eth1 was 192.168.1.3. As he was going to disable eth1, I would need to add an alias for .3, so I added "ifconfig eth0:1 192.168.1.3" to rc.local. However, there was something more that I had not caught on to in our phone conversations: the Windows machine was configured to be 1.4 also.

By default, VMware works in "bridged" mode - you use the 192.168.1.4 card by attaching a virtual network device to it, but you put a free address on the Windows virtual card - say 1.5 or 1.6. That's why Windows couldn't talk to 1.4 or 1.3, it had to go through eth0 to do that. So Windows at 1.4 is trying to pass packets through the host that is also using 1.4. Frankly, I'm surprised it could talk to any other IP on the 192.168.1 network. Of course for those, it didn't have to pass through the host's 1.4 NIC, but still I expected that the IP conflict would have confused things.

I know that's confusing. Look at it this way:

The Windows virtual machine uses a virtual network adaptor configured by VMware to use the real physical hardware "eth0". The Linux host had 192.168.1.4 assigned to eth0. He had configured the virtual card in Windows to use 1.4 also. Obviously 1.4 on either machine (real or virtual) can't talk to 1.4 on the other. Less obvious is that the Windows 1.4 can't talk to the Linux 1.3 either because its only path to that is through the Linux eth0 card (even before I made 1.3 an alias).

Bridged mode lets the virtual machine share the host's Ethernet connection, while appearing as a separate machine with its own MAC and TCP/IP address.

NAT mode nats through the hosts NIC, much like the 192.168.1.x machines nat through your router to the internet (yes, yes, I know what the deep networking geeks are about to complain about: nat vs. masquerading). The VMware server assigns DHCP addresses to the virtual network cards, and the NAT system takes it from there through the real NIC (though that in turn is obviously natting or masquerading itself).

Finally, Host mode only allows the virtual machine to talk to the hosting machine and other virtual machines configured the same way, but nothing else.

After making the changes and resetting Windows to be .5, everything worked as it should. If he wants to add back the other NICS, he can, but at least he's starting from a known working configuration.



Got something to add? Send me email.





(OLDER) <- More Stuff -> (NEWER)    (NEWEST)   

Printer Friendly Version

-> -> VMware Networks, Bridged vs. Nat vs. Host


3 comments



Increase ad revenue 50-250% with Ezoic


More Articles by

Find me on Google+

© Anthony Lawrence







Tue Dec 23 17:46:36 2008: 4959   anonymous


So probably we can say that with virtualization you've got to be even more accurate with configuration.. Leaving things standing "as-is" often doesn't work, because there's much more going inside than seems obvious to the surface. It really is an article that's going to take couple of passes to get into it :)






Sun May 3 20:35:00 2009: 6306   Ikioi

gravatar
Thank you so much. I wanted to run pfSense on Ubuntu (because pfSense didn't have the drivers for a Linksys USB NIC, so I needed bridging). Not thinking that one device could share different IPs, I set my LAN ip to 192.168.1.1 and the vmnet bridge to 192.168.1.1 also. I thought that's how it was supposed to be.

The problem was solved by changing the IP like you did. Since the NIC in question was the DHCP via pfSense, I just set the Ubuntu to DHCP. It was hooked into the WAN port of a Belkin wifi router set to Access point. Luckily, the router had no problem looping the DHCP request back to the same line. The pfSense DHCP now runs all the LAN IPs, So I didn't have to worry about manual settings, since it will never conflict.

Thanks for your article, I was really getting frustrated after about 6 hours of playing with settings. I would just suggest to others that the let DHCP do all the work inside and outside of VMware bridged connections. Manual settings for devices can be reserved, and by ensuring inside and outside the VMware share the same DHCP, all conflicts can be automatically be avoided. Another consideration might be using different subnets on different NICs if they serve different purposes. No use wasting IP space on a larger network.



Thu May 19 08:01:37 2011: 9497   Firedancerx

gravatar


Hi Lawrence, this is really a rare one - a candid expose of what life is for a techie, and by that meaning how our work impacts our family life too! I really look forward to keep up with more of your candid postings! Bravo and a big thank you!

------------------------
Kerio Samepage


Have you tried Searching this site?

Unix/Linux/Mac OS X support by phone, email or on-site: Support Rates

This is a Unix/Linux resource website. It contains technical articles about Unix, Linux and general computing related subjects, opinion, news, help files, how-to's, tutorials and more.

Contact us





I define UNIX as 30 definitions of regular expressions living under one roof. (Donald Knuth)

If you think your users are idiots, only idiots will use it. (Linus Torvalds)








This post tagged: