How to configure NIC teaming in VMware vSphere

VMware Fault Tolerance FAQ

Purpose

This article provides information for users of VMware Fault Tolerance (FT). The article contains Frequently Asked Questions that can help to resolve Fault Tolerance related issues.

Resolution

What is VMware Fault Tolerance?

VMware Fault Tolerance is a feature that allows a new level of guest redundancy. Information regarding this feature can be found in the vSphere Availability Guide for your version of ESX.

How do I turn it on?

The feature is enabled on a per virtual machine basis. Instructions for enabling Fault Tolerance can be found in the Turning on Fault Tolerance for Virtual Machines section of the vSphere Availability Guide for your version of ESX.

What happens when I turn on Fault Tolerance?

In very general terms, a second virtual machine is created to work in tandem with the virtual machine you have enabled Fault Tolerance on. This virtual machine resides on a different host in the cluster, and runs in virtual lockstep with the primary virtual machine. When a failure is detected, the second virtual machine takes the place of the first one with the least possible interruption of service. More specific information about how this is achieved can be found in the Protecting Mission-Critical Workloads with VMware Fault Tolerance whitepaper.

Why can't I turn Fault Tolerance on?

VMware Fault Tolerance can be enabled on any virtual machine that resides in a cluster that meets the necessary requirements. If you have difficulty enabling Fault Tolerance for a specific virtual machine, see The Turn on Fault Tolerance option is disabled (1010631).

How do I turn Fault Tolerance off?

Instructions for disabling Fault Tolerance can be found in the article in Disabling or Turning Off VMware FT (1008026).

How do I tell if my environment is ready for Fault Tolerance?

The VMware SiteSurvey Tool is used to check your environment for compliance with VMware Fault Tolerance. It can be downloaded at http://www.vmware.com/download/shared_utilities.html.

Where do I find the product's website?

VMware has a website for the Fault Tolerance product available online here at http://www.vmware.com/products/fault-tolerance/.

What happens during a failure?

When a host running the primary virtual machine fails, a transparent failover occurs to the corresponding secondary virtual machine. During this failover, there is no data loss or noticeable service interruption. In addition, VMware HA automatically restores redundancy by restarting a new secondary virtual machine on another host. Similarly, if the host running the secondary virtual machine fails, VMware HA starts a new secondary virtual machine on a different host. In either case there is no noticeable outage by an end user.

What is the logging time delay between the Primary and Secondary Fault Tolerance virtual machines?

The actual delay is based on the network latency between the Primary and Secondary. vLockstep executes the same instructions on the Primary and Secondary, but because this happens on different hosts, there could be a small latency, but no loss of state. This is typically less than 1 ms. Fault Tolerance includes synchronization to ensure that the Primary and Secondary are synchronized.

In a cluster with more than 3 hosts, can you tell Fault Tolerance where to put the Fault Tolerance virtual machine or does it chose on its own?

You can place the original (or Primary virtual machine). You have full control with DRS or VMotion to assign to it to any node. The placement of the Secondary, when created, is automatic based on the available hosts. But when the secondary is created and placed, you can VMotion it to the preferred host.

What happens if the host containing the primary virtual machine comes back online (after a node failure)?

This node is put back in the pool of available hosts. There is no attempt to start or migrate the primary to that host.

Is the failover from the primary virtual machine to the secondary virtual machine dynamic or does Fault Tolerance restart a virtual machine?

The failover from primary to secondary virtual machine is dynamic, with the secondary continuing execution from the exact point where the primary left off. It happens automatically with no data loss, no downtime, and little delay. Clients see no interruption. After the dynamic failover to the secondary virtual machine, it becomes the new primary virtual machine. A new secondary virtual machine is spawned automatically

Where are Fault Tolerance failover events logged?

All failover events are logged by vCenter.

I encountered an error message that I can't find in the knowledge base.  Where else should I check?

The vSphere Availability Guide contains a list of known errors in the Fault Tolerance Error Messages section.

Does Fault Tolerance support Intel Hyper-Threading Technology?Yes, Fault Tolerance does support Intel Hyper-Threading Technology on systems that have it enabled. Enabling or disabling Hyper-Threading has no impact on Fault Tolerance.

What happens if vCenter Server is offline when a failover event occurs?

Once Fault Tolerance is configured for a virtual machine, vCenter Server need not be online for FT to work. Even if vCenter Server is offline, failover will still occur from the primary to the secondary virtual machine. Additionally, the spawning of a new secondary virtual machine will also occur without vCenter Server.

Fault Tolerance Checklist 

Required c ESX/ESXi Hardware: Ensure that the processors are supported: AMD Barcelona+, Intel Penryn+ (run the CPU compatibility tool to determine compatibility).

Required c ESX/ESXi Hardware: Ensure that HV (Hardware Virtualization) is enabled in the BIOS.

Optional c ESX/ESXi Hardware: Ensure that power management (also known as power-capping) is turned OFF in the BIOS (performance implications).

Optional c ESX/ESXi Hardware: Ensure that hyper-threading is turned OFF in the BIOS (performance implications).

Required c Storage: Ensure that FT protected virtual machines are on shared storage (FC, iSCSI or NFS). When using NFS, increase timeouts and have a dedicated NIC for NFS traffic.

Required c Storage: Ensure that the datastore is not using physical RDM (Raw Disk Mapping). Virtual RDM is supported.

Required c Storage: Ensure that there is no requirement to use Storage VMotion for VMware FT VMs since Storage VMotion is not supported for VMware FT VMs.

Required c Storage: Ensure that NPIV (N-Port ID Virtualization) is not used since NPIV is not supported with VMware FT.

Optional c Storage: Ensure that virtual disks on VMFS3 are thick-eager zeroed (thin or sparsely allocated will be converted to thick-eager zeroed when VMware FT is enabled requiring additional storage space).

Optional c Storage: Ensure that ISOs used by the VMware FT protected VMs are on shared storage accessible to both primary and secondary VMs (else errors reported on secondary as if there is no media, which might be acceptable).

Optional c Network: Ensure that at least two NICs are used (NIC teaming) for ESX  management/VMotion and VMware FT logging. VMware recommends four VMkernel NICs: two dedicated for VMware VMotion and two dedicated for VMware FT.

Required c Network: Ensure that at least gigabit NICs are used (10 Gbit NICs can be used as well as jumbo frames enabled for better performance).

Optional c Redundancy: Ensure that the environment does not have a single point of failure (i.e. use NIC teaming, multiple network switches, and storage multipathing).

Required c vCenter Server: Ensure that the primary and secondary ESX hosts and virtual machines are in an HA-enabled cluster.

Required c vCenter Server: Ensure that there is no requirement to use DRS for VMware FT protected virtual machines; in this release VMware FT cannot be used with VMware DRS (although manual VMotion is allowed).

Required c vCenter Server: Ensure that host certificate checking is enabled (enabled by default) before you add the ESX/ESXi host to vCenter Server.

Required c ESX/ESXi: Ensure that the primary and secondary ESX/ESXi hosts are running the same build of VMware ESX/ESXi.

Required c Virtual Machines: Ensure that the virtual machines are NOT using more than 1 vCPU (SMP is not supported).

Required c Virtual Machines: Ensure that there is no user requirement to use NPT/EPT (Nested Page Tables/Extended Page Tables) since VMware FT disables NPT/EPT on the ESX host.

Required c Virtual Machines: Ensure that there is no user requirement to hot add or remove devices since hot plugging devices cannot be done with VMware FT.

Required c Virtual Machines: Ensure that there is no user requirement to use USB (USB must be disabled) and sound devices (must not be configured) since these are not supported for  ecord/Replay (and VMware FT).

Required c Virtual Machines: Ensure that there is no user requirement to have virtual machine snapshots since these are not supported for VMware FT. Delete snapshots from existing virtual machines before protecting with VMware FT.

Required c Virtual Machines: Ensure that virtual machine hardware is upgraded to v7.

Optional c Virtual Machines: Ensure that there are will be no more than four (to eight) VMware FT enabled virtual machine primaries or secondaries on any single ESX/ESXi host (suggested general guideline based on ESX/ESXi host and VM size and workloads which can vary).

Required c Guest OS: Ensure that the virtual machines do not use a paravirtualized guest OS.

Required c 3rd Party: Ensure MSCS clustered virtual machines will have MSCS clustering removed prior to protecting with VMware FT (and make sure that the virtual machines are not SMP).

Vmware vSphere 5 Documentation is now available. 

Download now. http://pubs.vmware.com/vsphere-50/index.jsp

vSphere Networking Basics

The key virtual networking components in a VMware Infrastructure are virtual Ethernet adapters and virtual switches. A virtual machine can be configured with one or more virtual Ethernet adapters. Virtual switches allow virtual machines on the same VMware ESX or ESXi host to communicate with each other using the same protocols that would be used over physical switches, without the need for additional hardware. They also support VLANS that are compatible with standard VLAN implementations from other vendors, such as Cisco.

Connecting Virtual Machines to Your Network

VMware technology lets you link local virtual machines to each other and to the external enterprise network through the virtual switch. The virtual switch emulates a traditional physical Ethernet network switch to the extent that it forwards frames at the data link layer. VMware ESX may contain multiple virtual switches, each providing more than 1,000 internal virtual ports for virtual machine use.

The virtual switch connects to the enterprise network through outbound Ethernet adapters. A maximum of twenty-four Gigabit Ethernet ports or ten 10/100 Ethernet ports can be used by the virtual switch for external connectivity. The virtual switch is capable of binding multiple VMNICs together, in a manner much like NIC teaming on a traditional server, offering greater availability and bandwidth to the virtual machines using the virtual switch.

Virtual Ethernet Adapters

There are many types of adapters available for virtual machines in VMware vSphere:

Vlance: An emulated version of the AMD 79C970 PCnet32 LANCE NIC, an older 10 Mbps NIC with drivers available in most 32bit guest operating systems except Windows Vista and later. A virtual machine configured with this network adapter can use its network immediately.
VMXNET: The VMXNET virtual network adapter has no physical counterpart. VMXNET is optimized for performance in a virtual machine. Because operating system vendors do not provide built-in drivers for this card, you must install VMware Tools to have a driver for the VMXNET network adapter available.
Flexible: The Flexible network adapter identifies itself as a Vlance adapter when a virtual machine boots, but initializes itself and functions as either a Vlance or a VMXNET adapter, depending on which driver initializes it. With VMware Tools installed, the VMXNET driver changes the Vlance adapter to the higher performance VMXNET adapter.
E1000:An emulated version of the Intel 82545EM Gigabit Ethernet NIC. A driver for this NIC is not included with all guest operating systems. Typically Linux versions 2.4.19 and later, Windows XP Professional x64 Edition and later, and Windows Server 2003 (32-bit) and later include the E1000 driver.
VMXNET 2 (Enhanced): The VMXNET 2 adapter is based on the VMXNET adapter but provides some high-performance features commonly used on modern networks, such as jumbo frames and hardware offloads. This virtual network adapter is available only for some guest operating systems on ESX/ESXi 3.5 and later.
VMXNET 3: The VMXNET 3 adapter is the next generation of a paravirtualized NIC designed for performance, and is not related to VMXNET or VMXNET 2. It offers all the features available in VMXNET 2, and adds several new features like multiqueue support (also known as Receive Side Scaling in Windows), IPv6 offloads, and MSI/MSI-X interrupt delivery.
VMXNET 3 is supported only for virtual machines version 7 and later, with a limited set of guest operating systems:

32 and 64bit versions of Microsoft Windows XP, 2003, 2003 R2, 2008,and 2008 R2
32 and 64bit versions of Red Hat Enterprise Linux 5.0 and later
32 and 64bit versions of SUSE Linux Enterprise Server 10 and later
32 and 64bit versions of Asianux 3 and later
32 and 64bit versions of Debian 4
32 and 64bit versions of Ubuntu 7.04 and later
32 and 64bit versions of Sun Solaris 10 U4 and later
There are two other virtual adapters available through VMware technology. Vswif is a paravirtualized device similar to vmxnet that is used by the VMware ESX service console. Vmknic is a device in the VMkernal that is used by the TCP/IP stack to serve NFS and software iSCSI clients.

Virtual Switches

VMware technology includes virtual switches that you can build on demand at run-time to provide different functions, including:

Layer 2 forwarding
VLAN tagging, stripping and filtering
Layer 2 security, checksum and segmentation offloading
This modular approach reduces complexity and maximizes system performance, VMware virtualization technology loads only those components it needs to support the specific physical and virtual Ethernet adapter types used in the configuration. Additionally, the modular design enables VMware and third-party developers to incorporate new modules to enhance the system in the future. Up to 248 virtual switches can be created on each VMware ESX host. Following are important features of virtual switches:

Virtual ports: The ports on a virtual switch provide logical connection points among virtual devices and between virtual and physical devices. Each virtual switch can have up to 1,016 virtual ports, with a limit of 4,096 ports on all virtual switches on a host. The virtual ports provide a rich control channel for communication with the virtual Ethernet adapters attached to them.
Uplink ports: Uplink ports are associated with physical adapters, providing a connection between the virtual network and the physical networks. They connect to physical adapters when they are initialized by a device driver or when the teaming policies for virtual switches are reconfigured. Virtual Ethernet adapters connect to virtual ports when you power on the virtual machine, when you take an action to connect the device, or when you migrate a virtual machine using VMware VMotion. A virtual Ethernet adapter updates the virtual switch port with MAC filtering information when it is initialized or when it changes.
Port groups: Port groups make it possible to specify that a given virtual machine should have a particular type of connectivity on every host, and they contain enough configuration information to provide persistent and consistent network access for virtual Ethernet adapters. Some of the information contained in a port group includes virtual switch name, VLANIDs and policies for tagging and filtering, the teaming policy and traffic shaping parameters. This is all the information needed for a switch port.
Uplinks: With VMware technology, uplinks are the physical Ethernet adapters that serve as bridges between the virtual and physical network. The virtual ports connected to them are called uplink ports. A host may have up to 32 uplinks.
Other things to consider:

Virtual switches do not learn from the network to populate their forward tables. This helps to minimize denial of service attacks.
Virtual switches make private copies of frame data used to make forwarding or filtering decisions. This ensures the guest operating systems cannot access sensitive data once the frame is passed onto the virtual switch.
VMware technology ensures that frames are contained within the appropriate VLAN on a virtual switch 1) by carrying the data outside the frame as it passes through the virtual switch, and 2) because there is no dynamic trunking support that could open up isolation leaks, making the data vulnerable to attack.
Virtual Switches vs. Physical Switches

Virtual switches are similar to modern physical Ethernet switches in many ways. Like a physical switch, it maintains a MAC:port forward table and performs frame destination lookup and frame forwarding. It also supports VLAN segmentation at the port level, so that each port can be configured as an access or trunk port, providing access to either single or multiple VLANs.

However, unlike physical switches, virtual switches do not require a spanning tree protocol, because VMware vSphere enforces a single-tier networking topology. There’s no way to interconnect multiple virtual switches. Also, network traffic cannot flow directly from one virtual switch to another within the same host. Virtual switches provide all the ports you need in one switch. You don’t need to cascade virtual switches or prevent bad virtual switch connections, and because they don’t share physical Ethernet adapters, leaks between switches do not occur. Each virtual switch is isolated and has its own forwarding table, so every destination the switch looks up can match only ports on the same virtual switch where the frame originated. This feature improves security, making it difficult for hackers to break virtual switch isolation

VMWare Interview Questions & Tips

1. VMWare Kernel is a Proprietary Kenral and is not based on any of the UNIX operating systems, it's a kernel developed by VMWare Company.

2. The VMKernel can't boot it by itself, so that it takes the help of the 3rd party operating system. In VMWare case the kernel is booted by RedHat Linux operating system which is known as service console.

3. The service console is developed based up on Redhat Linux Operating system, it is used to manage the VMKernel

4. To restart webaccess service on vmware
service vmware-webaccess restart – this will restart apache tomcat app

5. To restart ssh service on vmware
service sshd restart

6. To restart host agent(vmware-hostd) on vmware esx server
service mgmt-vmware restart

7. Path for the struts-config.xml
/usr/lib/vmware/webAccess/tomcat/apache-tomcat-5.5.17/webapps/ui/WEB-INF/

8. To start the scripted install the command is
esx ks=nfs:111.222.333.444:/data/KS.config ksdevice=eth0
location device name

9. Virtual Network in Simple……………….

Virtual Nic(s) on Virtual Machine(s) ----->

Physical Nic on the ESX Server (Virtual Switch - 56 Ports) ----->

Physical Switch Port Should be trunked with all the VLANS to which the VM's need access

All the ESX servers should be configured with Same number of Physical Nics (vSwitches) and Connectivity also should be same, So that vMotion succeeds
All the Virtual Machines are connected to one vSwitch with Different VLANS, this means the Physical Nic(vSwitch) needs to be trunked with the same VLANS on the Physical Switch Port

10 What are the three port groups present in ESX server networking
1. Virtual Machine Port Group - Used for Virtual Machine Network
2. Service Console Port Group - Used for Service Console Communications
3. VMKernel Port Group - Used for VMotion, iSCSI, NFS Communications

11. What is the use of a Port Group?
The port group segregates the type of communication.

12. What are the type of communications which requires an IP address for sure ?
Service Console and VMKernel (VMotion and iSCSI), these communications does not happen without an ip address (Whether it is a single or dedicated)

13. In the ESX Server licensing features VMotion License is showing as Not used, why?
Even though the license box is selected, it shows as "License Not Used" until, you enable the VMotion option for specific vSwitch


14. How the Virtual Machine Port group communication works ?
All the vm's which are configured in VM Port Group are able to connect to the physical machines on the network. So this port group enables communication between vSwitch and Physical Switch to connect vm's to Physical Machine's

15. What is a VLAN ?
A VLAN is a logical configuration on the switch port to segment the IP Traffic. For this to happen, the port must be trunked with the correct VLAN ID.

16. Does the vSwitches support VLAN Tagging? Why?
Yes, The vSwitches support VLAN Tagging, otherwise if the virtual machines in an esx host are connected to different VLANS, we need to install a separate physical nic (vSwitch) for every VLAN. That is the reason vmware included the VLAN tagging for vSwitches. So every vSwitch supports upto 1016 ports, and BTW they can support 1016 VLANS if needed, but an ESX server doesn’t support that many VM’s. :)

17. What is Promiscuous Mode on vSwitch ? What happens if it sets to Accept?
If the promiscuous mode set to Accept, all the communication is visible to all the virtual machines, in other words all the packets are sent to all the ports on vSwitch
If the promiscuous mode set to Reject, the packets are sent to inteded port, so that the intended virtual machine was able to see the communication.

18. What is MAC address Changes ? What happens if it is set to Accept ?
When we create a virtual machine the configuration wizard generates a MAC address for that machine, you can see it in the .vmx (VM Config) file. If it doesn't matches with the MAC address in the OS this setting does not allow incoming traffic to the VM. So by setting Reject Option both MAC addresses will be remains same, and the incoming traffic will be allowed to the VM.

19. What is Forged Transmits ? What happens if it is set to Accept ?
When we create a virtual machine the configuration wizard generates a MAC address for that machine, you can see it in the .vmx (VM Config) file. If it doesn't matches with the MAC address in the OS this setting does not allow outgoing traffic from the VM. So by setting Reject Option both MAC addresses will be remains same, and the outgoing traffic will be allowed from the VM.

20. What are the core services of VC ?
VM provisioning , Task Scheduling and Event Logging

21. Can we do vMotion between two datacenters ? If possible how it will be?
Yes we can do vMotion between two datacenters, but the mandatory requirement is the VM should be powered off.

22. What is VC agent? and what service it is corresponded to? What are the minimum req's for VC agent installation ?
VC agent is an agent installed on ESX server which enables communication between VC and ESX server.
The daemon associated with it is called vmware-hostd , and the service which corresponds to it is called as mgmt-vmware, in the event of VC agent failure just restart the service by typing the following command at the service console

" service mgmt-vmware restart "
VC agent installed on the ESX server when we add it to the VC, so at the time of installtion if you are getting an error like " VC Agent service failed to install ", check the /Opt size whether it is sufficient or not.

23. How can you edit VI Client Settings and VC Server Settings ?
Click Edit Menu on VC and Select Client Settings to change VI settings
Click Administration Menu on VC and Select VC Management Server Configuration to Change VC Settings

24. What are the files that make a Virtual Machine ?
.vmx - Virtual Machine Configuration File
.nvram - Virtual Machine BIOS
.vmdk - Virtual Machine Disk file
.vswp - Virtual Machine Swap File
.vmsd - Virtual MAchine Snapshot Database
.vmsn - Virtual Machine Snapshot file
.vmss - Virtual Machine Suspended State file
.vmware.log - Current Log File
.vmware-#.log - Old Log file

25. What are the devices that can be added while the virtual Machine running
In VI 3.5 we can add Hard Disk and NIC's while the machine running.

In vSphere 4.0 we can add Memory and Processor along with HDD and NIC's while the machine running

26. How to set the time delay for BIOS screen for a Virtual Machine?
Right Click on VM, select edit settings, choose options tab and select boot option, set the delay how much you want.

27. What is a template ?
We can convert a VM into Template, and it cannot be powered on once its changed to template. This is used to quick provisioning of VM's.

23. What to do to customize the windows virtual machine clone,?
copy the sysprep files to Virtual center directory on the server, so that the wizard will take the advantage of it.

24. What to do to customize the linux/unix virtual machine clone,?
VC itself includes the customization tools, as these operating systems are available as open source.

25. Does cloning from template happens between two datacenters ?
Yes.. it can, if the template in one datacenter, we can deploy the vm from that template in another datacenter without any problem.

26. What are the common issues with snapshots? What stops from taking a snapshot and how to fix it ?
If you configure the VM with Mapped LUN's, then the snapshot failed. If it is mapped as virtual then we can take a snapshot of it.
If you configure the VM with Mapped LUN's as physical, you need to remove it to take a snapshot.

27. What are the settings that are taken into to consideration when we initiate a snapshot ?
Virtual Machine Configuration (What hardware is attached to it)
State of the Virtual Machine Hard Disk file ( To revert back if needed)
State of the Virtual Machine Memory (if it is powered on)

28. What are the requirements for Converting a Physical machine to VM ?
An agent needs to be installed on the Physical machine
VI client needs to be installed with Converter Plug-in
A server to import/export virtual machines

29. What is VMWare consolidated backup ?
It is a backup framework, that supports 3rd party utilities to take backups of ESX servers and Virtual Machines. Its not a backup service.

30. To open the guided consolidation tool, what are the user requirements ?
The user must be member of administrator, The user should have "Logon as service" privileges - To give a user these privileges, open local sec policy, select Logon as service policy and add the user the user should have read access to AD to send queries