VMware vSphere Fault Tolerance

VMware Fault Tolerance (FT) is a new feature that is available with vSphere 4.0. VMware FT is based on vLockstep technology and aims to provide zero downtime, zero data loss, and continuous availability for applications. FT provides this continuous availability for applications by creating a secondary, live shadow instance of a virtual machine (VM) that is in virtual lockstep with the primary VM instance.

Next, VMware FT builds on VMware High Availability (HA). Therefore, HA has to be running correctly in order to enable FT. VMware FT can be turned on or off for individual virtual machines with a click of the mouse. In addition, since it leverages existing VMware HA clusters with a maximum node limit of 16 servers, any number of virtual machines in the cluster can be protected with VMware FT.

Finally, if you have set up VMware HA cluster and meet the Prerequisites and Requirements below, enabling FT is only a few mouse clicks away. Check out the step-by-step screen captures of enabling FT and testing failover with FT enabled at FT Example Down Below.

VMware vSphere FT Prerequisites

For VMware FT to perform as expected, it must run in an environment that meets specific requirements.

• The primary and secondary fault-tolerant virtual machines must be in a VMware HA cluster.
• Primary and secondary virtual machines must not run on the same host. FT automatically places the secondary virtual machine on a different host.
• Virtual machine files must be stored on shared storage.
• Shared storage solutions include NFS, FC, and iSCSI.
• For virtual disks on VMFS-3, the virtual disks must be thick, meaning they cannot be thin or sparsely allocated.
• Turning on VMware FT automatically converts the virtual machine to thick-eager zeroed disks.
• Virtual Raw Disk Mapping (RDM) is supported. Physical RDM is not supported.
• Multiple gigabit Network Interface Cards (NICs) are required.
• A minimum of two VMKernel Gigabit NICs must be dedicated to VMware FT Logging and vMotion.
• The FT Logging interface is used for logging events from the primary virtual machine to the secondary FT virtual machines.
• For best performance, use 10Gbit NIC rather than 1Gbit NIC, and enable the use of jumbo frames.
• VMware FT requires that Hardware Virtualization (HV) be turned on in the BIOS. The process for enabling HV varies among BIOSs. Contact your vendor for specifics.

VMware vSphere FT Requirements

There are a number of requirements which must be met before FT can be set up:

• CPUs: limited processors must be the same family (no mix/match).
• Requires Intel 31xx, 33xx, 52xx, 54xx, 55xx, 74xx or AMD 13xx,23xx, 83xx series of processors.
• SMP virtual machines are not supported.
• Hardware Virtualization must be enabled in the BIOS.
• Hosts must be in a VMware High Availability-enabled cluster.
• Storage: shared storage (FC, iSCSI, or NAS).
• Network: minimum of 3 NICs for various types of traffic (ESX Management/VMotion, virtual machine traffic, FT logging).
• GigE required for vMotion and FT logging.
• Minimized single points of failures in the environment. For example, NIC teaming, multiple network switches, storage multipathing.
• Primary and secondary hosts must be running the same build of ESX.

http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&externalId=1010601&sliceId=1&docTypeID=DT_KB_1_1&dialogID=29660642&stateId=1%200%2029664181

Guest Operating Systems

The following table displays guest operating system support for VMware FT. For specific guest operating system version information, see the Guest Operating System Installation Guide at http://www.vmware.com/pdf/GuestOS_guide.pdf.

The following values appear in the table:

• Yes - Virtual machine can be FT-enabled while powered on.
• Yes/Off - Virtual machine must be powered off before FT is enabled.
• No - Not supported by VMware FT.

Guest Operating System	Fault Tolerance Support with Intel Xeon Based on 45nm Core 2 Microarchitecture	Fault Tolerance Support with Intel Xeon Based on Core i7 Microarchitecture	Fault Tolerance Support with AMD 3rd Generation Opteron
Windows Server 2008	Yes	Yes/Off	Yes/Off
Windows Vista	Yes	Yes/Off	Yes/Off
Windows Server 2003 (64 bit)	Yes	Yes/Off	Yes/Off
Windows Server 2003 (32 bit)	Yes	Yes/Off	Yes/Off (Requires Service Pack 2 or greater)
Windows XP (64 bit)	Yes	Yes/Off	Yes/Off
Windows XP (32 bit)	Yes	Yes/Off	No
Windows 2000	Yes/Off	Yes/Off	No
Windows NT 4.0	Yes/Off	Yes/Off	No
Linux (all ESX-supported distributions)	Yes	Yes/Off	Yes/Off
Netware Server	Yes/Off	Yes/Off	Yes/Off
Solaris 10 (64-bit)	Yes	Yes/Off	Yes/Off (Requires Solaris U1)
Solaris 10 (32-bit)	Yes	Yes/Off	No
FreeBSD (all ESX-supported distributions)	Yes	Yes/Off	Yes/Off

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1008027

VMware vSphere FT Step by Step Example

1. Image depicting the Network configuration.




2. Turning on FT.



3. 54% through setting up FT.



4. FT setup complete.



5. vLockstep Interval and Log Bandwidth information are now updated.



6. Primary VM is located on 10.10.10.146.



7. Secondary VM is running on the Secondary Host- 10.10.10.145.



8. Testing FT using the built in Test Failover command.



9. Failover test completes. Notice that 10.10.10.146 has become the secondary location.



10. After the Failover Test, the primary host server is now 10.10.10.145 and the primary VM is running on it.



11. Furthermore, after the Failover Test, the secondary VM runs on the secondary host server, 10.10.10.146.



12. Failover test completion will result in showing the primary VM on the new primary host server and the new secondary host server with the accompanying vLockstep Interval and Log Bandwidth.



13. Finally, VMware vSphere FT is ready to provide continuous protection to your VMs.
 

VMware Fault Tolerance FAQ

Purpose

This article provides information for users of VMware Fault Tolerance (FT). The article contains Frequently Asked Questions that can help to resolve Fault Tolerance related issues.

Resolution

What is VMware Fault Tolerance?

VMware Fault Tolerance is a feature that allows a new level of guest redundancy. Information regarding this feature can be found in the vSphere Availability Guide for your version of ESX.

How do I turn it on?

The feature is enabled on a per virtual machine basis. Instructions for enabling Fault Tolerance can be found in the Turning on Fault Tolerance for Virtual Machines section of the vSphere Availability Guide for your version of ESX.

What happens when I turn on Fault Tolerance?

In very general terms, a second virtual machine is created to work in tandem with the virtual machine you have enabled Fault Tolerance on. This virtual machine resides on a different host in the cluster, and runs in virtual lockstep with the primary virtual machine. When a failure is detected, the second virtual machine takes the place of the first one with the least possible interruption of service. More specific information about how this is achieved can be found in the Protecting Mission-Critical Workloads with VMware Fault Tolerance whitepaper.

Why can't I turn Fault Tolerance on?

VMware Fault Tolerance can be enabled on any virtual machine that resides in a cluster that meets the necessary requirements. If you have difficulty enabling Fault Tolerance for a specific virtual machine, see The Turn on Fault Tolerance option is disabled (1010631).

How do I turn Fault Tolerance off?

Instructions for disabling Fault Tolerance can be found in the article in Disabling or Turning Off VMware FT (1008026).

How do I tell if my environment is ready for Fault Tolerance?

The VMware SiteSurvey Tool is used to check your environment for compliance with VMware Fault Tolerance. It can be downloaded at http://www.vmware.com/download/shared_utilities.html.

Where do I find the product's website?

VMware has a website for the Fault Tolerance product available online here at http://www.vmware.com/products/fault-tolerance/.

What happens during a failure?

When a host running the primary virtual machine fails, a transparent failover occurs to the corresponding secondary virtual machine. During this failover, there is no data loss or noticeable service interruption. In addition, VMware HA automatically restores redundancy by restarting a new secondary virtual machine on another host. Similarly, if the host running the secondary virtual machine fails, VMware HA starts a new secondary virtual machine on a different host. In either case there is no noticeable outage by an end user.

What is the logging time delay between the Primary and Secondary Fault Tolerance virtual machines?

The actual delay is based on the network latency between the Primary and Secondary. vLockstep executes the same instructions on the Primary and Secondary, but because this happens on different hosts, there could be a small latency, but no loss of state. This is typically less than 1 ms. Fault Tolerance includes synchronization to ensure that the Primary and Secondary are synchronized.

In a cluster with more than 3 hosts, can you tell Fault Tolerance where to put the Fault Tolerance virtual machine or does it chose on its own?

You can place the original (or Primary virtual machine). You have full control with DRS or VMotion to assign to it to any node. The placement of the Secondary, when created, is automatic based on the available hosts. But when the secondary is created and placed, you can VMotion it to the preferred host.

What happens if the host containing the primary virtual machine comes back online (after a node failure)?

This node is put back in the pool of available hosts. There is no attempt to start or migrate the primary to that host.

Is the failover from the primary virtual machine to the secondary virtual machine dynamic or does Fault Tolerance restart a virtual machine?

The failover from primary to secondary virtual machine is dynamic, with the secondary continuing execution from the exact point where the primary left off. It happens automatically with no data loss, no downtime, and little delay. Clients see no interruption. After the dynamic failover to the secondary virtual machine, it becomes the new primary virtual machine. A new secondary virtual machine is spawned automatically

Where are Fault Tolerance failover events logged?

All failover events are logged by vCenter.

I encountered an error message that I can't find in the knowledge base.  Where else should I check?

The vSphere Availability Guide contains a list of known errors in the Fault Tolerance Error Messages section.

Does Fault Tolerance support Intel Hyper-Threading Technology?Yes, Fault Tolerance does support Intel Hyper-Threading Technology on systems that have it enabled. Enabling or disabling Hyper-Threading has no impact on Fault Tolerance.

What happens if vCenter Server is offline when a failover event occurs?

Once Fault Tolerance is configured for a virtual machine, vCenter Server need not be online for FT to work. Even if vCenter Server is offline, failover will still occur from the primary to the secondary virtual machine. Additionally, the spawning of a new secondary virtual machine will also occur without vCenter Server.

Fault Tolerance Checklist 

Required c ESX/ESXi Hardware: Ensure that the processors are supported: AMD Barcelona+, Intel Penryn+ (run the CPU compatibility tool to determine compatibility).

Required c ESX/ESXi Hardware: Ensure that HV (Hardware Virtualization) is enabled in the BIOS.

Optional c ESX/ESXi Hardware: Ensure that power management (also known as power-capping) is turned OFF in the BIOS (performance implications).

Optional c ESX/ESXi Hardware: Ensure that hyper-threading is turned OFF in the BIOS (performance implications).

Required c Storage: Ensure that FT protected virtual machines are on shared storage (FC, iSCSI or NFS). When using NFS, increase timeouts and have a dedicated NIC for NFS traffic.

Required c Storage: Ensure that the datastore is not using physical RDM (Raw Disk Mapping). Virtual RDM is supported.

Required c Storage: Ensure that there is no requirement to use Storage VMotion for VMware FT VMs since Storage VMotion is not supported for VMware FT VMs.

Required c Storage: Ensure that NPIV (N-Port ID Virtualization) is not used since NPIV is not supported with VMware FT.

Optional c Storage: Ensure that virtual disks on VMFS3 are thick-eager zeroed (thin or sparsely allocated will be converted to thick-eager zeroed when VMware FT is enabled requiring additional storage space).

Optional c Storage: Ensure that ISOs used by the VMware FT protected VMs are on shared storage accessible to both primary and secondary VMs (else errors reported on secondary as if there is no media, which might be acceptable).

Optional c Network: Ensure that at least two NICs are used (NIC teaming) for ESX  management/VMotion and VMware FT logging. VMware recommends four VMkernel NICs: two dedicated for VMware VMotion and two dedicated for VMware FT.

Required c Network: Ensure that at least gigabit NICs are used (10 Gbit NICs can be used as well as jumbo frames enabled for better performance).

Optional c Redundancy: Ensure that the environment does not have a single point of failure (i.e. use NIC teaming, multiple network switches, and storage multipathing).

Required c vCenter Server: Ensure that the primary and secondary ESX hosts and virtual machines are in an HA-enabled cluster.

Required c vCenter Server: Ensure that there is no requirement to use DRS for VMware FT protected virtual machines; in this release VMware FT cannot be used with VMware DRS (although manual VMotion is allowed).

Required c vCenter Server: Ensure that host certificate checking is enabled (enabled by default) before you add the ESX/ESXi host to vCenter Server.

Required c ESX/ESXi: Ensure that the primary and secondary ESX/ESXi hosts are running the same build of VMware ESX/ESXi.

Required c Virtual Machines: Ensure that the virtual machines are NOT using more than 1 vCPU (SMP is not supported).

Required c Virtual Machines: Ensure that there is no user requirement to use NPT/EPT (Nested Page Tables/Extended Page Tables) since VMware FT disables NPT/EPT on the ESX host.

Required c Virtual Machines: Ensure that there is no user requirement to hot add or remove devices since hot plugging devices cannot be done with VMware FT.

Required c Virtual Machines: Ensure that there is no user requirement to use USB (USB must be disabled) and sound devices (must not be configured) since these are not supported for  ecord/Replay (and VMware FT).

Required c Virtual Machines: Ensure that there is no user requirement to have virtual machine snapshots since these are not supported for VMware FT. Delete snapshots from existing virtual machines before protecting with VMware FT.

Required c Virtual Machines: Ensure that virtual machine hardware is upgraded to v7.

Optional c Virtual Machines: Ensure that there are will be no more than four (to eight) VMware FT enabled virtual machine primaries or secondaries on any single ESX/ESXi host (suggested general guideline based on ESX/ESXi host and VM size and workloads which can vary).

Required c Guest OS: Ensure that the virtual machines do not use a paravirtualized guest OS.

Required c 3rd Party: Ensure MSCS clustered virtual machines will have MSCS clustering removed prior to protecting with VMware FT (and make sure that the virtual machines are not SMP).

Fault Tolerance

Fault Tolerance

Posted on May 14, 2012

Fault Tolerance (FT) is a feature of vSphere HA cluster. Hence to be able to use FT you need to configure an HA cluster first. FT is enabled on those VMs which need a higher level of protection. On enabling FT on the selected VM, a selected VM becomes primary and a new secondary VM is created on one of the hosts in the HA cluster.
For being able to configure FT on VMs:

• Configure a VMKernel portgroup enabled for FT Logging on all ESXi hosts which would host FT VMs
• Virtual disks on FT VMs have to be “Thick Provisioned Eager Zeroed”
• ISO/Floppy images have to be stored on Shared Storage
• Hosts in cluster Should meet vMotion requirements, (again I don’t think this is requirement per se but would be very beneficial)

FAQs on FT VMs:
Q. Does FT support vSMP on VMs?
A. No. VMs with vSMP cannot be enabled for FT. i.e. VMs with more than 1 vCPU cannot be protected with FT.
Q. Is FT supported on AMD processors.
A. Yes, both Intel & AMD processors are supported for FT. The VMware’s KB#1008027 article lists what processors and guest operating systems are supported for FT.
Q. How many FT VMs can be hosted on a single ESXi host?
A. Maximum 4 FT VMs supported per ESXi host (either Primaries or Secondaries).
Q. What is a primary VM?
A. Primary VM is a VM which is serving the user requests. The access to this VM is always read-write.
Q. What is a secondary VM?
A. Secondary is a backup VM, which is promoted to primary in case of (original) Primary VM failure. When it is a Secondary VM, the access to this VM is always read-only.

From: http://virtual-drive.in/