NetApp vStorage Integration:
Intelligent Data Management for VMware
http://www.netapp.com/us/communities/tech-ontap/tot-vstorage-0309.html
As the many Tech OnTap articles about VMware over the
past several years attest, NetApp and VMware are long-time partners.
Our goal is to provide close integration with VMware to aim for the
highest possible functionality for anyone deploying VMware® with
NetApp® storage. Ongoing integration projects cover a number of areas,
including:
- Enhanced data protection (recent related article)
- Disaster recovery integration with SRM (recent related article)
- Multipathing
- MetroCluster
- Intelligent data management
In this article, I focus on intelligent data management.
Much of the discussion is based on a presentation I gave at VMworld
Europe 2009 in conjunction with Scott Davis, Chief data center
architect at VMware. (Here is the presentation slide set. For those with access to the VMworld site, you can view the presentation here.)
At VMworld 2008, VMware announced vStorage, an initiative to provide deeper levels of integration with storage functionality provided by partners such as NetApp.
NetApp has been working closely with VMware to refine the vStorage
APIs and to develop technologies that take advantage of them. (See the
sidebar.)
This article focuses on three areas:
- FlexClone® capabilities for file cloning
- Thin provisioning
- Intelligent data copy
In each area, I’ll try to explain what capabilities are
available now as well as provide a glimpse of the level of
integration you can expect in the future. Where appropriate, I’ll
provide links to the technology demos that we showed at VMworld
Europe.
Smart Cloning
The use of NetApp FlexClone with VMware has been
discussed in a number of previous articles, including a case study
about fast provisioning at North Carolina State University. A
companion article in this month’s issue discusses rapid provisioning
of VMware virtual desktops by using the NetApp Rapid Cloning Utilities
(RCU) version 1.
The ability to clone files quickly obviously has great
potential utility in a VMware environment. The standard virtual
machine (VM) provisioning process requires a full data copy, which can
take 5 to 20 minutes, depending on the size of the template VM. By
comparison, a file can be cloned by using FlexClone in a matter of
seconds. Cloning a VMDK file is just the first step. To make a
bootable clone of a virtual machine, you must perform all of the
following steps:
- Identify the source VM to be cloned.
- Clone the source virtual disk file with FlexClone.
- Create unique VMs, attaching a newly cloned virtual disk to each VM.
- Customize each VM to create a unique Windows® instance using a Virtual Center customization specification.
- Start each VM
RCU is a new tool that helps manage and automate these
steps. RCU takes advantage of the new FlexClone capability in Data
ONTAP® 7.3.1: the ability to clone individual files.
With this new capability, FlexClone can now be used to clone
individual VMDK files or entire data stores, resulting in the ability
to create thousands of virtual machines in minutes, while using only a
small amount of storage capacity beyond that required to store a
single virtual machine image.
NetApp TR-3705: NetApp and VMware VDI Best Practices
provides complete details of the process for using RCU version 1 to
rapidly clone virtual machines in a virtual desktop environment; the
process is the same for a virtual server environment.
Individual iSCSI and FCP RDMs and VMDK files in NFS
datastores can currently be cloned. Full vStorage integration enhances
performance for these use cases and adds the ability to clone VMFS
datastores on both iSCSI and FCP LUNs.
RCU version 2 is due out soon. This enhanced version
includes a plug-in for VMware Virtual Center. The following
demonstration shows the functionality it provides.
Figure 1) Demonstration of NetApp Rapid Cloning Utilities (RCU) integrated with VMware Virtual Center. (Runtime: 5 minutes)
You may be aware that VMware has introduced its own
space-efficient cloning capability for virtual desktop environments
called “linked clones.” In the future, through the vStorage APIs, this
capability will be plumbed directly to the NetApp file FlexClone
capability whenever the ESX host is connected to NetApp storage.
Figure 2) Animation illustrating the advantage of NetApp integration for cloning operations.
Thin Provisioning
Thin provisioning is a part of the core NetApp DNA.
When NetApp entered the SAN market it was the first major storage
vendor to offer thinly provisioned LUNs. However, there are two
challenges involved with thin provisioning, with SAN in general, and
with VMware in particular.
- SAN protocols don’t return any error codes when a storage container runs out of space; therefore there’s no defined behavior for what a storage system should do. If a volume runs out of space, a VM may crash.
- Guest OS file systems are built inside of LUNs on SAN storage. As the file system grows in size, it slowly consumes space in the LUN. However, file systems may also shrink in terms of the space they need, but there is no mechanism for a guest OS to tell an array that blocks are no longer in use and can be returned to the pool of free space.
Improved error handling. With regard to
the first challenge, NetApp has a good understanding of how to handle
running out of space and how to do thin provisioning in complex
environments. At NetApp’s urging, NetApp and VMware approached the
SCSI standards community about creating protocol extensions to enable
the SCSI protocol to better support thin provisioning.
This standard is now in place and VMware will be
enhancing how it operates in thinly provisioned environments. When
implemented, VMware will pause a virtual machine when it receives an
“End of Space” error, giving the administrator a chance to resume a VM
when more space is made available.
As an alternative, a thinly provisioned volume on
NetApp storage can currently be configured to “autogrow.” With this
feature you can set constraints on how big a LUN or volume can get;
when a set threshold is crossed, the LUN automatically receives
incrementally more space so that the operation of critical virtual
machines is not interrupted.
Figure 3) Thin provisioning
in a VMware environment. Individual volumes share space from the same
pool of storage. With the proposed extensions, VMware will pause
operations and allow volume size to be increased. NetApp allows space to
be increased without interruption.
Space reclamation. The second challenge
was described in detail in a previous Tech OnTap article. Briefly, if
you have a client file system in a thinly provisioned LUN and—for
instance—you create a large file and then delete it, the client file
system considers that space free and available; but from the
perspective of the storage system that space is still allocated to the
client. By running the space reclamation procedure from within NetApp
SnapDrive® running on the client operating system, you can
periodically free up such space and return it to the free storage pool
on the storage system, where it becomes available for use by any thinly
provisioned volume. This process works today on physical Windows
systems, and it also works in VMware environments in which individual
VMs mount LUNs directly by using NetApp SnapDrive.
To further enhance capabilities for thin provisioning,
NetApp and VMware are working on a number of additional enhancements:
- The ability to identify thinly provisioned LUNs using VMware Virtual Center (This prevents the use of VMware thin provisioning on a LUN that is already thin provisioned by the storage system).
- High utilization alerts that are sent from the storage system to Virtual Center
In addition, NetApp and VMware are exploring the following:
- The ability to automatically free up space when VMDK files get deleted from thinly provisioned volumes
- The ability to do space reclamation on VMFS datastores and NFS datastores
Data Copying and Storage vMotion
Storage vMotion is the storage analog of virtual machine
vMotion—data in active use by an ESX host can be transparently
migrated from one storage pool to another. This provides zero-downtime
data migration and simplifies storage maintenance, tiering, load
balancing upgrades, and other storage-related tasks.
In the standard VMware storage vMotion, this is
accomplished by copying the data from the source storage pool through
the ESX server and writing it out to the target storage. This process
is time consuming and can have significant performance impact on
hosts, storage, and networks.
NetApp has a long history of moving data efficiently
during data protection operations by using NetApp SnapMirror® or
SnapVault® software, NDMP, and so on. We are working with VMware to
connect the existing Data ONTAP data movement engines to storage
vMotion so that data can be copied directly from the source storage to
the target storage, bypassing the ESX host. If the source and
destination storage are on the same storage system, FlexClone is
automatically used (when appropriate) to accomplish the desired task.
Figure 4) Animation showing the advantages of integrating storage vMotion with NetApp storage.
Conclusion
In today’s business climate, doing more with less is a
necessity. NetApp’s goal is that whenever VMware needs to copy, clone,
or move a block of data, an ESX host can simply pass a NetApp storage
system the request and it will figure out the most efficient way to
accomplish the operation—freeing ESX server CPU cycles and bandwidth
for production work. NetApp FlexClone makes it possible to accomplish
many tasks that would otherwise require a full data copy in a fraction
of the time, using only incremental additional space. Closely
integrating NetApp’s advanced thin provisioning capabilities with
VMware further boosts storage efficiency for VMware virtual server and
virtual desktop environments.
|
Got opinions about vStorage and VMware data management?
Ask questions, exchange ideas, and share your thoughts online in NetApp communities. |
No comments:
Post a Comment