NetApp vStorage Integration: Intelligent Data Management for VMware

NetApp vStorage Integration:
Intelligent Data Management for VMware

http://www.netapp.com/us/communities/tech-ontap/tot-vstorage-0309.html

By Arthur Lent, NetApp

As the many Tech OnTap articles about VMware over the past several years attest, NetApp and VMware are long-time partners. Our goal is to provide close integration with VMware to aim for the highest possible functionality for anyone deploying VMware® with NetApp® storage. Ongoing integration projects cover a number of areas, including:

• Enhanced data protection (recent related article)
• Disaster recovery integration with SRM (recent related article)
• Multipathing
• MetroCluster
• Intelligent data management

In this article, I focus on intelligent data management. Much of the discussion is based on a presentation I gave at VMworld Europe 2009 in conjunction with Scott Davis, Chief data center architect at VMware. (Here is the presentation slide set. For those with access to the VMworld site, you can view the presentation here.)

At VMworld 2008, VMware announced vStorage, an initiative to provide deeper levels of integration with storage functionality provided by partners such as NetApp. NetApp has been working closely with VMware to refine the vStorage APIs and to develop technologies that take advantage of them. (See the sidebar.)

This article focuses on three areas:

• FlexClone® capabilities for file cloning
• Thin provisioning
• Intelligent data copy

In each area, I’ll try to explain what capabilities are available now as well as provide a glimpse of the level of integration you can expect in the future. Where appropriate, I’ll provide links to the technology demos that we showed at VMworld Europe.

Smart Cloning

The use of NetApp FlexClone with VMware has been discussed in a number of previous articles, including a case study about fast provisioning at North Carolina State University. A companion article in this month’s issue discusses rapid provisioning of VMware virtual desktops by using the NetApp Rapid Cloning Utilities (RCU) version 1.

The ability to clone files quickly obviously has great potential utility in a VMware environment. The standard virtual machine (VM) provisioning process requires a full data copy, which can take 5 to 20 minutes, depending on the size of the template VM. By comparison, a file can be cloned by using FlexClone in a matter of seconds. Cloning a VMDK file is just the first step. To make a bootable clone of a virtual machine, you must perform all of the following steps:

• Identify the source VM to be cloned.
• Clone the source virtual disk file with FlexClone.
• Create unique VMs, attaching a newly cloned virtual disk to each VM.
• Customize each VM to create a unique Windows® instance using a Virtual Center customization specification.
• Start each VM

RCU is a new tool that helps manage and automate these steps. RCU takes advantage of the new FlexClone capability in Data ONTAP® 7.3.1: the ability to clone individual files. With this new capability, FlexClone can now be used to clone individual VMDK files or entire data stores, resulting in the ability to create thousands of virtual machines in minutes, while using only a small amount of storage capacity beyond that required to store a single virtual machine image.

NetApp TR-3705: NetApp and VMware VDI Best Practices provides complete details of the process for using RCU version 1 to rapidly clone virtual machines in a virtual desktop environment; the process is the same for a virtual server environment.

Individual iSCSI and FCP RDMs and VMDK files in NFS datastores can currently be cloned. Full vStorage integration enhances performance for these use cases and adds the ability to clone VMFS datastores on both iSCSI and FCP LUNs.

RCU version 2 is due out soon. This enhanced version includes a plug-in for VMware Virtual Center. The following demonstration shows the functionality it provides.

Figure 1) Demonstration of NetApp Rapid Cloning Utilities (RCU) integrated with VMware Virtual Center. (Runtime: 5 minutes)

You may be aware that VMware has introduced its own space-efficient cloning capability for virtual desktop environments called “linked clones.” In the future, through the vStorage APIs, this capability will be plumbed directly to the NetApp file FlexClone capability whenever the ESX host is connected to NetApp storage.

Figure 2) Animation illustrating the advantage of NetApp integration for cloning operations.

Thin Provisioning

Thin provisioning is a part of the core NetApp DNA. When NetApp entered the SAN market it was the first major storage vendor to offer thinly provisioned LUNs. However, there are two challenges involved with thin provisioning, with SAN in general, and with VMware in particular.

• SAN protocols don’t return any error codes when a storage container runs out of space; therefore there’s no defined behavior for what a storage system should do. If a volume runs out of space, a VM may crash.
• Guest OS file systems are built inside of LUNs on SAN storage. As the file system grows in size, it slowly consumes space in the LUN. However, file systems may also shrink in terms of the space they need, but there is no mechanism for a guest OS to tell an array that blocks are no longer in use and can be returned to the pool of free space.

Improved error handling. With regard to the first challenge, NetApp has a good understanding of how to handle running out of space and how to do thin provisioning in complex environments. At NetApp’s urging, NetApp and VMware approached the SCSI standards community about creating protocol extensions to enable the SCSI protocol to better support thin provisioning.

This standard is now in place and VMware will be enhancing how it operates in thinly provisioned environments. When implemented, VMware will pause a virtual machine when it receives an “End of Space” error, giving the administrator a chance to resume a VM when more space is made available.

As an alternative, a thinly provisioned volume on NetApp storage can currently be configured to “autogrow.” With this feature you can set constraints on how big a LUN or volume can get; when a set threshold is crossed, the LUN automatically receives incrementally more space so that the operation of critical virtual machines is not interrupted.

Figure 3) Thin provisioning in a VMware environment. Individual volumes share space from the same pool of storage. With the proposed extensions, VMware will pause operations and allow volume size to be increased. NetApp allows space to be increased without interruption.

Space reclamation. The second challenge was described in detail in a previous Tech OnTap article. Briefly, if you have a client file system in a thinly provisioned LUN and—for instance—you create a large file and then delete it, the client file system considers that space free and available; but from the perspective of the storage system that space is still allocated to the client. By running the space reclamation procedure from within NetApp SnapDrive® running on the client operating system, you can periodically free up such space and return it to the free storage pool on the storage system, where it becomes available for use by any thinly provisioned volume. This process works today on physical Windows systems, and it also works in VMware environments in which individual VMs mount LUNs directly by using NetApp SnapDrive.

To further enhance capabilities for thin provisioning, NetApp and VMware are working on a number of additional enhancements:

• The ability to identify thinly provisioned LUNs using VMware Virtual Center (This prevents the use of VMware thin provisioning on a LUN that is already thin provisioned by the storage system).
• High utilization alerts that are sent from the storage system to Virtual Center

In addition, NetApp and VMware are exploring the following:

• The ability to automatically free up space when VMDK files get deleted from thinly provisioned volumes
• The ability to do space reclamation on VMFS datastores and NFS datastores

Data Copying and Storage vMotion

Storage vMotion is the storage analog of virtual machine vMotion—data in active use by an ESX host can be transparently migrated from one storage pool to another. This provides zero-downtime data migration and simplifies storage maintenance, tiering, load balancing upgrades, and other storage-related tasks.

In the standard VMware storage vMotion, this is accomplished by copying the data from the source storage pool through the ESX server and writing it out to the target storage. This process is time consuming and can have significant performance impact on hosts, storage, and networks.

NetApp has a long history of moving data efficiently during data protection operations by using NetApp SnapMirror® or SnapVault® software, NDMP, and so on. We are working with VMware to connect the existing Data ONTAP data movement engines to storage vMotion so that data can be copied directly from the source storage to the target storage, bypassing the ESX host. If the source and destination storage are on the same storage system, FlexClone is automatically used (when appropriate) to accomplish the desired task.

Figure 4) Animation showing the advantages of integrating storage vMotion with NetApp storage.

Conclusion

In today’s business climate, doing more with less is a necessity. NetApp’s goal is that whenever VMware needs to copy, clone, or move a block of data, an ESX host can simply pass a NetApp storage system the request and it will figure out the most efficient way to accomplish the operation—freeing ESX server CPU cycles and bandwidth for production work. NetApp FlexClone makes it possible to accomplish many tasks that would otherwise require a full data copy in a fraction of the time, using only incremental additional space. Closely integrating NetApp’s advanced thin provisioning capabilities with VMware further boosts storage efficiency for VMware virtual server and virtual desktop environments.

Got opinions about vStorage and VMware data management? Ask questions, exchange ideas, and share your thoughts online in NetApp communities.

Arthur Lent Technical Director NetApp Arthur is the chief architect for the NetApp Virtualization and Grid Infrastructure Business Unit. As an expert in storage for virtual environments, he is responsible for defining and driving NetApp product capabilities for server virtualization technologies, including VMware ESX, Microsoft® Hyper-V®, and Citrix XenServer. Previously, Arthur was the lead architect responsible for the development of NetApp FC SAN and iSCSI products, providing the technology foundation that enabled NetApp to become a leading iSCSI and FC SAN vendor.

Microsoft Office Labs vision 2019 (montage + video)

Microsoft Office Labs vision 2019 (montage + video)

India to lead world in cloud computing: Ballmer

EconomicTimesofIndia.com

Tags:

• Zinnov Management Consulting|
• Terremark

NEW DELHI: Microsoft Corp sees India as the global hub for cloud computing, the concept of renting computing power that has taken the technology world by storm and in the words of the NYT, what Silicon
Valley cannot seem to get its head out of. ( Watch )
"India will not only see a surge in cloud computing services but companies all over the world will look to India to support their transition to cloud computing,'' Microsoft CEO Steve Ballmer said on Thursday.
The world's biggest software maker is among a handful of companies betting big on cloud services, aiming to convince enterprises to give up building and managing data centres and switch to their computer capacity instead; the others being rivals such as Amazon, Google and AT&T as well as smaller firms like Rackspace and Terremark.

Microsoft believes India will move directly to the cloud, much like it bypassed the landline revolution that never happened and leapt to mobile phones, Mr Ballmer told a packed press meet. Mr Ballmer is in India to underline the importance of the company's cloud services platform Azure, wherein people can use applications from email to payroll systems hosted online.
The transition that India will champion will seed 3 lakh jobs in five years, during which the business is estimated to grow to $70 billion, Microsoft said, quoting a study by Zinnov Management Consulting. Jobs will be generated in areas like cloud consulting, enabling software as a service, integrating offerings like Azure with IBM's Blue Cloud or salesforce.com's customer applications on cloud, and creating new applications.
For Indian businesses too, there is great potential, given that 30%, or $7 billion, of the global cloud computing work is to be offshored, said the Zinnov study.
Microsoft already has more than 600 customers for its cloud services, but wants a deeper head start over rivals after catcalls of playing catch-up in other tech fields have been growing louder by the day. Indeed, Mr Ballmer was speaking to reporters a day after Apple overtook his company as the world's biggest technology firm in terms of market value, more than a decade after he took over its reins.
The company is, therefore, sparing no efforts in making its cloud computing push a success in India, a market that is "developing very nicely", where "piracy is reducing and intellectual property protection is better than in China". Mr Ballmer also noted that India is among the top five or six countries in terms of talent and market potential.
Microsoft said cloud is important in India as it is a catalyst for IT adoption. "We are successful at exporting IT services and talent. But when it comes to using technology domestically, we are quite poor,'' said Microsoft India chairman Ravi Venkatesan.
There isn't much use of computers and technology in schools, homes, government offices or by the more than 4 million small and medium businesses.

The future of IT will be reduced to three kinds of jobs

By Jason Hiner

 

Takeaway: The IT profession and the IT job market are in the midst of seismic changes that are going to shift the focus to three types of jobs.

There’s a general anxiety that has settled over much of the IT profession in recent years. It’s a stark contrast to the situation just over a decade ago. At the end of the 1990s, IT pros were the belles of the ball. The IT labor shortage regularly made headlines and IT pros were able to command excellent salaries by getting training and certification, job hopping, and, in many cases, being the only qualified candidate for a key position in a thinly-stretched job market. At the time, IT was held up as one of the professions of the future, where more and more of the best jobs would be migrating as computer-automated processes replaced manual ones.
Unfortunately, that idea of the future has disappeared, or at least morphed into something much different.

The glory days when IT pros could name their ticket evaporated when the Y2K crisis passed and then the dot com implosion happened. Suddenly, companies didn’t need as many coders on staff. Suddenly, there were a lot fewer startups buying servers and hiring sysadmins to run them.
Around the same time, there was also a general backlash against IT in corporate America. Many companies had been throwing nearly-endless amounts of money at IT projects in the belief that tech was the answer to all problems. Because IT had driven major productivity improvements during the 1990s, a lot of companies over-invested in IT and tried to take it too far too fast. As a result, there were a lot of very large, very expensive IT projects that crashed and burned.
When the recession of 2001 hit, these massively overbuilt IT departments were huge targets for budget cuts and many of them got hit hard. As the recession dragged out in 2002 and 2003, IT pros mostly told each other that they needed to ride out the storm and that things would bounce back. But, a strange thing happened. IT budgets remained flat year after year. The rebound never happened.
Fast forward to 2011. Most IT departments are a shadow of their former selves. They’ve drastically reduced the number of tech support professionals, or outsourced the help desk entirely. They have a lot fewer administrators running around to manage the network and the servers, or they’ve outsourced much of the data center altogether. These were the jobs that were at the center of the IT pro boom in 1999. Today, they haven’t totally disappeared, but there certainly isn’t a shortage of available workers or a high demand for those skill sets.
That’s because the IT environment has changed dramatically. More and more of traditional software has moved to the web, or at least to internal servers and served through a web browser. Many technophobic Baby Boomers have left the workforce and been replaced by Millennials who not only don’t need as much tech support, but often want to choose their own equipment and view the IT department as an obstacle to productivity. In other words, today’s users don’t need as much help as they used to. Cynical IT pros will argue this until they are blue in the face, but it’s true. Most workers have now been using technology for a decade or more and have become more proficient than they were a decade ago. Plus, the software itself has gotten better. It’s still horribly imperfect, but it’s better.
So where does that leave today’s IT professionals? Where will the IT jobs of the future be?

1. Consultants

Let’s face it, all but the largest enterprises would prefer to not to have any IT professionals on staff, or at least as few as possible. It’s nothing personal against geeks, it’s just that IT pros are expensive and when IT departments get too big and centralized they tend to become experts at saying, “No.” They block more progress than they enable. As a result, we’re going to see most of traditional IT administration and support functions outsourced to third-party consultants. This includes a wide range from huge multi-national consultancies to the one person consultancy who serves as the rented IT department for local SMBs. I’m also lumping in companies like IBM, HP, Amazon AWS, and Rackspace, who will rent out both data center capacity and IT professionals to help deploy, manage, and troubleshoot solutions. Many of the IT administrators and support professionals who currently work directly for corporations will transition to working for big vendors or consultancies in the future as companies switch to purchasing IT services on an as-needed basis in order to lower costs, get a higher level of expertise, and get 24/7/365 coverage.

2. Project managers

Most of the IT workers that survive and remain as employees in traditional companies will be project managers. They will not be part of a centralized IT department, but will be spread out in the various business units and departments. They will be business analysts who will help the company leaders and managers make good technology decisions. They will gather business requirements and communicate with stakeholders about the technology solutions they need, and will also be proactive in looking for new technologies that can transform the business. These project managers will also serve as the company’s point of contact with technology vendors and consultants. If you look closely, you can already see a lot of current IT managers morphing in this direction.

3. Developers

By far, the area where the largest number of IT jobs is going to move is into developer, programmer, and coder jobs. While IT used to be about managing and deploying hardware and software, it’s going to increasingly be about web-based applications that will be expected to work smoothly, be self-evident, and require very little training or intervention from tech support. The other piece of the pie will be mobile applications — both native apps and mobile web apps. As I wrote in my article, We’re entering the decade of the developer, the current changes in IT are “shifting more of the power in the tech industry away from those who deploy and support apps to those who build them.” This trend is already underway and it’s only going to accelerate over the next decade.

From TechRepublic.com

Configuring VLANs for vSphere virtualization

• 
  
• 
  
• 
  
• 
  
• 
  
• 
  
• 
  
• 
  
• 
  

By Rick Vanover

Takeaway: One of the best features of the virtualized infrastructure is the ability to have VLANs presented to guest VMs through a virtual switch. Rick Vanover goes back to basics to show how to bring a VLAN to an ESXi host.

VMware’s vSphere virtualization suite allows for ESXi hosts to support 802.1Q VLAN tagging, which presents multiple networks to the virtualized infrastructure. The basic premise is that a virtual switch (either the standard virtual switch or the new vNetwork Distributed Virtual Switch) will be presented with a tag (a VLAN ID) to the ESXi hosts. The use of VLANs for virtualization functions in a very similar way that the network switches move multiple networks around the datacenter between other switches.

With a VLAN configured, the ESXi host can present multiple networks to all types of communication. For example, the ESXi management interface can have an IP address on one network, while the vmkernel vMotion interface can be configured for a separate IP address on a separate network; both networks can be configured on the same physical cable with the use of VLANs. You would need to configure the physical cable on the physical switch as a VLAN Trunk Protocol (VTP) port; this is the opposite of the standard cabling that may usually go with a server to the physical switch, the access port (AP).

With a VTP in use, vSphere supports up to 512 port groups on a virtual switch. The port groups can be used to configure the management and vMotion interfaces as described above; the groups can also be configured to attach guest virtual machines to the same virtual switch, which can mean putting all of these communication types on the same physical media (cable). Depending on the rules of separation for each virtual environment, this may not be permitted, but it is possible. Figure A shows all of these roles stacked on one interface for a sample ESXi host configuration, using VLANs for each port group.

Figure A

Click the image to enlarge.

As a general rule of thumb, my virtualization practice puts vSphere-centric communication on the same physical media — this includes the ESXi management and vMotion interfaces. Each interface is also a good candidate for its own VLANs. If Ethernet-based storage such as iSCSI or NFS should be used, this would make a good candidate for their own media. While you could stack the ESXi management, vMotion interfaces, and storage interfaces on the same physical media, there may be limitations that impact the storage networking. In my practice, I carve iSCSI and NFS connectivity to their own media, and I may also use VLANs on the connection. The last category is virtual machine guest networking, and I usually keep these separate via their own connections to the switching environment and utilized VLANs.

This primer is an overview on how to use VLANs with vSphere virtualization. For more information about getting into the right configuration for this pillar of infrastructure, read the VMware Virtual Networking Concepts white paper and the What’s New in vSphere Networking white paper.

From -TechRepublic.Com

VMware vSphere Examples and Scenarios Guide

VMware vSphere Examples and Scenarios Guide


http://pubs.vmware.com/vsphere-50/topic/com.vmware.ICbase/PDF/vsphere-esxi-vcenter-server-50-examples-scenarios-guide.pdf


Pl Click on the link to download the pdf.

vSphere Command-Line Interface Concepts and Examples

vSphere Command-Line Interface Concepts and Examples

http://pubs.vmware.com/vsphere-50/topic/com.vmware.ICbase/PDF/vsphere-esxi-vcenter-server-50-command-line-interface-solutions-and-examples-guide.pdf

Click on the above link to download pdf

Pl comment if u like this blog. Pl comment if any improvements required to this blog