Background

For many years, the total amount of storage users needed to support application and file environments just wasn’t a prime IT concern. The amount of raw storage needed to generate a given amount of usable storage was just arithmetic; utilization was known, but it didn’t really drive buying behavior. IT managers just bought more capacity when it was needed because getting the job done was all that mattered. Explosive data growth, the rapid adoption of server virtualization technology, and economic concerns have combined to change that; while getting the job done—effectiveness—is still crucial, doing it as efficiently as possible is now equally critical in most IT organizations. IT managers are looking for ways to be more productive and get more out of key applications with the same—or more likely less—budget.

ESG research indicates that a number of factors are driving IT decision makers towards more efficient storage solutions.  As shown in Figure 1, accelerating data growth, storage system costs, and increasing complexity are cited as significant challenges by IT managers.[1]

In addition to the storage challenges listed in Figure 1, ESG research indicates that reduced operational costs, business process improvement, and reductions in capital expenditures are also top priorities when making purchasing decisions.[2] As a matter of fact, compared to three years ago, 68% of respondents are significantly more aware of energy consumption and cooling requirements than in the past.  Put it all together and it’s clear that IT managers are looking for modular, cost effective storage solutions that are energy efficient and scalable.

EMC Unified Storage

EMC Unified Storage is built upon EMC’s CLARiiON and Celerra storage platforms, designed to address the most crucial cost and management challenges faced by storage and IT managers.  Both the Celerra and the CX4 offer storage efficiency technologies and comprehensive VMware integration.  All of these platforms and technologies are managed from the EMC Unisphere management interface.

EMC has recently announced a program guaranteeing that customers will require 20% less raw unified storage capacity (NAS and/or SAN) with EMC compared to other storage vendors. [3] This is based on “out of the box” best practices for configuration provided by the vendors.  This promise of savings does not take into account the compounded effect of the additional EMC storage efficiency technologies covered in this report.

EMC Unified Storage is designed to drive down consolidation costs and increase efficiency using features and attributes including:

• Fully Automated Storage Tiering (FAST), which enables sub-LUN/sub-volume automated, real-time data migration and placement on the appropriate tier with no intervention by administrators and no interruption to applications.
• FAST Cache technology, which utilizes EFD (enterprise flash drives) capacity as an extended cache pool to provide a performance boost across an entire storage system.
• Block data compression, which allows customers to compress inactive data and reclaim valuable storage capacity. Celerra has offered file level compression for some time with Celerra Data Deduplication.
• Virtual provisioning, which uses just in time capacity allocation and simplified volume management capabilities to reduce the total cost of ownership.
• High-speed, energy-efficient enterprise flash drive technology for applications with demanding performance requirements.
• Low power SATA drive technology, drive spin-down capability, and adaptive cooling to dynamically reduce energy use and improve efficiency.
• For file systems, Celerra also has the ability to tier to secondary archive devices such as Centera, Atmos and Data Domain
• Integration with VMware vCenter for high availability, data protection, and ease of use, including VM-aware storage management, vCenter plugins, Site Recovery manager (SRM) failover and failback, and VAAI (vStorage API for Array Integration). Many of these capabilities are being consolidated into a single universal vCenter plug-in known as EMC VSI (Virtual Storage Integrator).

This report documents ESG Lab hands-on testing of the EMC Unified Storage product portfolio with a focus on its ability to increase performance, availability, and investment protection as it reduces cost, complexity, and management  requirements.

ESG Lab Validation

ESG Lab performed hands-on evaluation and testing of EMC’s Unified Storage solution at multiple EMC facilities.  Testing was designed to demonstrate the simplicity and efficiency of managing and maintaining a mid-tier storage environment.

Simply Unified Storage Management

EMC Unisphere gives storage administrators the ability to manage the complete storage ecosystem with one simple web-based management tool.  Unisphere is a unified management interface for three EMC products:  CLARiiON, Celerra and RecoverPoint.  It has evolved from the CLARiiON Navisphere administrative interface and is compatible with previous releases of CLARiiON FLARE and with the current release of Celerra DART.  It also still supports existing customers’ CLI scripts.  EMC’s stated direction with Unisphere is to continue to integrate more device and information management capability into the product.  Over time, Unisphere will also provide a common communication point for broader IT infrastructure management tools like VMware vCenter.

ESG Lab Testing

ESG Lab tested Unisphere by examining the components contained in the management tool.  Unisphere can be accessed locally or through an array controller.  By setting up a domain containing Celerra and CLARiiON environments, ESG Lab was able to log in with one account and aggregate those multiple environments into a customizable dashboard.  From the dashboard view, ESG lab was able to drill down to any function using the tabs located at the top of the screen or by clicking directly on one of the links presented in the summary views.  An intuitive “Most Free Space” view is highlighted in Figure 3.

ESG Lab examined the storage pool feature that has been updated in the latest release of Unisphere.   In earlier releases, pools were used for thin provisioning of LUNs.  Release 30 introduces Thick LUNs to the pool, which can be contained in the same pool as thin LUNs.  ESG Lab observed that many of the new data services, such as FAST tiering (shown in Figure 4), are available only in pool LUNS.  ESG Lab was able to configure a pool that contained a combination of different disks including flash, Fibre Channel, and SATA, since pools contain no restrictions in terms of the types or location of drives.

Fully Automated Storage Tiering

As information continues to grow exponentially, storage systems need to be intelligent enough to automatically optimize for both performance and cost savings.   EMC FAST (Fully Automated Storage Tiering) and FAST Cache capabilities were designed with these goals in mind.   As shown in Figure 5, applications can have access to a mix of FAST and FAST Cache storage configurations in a networked environment.  FAST operates within a pool LUN, moving sub-LUN slices of data between storage drives according to activity level.  FAST Cache utilizes standard Enterprise Flash Drives (EFD) that can be dynamically added to the storage hardware, and there are no requirements for pool based LUNs.   Applications can automatically take advantage of the performance improvements as data is moved to the cache as needed.

EMC’s approach, using standard Enterprise Flash Drives (EFD), provides a flexible and scalable solution for cache.  Adding EFD’s to an array requires no scheduled outage and can scale to 2 TB mirrored R/W dynamically in a FAST Cache configuration.   Both FAST and FAST Cache utilize the same flash drive technology allowing EFD’s to be easily reconfigured for use with either solution.

Both solutions provide highly complementary performance improvements and are optimized to work together.  FAST Sub-LUN tiering optimizes capacity and lowers costs as colder data can be moved to slower and less expensive drives.  FAST Cache accelerates performance to address unexpected spikes in workloads.  FAST Cache provides greater performance returns since it examines data at a higher granularity (64k chunks) than FAST (1 GB chunks).   Since both FAST and FAST Cache utilize the same flash drive technology, EFD’s can be easily reconfigured for use as either cache, or as Tier 0 storage. EMC’s unified storage systems can support up to 64TB of FLASH drive technology.

ESG Lab Testing

ESG Lab performed several performance tests for both FAST and FAST Cache. The test bed for FAST contained a CLARiiON CX4-960 with a mix of EFD, Fibre Channel and SATA II drives. [4] Testing began on an Oracle test bed with a multi-user online transaction processing (OLTP) system that emulates a warehouse order entry application.  The capacity consumed by the Oracle database was 1.2 TB.

ESG Lab examined the configuration of FAST in a CLARiiON storage pool.  The properties of the tier were easy to configure and allowed ESG Lab to tier manually or according to a schedule.  Data was moved automatically between tiers based on activity level.  FAST moved in sub-LUN portions as a background process running within the CLARiiON.  As shown in Figure 6 , ESG Lab created a policy to move data within a 110 GB LUN between three tiers of storage (Flash, FC and SATA).  Tiering was set to manual to allow ESG Lab to run a baseline performance test first before moving the data.

ESG Lab ran a baseline performance test with only the 45 Fibre Channel disks.  The test was repeated after a FAST pool was created, adding the five flash drives and 15 SATA drives.   Performance in the form of relative transactions per minute improved significantly after data was moved to different drives in the tier, optimizing the faster flash drives for hot data and moving less frequently accessed data to the Fibre Channel and SATA drives.  Figure 7 shows transactions per minute were 108% faster after data was moved using FAST.

ESG Lab performed testing to examine the configuration and performance improvements of FAST Cache when added to Fibre Channel disk drives.   The test bed for FAST cache contained a CLARiiON CX4-960 with 45 15k RPM 600 GB Fibre Channel Drives and just eight 73 GB Flash drives.  The Fibre Channel drives were presented to a test server running Oracle on Windows 2003 SP2.  The database and server configuration were the same as the FAST test.

ESG Lab used Unisphere to configure FAST Cache, as seen in Figure 8. In this example, eight 73 GB flash drives were configured for read/write caching. Once enabled on the drives, FAST Cache operation was completely automatic.  There was no need to adjust settings or tie FAST Cache to any LUNs.

ESG Lab ran a baseline performance test with 45 Fibre Channel disks and repeated the test after enabling FAST Cache.  Performance in the form of relative transactions per minute improved significantly, 143% faster, with FAST Cache enabled, as shown in Figure 9.

Table 1 represents the drive configurations and performance results of both FAST and FAST Cache testing performed by ESG Lab. 

What the Numbers Mean

• The capacity of an all-Fibre-Channel database environment was doubled using five 200 GB EFD drives and 15 2 TB SATA drives as OLTP performance was increased by 108%.
• An Oracle OLTP environment saw performance increase more than 143% using FAST Cache without adding any additional drives.
• Latency, which translates directly into delays experienced by users, reduced by 65% with flash drives using both FAST and FAST Cache.

In addition, ESG Lab audited results from a test performed with SATA drives used in place of Fibre Channel.  The results showed the same performance gains as the Fibre Channel tests.  Since FAST Cache offloads most of the IO to flash drives, it minimizes the reads/writes performed against the physical disks behind cache.  This enables SATA drives to deliver performance numbers comparable to Fibre Channel configurations.

Overall, ESG Lab recorded improvements in both database transactions and response time with FAST and FAST Cache.  Specifically, database transactions increased 108% with FAST and 143% with FAST Cache.  Response times for both FAST and FAST Cache improved by 65%.

As seen in Table 2, an ESG Lab audit of cost of CX4 infrastructure indicates that a 54 TB configuration using EFD, FC, and SATA drives with FAST costs less to acquire than a 54 TB FC only configuration, provides better performance and consumes less power.

Optimizing Virtual Server Environments

CLARiiON and Celerra storage systems support VMware’s vStorage API for Array Integration (VAAI) for vCenter integration. This enables VMware administrators to easily provision and manage virtual machine storage. Improved integration hides the complexity of the underlying storage systems, but gives VMware administrators the ability to perform advanced administrative tasks including cloning volumes, creating pools and compression.

ESG Lab Testing

ESG Lab tested the integration of CLARiiON and Celerra in a vCenter environment by examining the functionality of the plugins for CLARiiON and Celerra in vSphere.  Using the navigation pane in vSphere, ESG Lab was able to right click on a VMware cluster and select EMC Celerra as one of the configuration options. From that selection, ESG Lab could quickly and easily provision storage from an available storage pool for a VMware cluster node as shown in Figure 10.

ESG Lab was also able to navigate to a VMware image and select an option under the EMC Celerra plugin to create a full or fast clone of the image.  During the cloning process, ESG Lab was able to create a storage pool to use for the new image.

ESG Lab also examined the plugin for CLARiiON with vSphere.  The CLARiiON plugin presents a tab available directly from the vSphere home page.  ESG Lab was able to log in to a vSphere environment and, from the CLARiiON page, provision new datastores and delete existing datastores.

In addition, ESG Lab looked at EMC’s new unified vCenter plugin which is planned for general availability when the next version of VMware vSphere is released.   Managing a storage environment was more intuitive as all the tasks are included as a tabbed page, much like the Unisphere interface.  ESG Lab was able navigate to a virtual machine and see the storage configuration for that image.  Figure 11 shows the EMC Storage tab available from the VMware image configuration page.

ESG Lab also examined VMware-aware Unisphere, which allows a storage administrator to look into a virtual infrastructure.  Unisphere talks directly to ESX APIs and provides end to end mapping from ESX, to LUN, to virtual machine mappings.  ESG Lab was able to monitor the state of storage requirements, including alert messages from ESX when storage is full.

Storage Efficient Compression

EMC’s block data compression feature allows administrators to compress inactive data in live volumes and reclaim storage capacity. This builds on the capacity reduction provided by Virtual Provisioning (a.k.a., thin provisioning).  Virtual Provisioning frees up allocated but unused LUN capacity that can be returned to the common storage pool. Compression provides additional benefits as it recognizes repeating patterns in allocated and used application data.  Compression is supported on both full and thin provisioned LUNs.  If the LUN is fully provisioned and part of a RAID group, the LUN is migrated to a storage pool at the same time it is compressed and the free space is returned to the RAID group.  Fully provisioned LUNs that are compressed are automatically converted to thin provisioned LUNs.

ESG Lab Testing

ESG Lab tested the results of compression on a CLARiiON by selecting a 100 GB LUN with 75 GB of mixed data types in an NTFS file system.   The data set was derived from production EMC IT department’s data and included a mix of media, Office, binary, and text files.  ESG Lab configured compression by navigating to the compression tab under the LUN properties page.  The compression properties were simple to configure and included the option to specify the rate of compression depending on the requirements for availability during the compression process.  ESG Lab chose to use high compression for this test.

Compression of a 100 GB LUN completed in 1 hour and 40 minutes.  Examining the results of the compression, as shown in Figure 14, ESG Lab observed that the LUN capacity was reduced from 100 GB to 54 GB, a savings in space of almost 50%. As expected, after compression, the LUN had been converted from a thick to a thin LUN.

ESG Lab Validation Highlights

• ESG Lab confirmed that Unisphere provides a simple and intuitive management interface for multiple EMC storage systems.
• vCenter plugins for CLARiiON, Celerra and RecoverPoint were used to perform routine storage management tasks from a VMware management console.  A unified plug-in, with planned availability in the next version of VMware vSphere, provided an intuitive interface for centralized storage management.
• With simple “set it and forget it” compression capability, ESG Lab observed a 46% reduction in CLARiiON disk capacity for common office files collected from EMC’s production IT environment.
• A 108% performance increase was recorded using FAST Sub-LUN tiering.  These results were consistent with both Fibre Channel and SATA drives.
• A 143% performance improvement was recorded with FAST Cache as compared to traditional Fibre Channel storage.

Issues to Consider

• The current VMware vCenter plugins for Celerra, CLARiiON and RecoverPoint have difference in capabilities and a different look and feel. The unified plugin that ESG Lab tested (planned for generally availability when the next version of VMware VSphere is released) has an intuitive and consistent look and feel.
• While the energy efficiency of the CLARiiON product line has been dramatically improved, full policy-based drive spin-down could be used to magnify the savings (currently, drives spin down after 30  minutes of disk inactivity)—particularly for large near-line archives that are infrequently accessed.  While this technology is currently available in the CLARiiON CX4 and EMC Disk Library product, it is not yet supported within the Celerra line.  EMC has advised ESG Lab that policy-based drive spin-down is planned for a future code release.
• While the management of EMC’s block and file storage tested by ESG Lab for this report is unified, the underlying hardware and software architectures are still separate and distinct. To be fair, EMC has made tremendous strides toward truly unified storage architecture, based on discussion with EMC, ESG Labs believes unification of midrange storage is on the roadmap.

The Bigger Truth

IT professionals are being tasked with justifying storage strategy even as senior managers struggle with budgets hamstrung by financial crisis.  How can organizations keep pace with capacity growth of 50% or more annually while staying within budget?   How will storage investments be protected—now and in the future?   How can more capacity be managed with better performance and service levels with existing staff?  Will storage investments complement—or complicate—virtual server consolidation initiatives? How will IT create a winning strategy that works for both the team and the organization?

ESG is not only impressed with EMC’s ability to continually increase the capacity, performance, and capabilities of its full product line, but also by its focus on continuously improving manageability.  In 2008, ESG Lab examined the fourth generation CLARiiON CX4 and confirmed that EMC stuck with its game plan of continuously simplifying routine management tasks, even as a number of innovative technologies were introduced into the CX4 product line which lowered the cost of ownership for CLARiiON customers.

EMC Unified Storage builds on EMC’s heritage of providing a rich set of comprehensive availability and protection options, adding advanced storage efficiency and integrated, consistent management for both SAN and NAS storage.

ESG Lab was able to manage storage area network (SAN), network-attached storage (NAS), and storage efficiency technologies simply and intuitively using the EMC Unisphere storage management interface. VM-aware storage management, vCenter plugins, and VAAI provided comprehensive storage management from within the vSphere management environment.

While ESG lab confirmed that EMC has made significant advances with the unified management of CLARiiON, Celera and RecoverPoint EMC still offers multiple separate architectures for block, NAS, and object storage. EMC has been listening to their customers and has been making significant progress toward a comprehensive, unified architecture starting at the component level, using the same disks, shelves, and controllers in the CLARiiON, Celerra, VMAX and VPLEX product lines. While there is still work to be done to get the rest of the way there, EMC has taken great steps in the right direction.

EMC Unified Storage includes a number of innovative technologies which lower the cost of ownership for EMC customers. ESG Lab used block data compression to compress inactive data and reclaim valuable storage capacity.  FAST Cache leverages enterprise flash drives to avoid the wasted cost of over-provisioned disk capacity for applications with extreme performance requirements. ESG Lab hands-on testing has confirmed that FAST sub-LUN automated tiering can use a combination of EFD, FC, and SATA drives to provide exactly the same capacity as a large pool of Fibre Channel drives at a lower cost of acquisition and with better performance.

With EMC Unified Storage, EMC has dramatically enhanced simplicity, performance, and storage efficiency for consolidated, virtualized environments. While the speeds and feeds are impressive, ESG Lab is most impressed by the continuous improvements in manageability and the long list of valuable new capabilities that have been built into the offering.  From the speed and automated operation of FAST to the dramatic manageability improvements of Unisphere and VM-aware technology, the breadth and depth of the features built into EMC Unified Storage can be used to meet the precise needs of any organization. If your organization is struggling to keep up with data growth while providing ever higher levels of performance and availability with stagnant or shrinking budgets, ESG Lab recommends that you consider EMC Unified Storage as the foundation for your own winning storage strategy.

Appendix

[2] Source: ESG Research Report, 2010 IT Spending Intentions Survey, January 2010.

[3] See EMC 20% guarantee and efficiency calculator, http://www.emc.com/products/unified-storage-guarantee/index.htm?Pid=prod_tech_unified-unifiedguarantee-070610.

[4] Configuration details can be found in the Appendix.

[5] Published list pricing data used to calculate savings can be found in the Appendix

[6] Source: ESG Research report, The Impact of Server Virtualization on Storage, December 2007.

Background

The management of file-based, or “unstructured,” content (i.e., multimedia files, Web pages, office productivity documents, etc.) has become one of the most pressing and persistent challenges facing today’s IT organizations; IT managers must store, deliver, and manage large volumes of unstructured data while meeting increasingly demanding service levels. ESG research indicates that the majority of end-users currently using or considering scale-out NAS solutions are most concerned with improving management efficiency, scalability, performance, and availability.[1]

Fujitsu Scalable File Server Solution or UDS File Services

Fujitsu Scalable File Server Solution (UDS in North America) File Services offerings, shown in Figure 2, offer a broadly scalable NAS solution that can address the unique challenges of scaling out NAS services for businesses of all sizes.

Fujitsu brings a number of economic benefits associated with scale-out NAS platforms to the table with its Symantec FileStore based solution. Scale-out NAS architectures have a number of cost advantages over scale-up solutions, ranging from start up costs to managing technology refreshes—and many steps in between. Scale-out NAS carries a lower overall cost compared to scale-up systems for a number of reasons:

• Ability to scale capacity without scaling headcount. With Fujitsu Scalable File Server Solution, it is just as easy to manage a clustered storage system with sixteen nodes as it is to manage one with two nodes. Scale-out file storage systems enable this through clustering and a global namespace, which provides a single point of management for massive amounts of file data.
• Low entry cost. The entry cost for scale-out systems varies depending on the minimum configurations supported. Fujitsu Scalable File Server Solution starts as small as two nodes and scales up from there. With clustered scale-out systems, you can add resources and scale as needed, online.
• Just-in-time scalability. Because of the modular nature of scale-out systems, there is no need to buy (and power or cool) frames, power supplies, and mostly empty cabinets in advance of storage capacity needs.
• Higher utilization rates. Because all of the NAS heads in FileStore clusters can address the entire pool of usable capacity, no capacity is locked away behind underutilized NAS heads—a common problem in scale-up systems. It is not unusual to see utilization rates of 30% or less in scale-up systems and 60% or more in scale-out systems.
• Reduced change management planning cycles. The modularity and scalability of scale-out NAS allow for extremely fast provisioning. Fujitsu Scalable File Server Solution is plug-and-play; add a storage or processor node and the system self-discovers and expands the file system or incorporates it into load balancing algorithms on the fly. There is typically no disruption of service, nor is there a requirement to plan data layouts, create LUNs, or perform data migration.
• Non-disruptive technology refresh. The process of managing technology refreshes with Fujitsu Scalable File Server Solution is faster and easier than with monolithic NAS because the cluster maps logical mount points to physical mount points in a virtualized manner, allowing back-end technology changes to be made with little or no disruption to client access.

Fujitsu Scalable File Server Solution offers advanced features still fairly new to scale-out NAS, with built-in support for storage tiering and solid state disk. Dynamic Storage Tiering allows for policy-based movement of files across storage tiers. Based on activity or desired performance levels, data can automatically be demoted to lower performing, bulk-storage tiers, including MAID, or promoted to primary tiers if it suddenly becomes very active. Consider the benefits this capability brings to environments with tiers that include solid state disk (SSD). For example, a Fujitsu Scalable File Server Solution customer can initially choose to store data in a secondary tier (SATA or Nearline SAS) and then promote it to a primary tier based on IO access. Dynamic Storage Tiering can then push the data back to the secondary tier based on IO inactivity, eliminating the need to peg data in valuable SSD real estate and then manually monitor and move it. All of this storage tiering and data movement occurs completely in the background, transparently to the end-user. The locations of the files, and even their node numbers, do not change.

This report examines Fujitsu’s Scalable File Server Solution with a goal of evaluating ease of administration, performance, and scalability as well as advanced enterprise-class features such as file system mirroring, replication, and Dynamic Storage Tiering.

ESG Lab Validation

ESG Lab performed hands-on evaluation and testing of the Fujitsu Scalable File Server Solution at Fujitsu’s Sunnyvale Campus. Testing was designed to evaluate manageability, performance, scalability, and availability. The test bed, shown in Figure 3, was comprised of the Scalable File Server Solution’s entry-level NAS storage system with two cluster nodes. Windows and Linux workstations were utilized as CIFS and NFS clients.

Ease of Deployment and Flexible Management

ESG Lab first examined the management interface of Fujitsu’s Scalable File Server solution, evaluating the ease of use and functionality of creating storage pools, file systems, and shares for CIFS, NFS, and FTP clients. Fujitsu utilizes the browser-based Symantec FileStore management tool to provide a full featured interface for all file service and storage functions. The Fujitsu Scalable File Server Solution ships with a storage array and server cluster preconfigured, so administrators can create shares and start providing file services immediately.

ESG Lab Testing

The first step in the setup of the Fujitsu Scalable File Server Solution platform was to configure TCP/IP networking using a CLI connection directly attached to the system console. ESG Lab configured the network IP address for each physical interface along with the virtual IP addresses associated with the physical IP. A console IP address was also assigned at this point. Figure 4 shows the TCP/IP configuration wizard in the CLI.

Once the IP addresses were assigned, all management was performed using the browser-based management tool. A Web browser was pointed to the same IP address that was used for the console CLI. A local administrator username and password were entered. The administrator can be a local user account or one contained in an Active Directory domain or LDAP source.

The home page of the management interface, seen in Figure 5, shows the status of the Fujitsu Scalable File Server Solution. All NAS and file system functions are managed from this page.

From the File System screen, ESG Lab selected the option to create a new file system. The selection started a two-step wizard, shown in Figure 6 that allowed ESG Lab to configure the size, layout, and storage pool location for the file system. ESG Lab created a 15 GB file system named “ESG1” with a striped layout in pool DG0001.

Next, ESG Lab clicked the shares tab in the System Status window and selected the option to add a CIFS share. As seen in Figure 7, one dialog box was displayed where the file system to share was selected, and the name and properties of the share were set.

ESG Lab tested access by mapping the network share to a network drive from a Windows client; ESG1 was mapped to the Y: drive. Figure 8 shows files and folders successfully copied to the share.

This test was repeated using the same file system, ESG1, to create an NFS share. As before, the shares tab in the system status screen was used to access the add share wizard.  Once the NFS share was exported, the share was mounted by a Linux client and files were copied into the NFS share successfully. The entire process, creating a file system and sharing it out to both NFS and Windows clients, took less than five minutes.

Next, ESG Lab used a snapshot to simulate the recovery of an accidentally deleted file. Single, manual snapshots can be created or a schedule can be set to take recurring snapshots of a file system on a periodic basis. ESG Lab created a snapshot schedule as seen in Figure 9.

Once created, the snapshot was made visible to Windows clients using the same add share wizard that was used to create CIFS shares, seen earlier in Figure 7.

Connectivity to the snapshot was tested using a Windows client. First, a file was deleted from the ESG1 share and then ESG Lab copied the file from the snapshot and back to the ESG1 share.

Performance and Scalability

One of the key benefits of Fujitsu’s Scalable File Server Solution is its ability to provide a high-level of aggregate capacity and performance that scales in near-linear fashion as nodes and storage are added to the cluster. With a maximum file system size of 256 TB and total addressable storage space of 2 PB, Fujitsu’s Scalable File Server Solution can scale to meet the needs of applications that require high performance, high capacity, or a combination of both. As additional cluster nodes are added, performance increases with the additional processing power, memory, and bandwidth of each node. As additional disks and disk arrays are added, the capacity—and performance—of the solution increases as well.

Fujitsu Scalable File Server Solution also provides advanced features that enhance the scalability of the system. Not only can file systems be expanded non-disruptively, they can also be shrunk to reclaim space for other purposes.

ESG Lab Testing

ESG Lab tested both expanding and shrinking a live file system mounted by an active client. These functions were accessed from the File Systems tab in the System Status window. ESG Lab first expanded the 15 GB ESG1 file system by 10 GB, to 25 GB total, as shown in Figure 11. After expanding the file system, files were copied into that file system to validate connectivity. The client was still connected to the share and was able to copy files with no issues.

ESG Lab then used the same wizard to shrink the file system down to 20 GB. Connectivity was tested by again copying files into the now smaller file system. The size was verified by looking at the properties of the associated share from the Windows client and confirmed: the new file system was 20 GB, adjusted with no disruption to connectivity or remounting of the share.

ESG Lab next tested the ability to grow the cluster non-disruptively.  Starting with a three node cluster, a scripted copy operation was started from a client to generate a continuous load as a fourth node was added. First, the physical IP address for a pre-wired and staged new node was added to the cluster configuration and a group of virtual IP addresses were assigned to the new node as shown previously in Figure 4.  Once the IP addresses were assigned, the node was rebooted and F12 was pressed to initiate a network boot. FileStore software was then automatically installed on the new node. Once the installation was complete, the new node appeared on the Cluster page of the FileStore management GUI as an available node, as shown in Figure 13.

At this point, adding the node to the cluster was accomplished by selecting the IP address and clicking “Add Node.”  Within a few seconds, the node was added to the cluster. While IO paused momentarily, as seen in Figure 14, it resumed with no loss of connectivity and no remount required.

SPECsfs2008 Results

ESG Lab audited published results of the SPECsfs2008 industry standard benchmark suite maintained by the Standard Performance Evaluation Corporation (SPEC). SPECsfs2008 testing measures file server throughput and response time, providing a standardized method for comparing file server performance across disparate vendor platforms. SPECsfs2008 results summarize the server’s capabilities in the context of both the number of file operations that can be handled per second, in addition to the overall latency of the file operations. While Fujitsu has not published results for a Fujitsu Scalable File Server Solution configuration as of this writing, there is a result for the 12-node Symantec FileStore Cluster ESG evaluated for this report.

SPECsfs2008 results are audited by the Standard Performance Evaluation Corporation and peer reviewed to ensure consistency. Full configuration data for each SPEC benchmark result are publicly available for download and review.[2] While this can be useful for comparison between vendors, it is important to note that not all vendors participate and publish results.

As seen in Figure 15, the 12-node cluster provided an excellent result of 176,728 SPECsfs2008 requests per second with an average response time of 4.1 milliseconds. Response time is an extremely important component of SPEC results as this is the delay that an application will experience (and pass on to users) when a storage system is stressed to its limits. A system servicing 176,728 SPECsfs2008 IO requests per second with a response time of 4.1 ms is exceptional.  In fact, FileStore has posted one of the highest SPECsfs2008 NFS result as of this writing.

Availability and System Protection

Central to the functionality of Fujitsu’s Scalable File Server Solution is high availability. The system is designed to ensure that the file system remains functional at all times—even in the unlikely event of a software or hardware failure. Clustering technologies are used to eliminate unscheduled downtime and protect data from:

• Server failures (e.g., failed processor or power supply).
• Storage failures (e.g., failed drive or controller).
• Network failures (e.g., failed network cable or NIC).

In addition, the ability to mirror the file system provides additional redundancy to reduce the need to recover the file system if a storage array is taken offline. Similarly, file system replication provides the capacity to copy entire file systems to an off-site location over a wide area network to protect against site outages. Figure 16 shows an overview of data availability options available with Fujitsu Scalable File Server Solution. In addition, anti-virus software is built into the cluster and can be activated with a license key with no installation required. Policies for anti-virus scanning can be set up individually per share. These policies can be set to scan shares on a schedule or set to auto protect mode which scans on file reads.

ESG Lab Testing

ESG Lab validated the failover capabilities of a Fujitsu Scalable File Server Solution by injecting faults into a two-node cluster. The first scenario ESG Lab tested was a network failure. ESG Lab used a client connected to the IP address on port eth1 on Node2 to continuously copy files from one file system to another. A network failure was simulated by removing the Ethernet cable from port eth1 on Node2 while the file copy was in progress. The virtual IP addresses assigned to port eth1 were moved to other available physical addresses in the cluster and the copy traffic moved to port eth2 transparently, with no administrator intervention, after only a brief pause. Figure 17 shows IO activity using Windows Task Manager on the client.

The second test simulated a storage failure by running a firmware upgrade on one controller in the ETERNUS DX80 array. ESG Lab used the same client running the same script to continuously copy files while the controller firmware was upgraded. As before, a brief pause in IO was observed while the ETERNUS DX80 failed all disk volumes over in the array. No loss of connectivity to the file system was experienced by the client during failover of the controller.

Finally, ESG Lab simulated a cluster node failure. The test was executed on a four-node cluster configuration. A client accessing two shares on Node4 was used to continuously copy files from one file system to another. ESG Lab then deleted Node4 from the cluster, at which time the virtual IP addresses originally assigned to Node4 were moved to physical IP addresses on the remaining three nodes. While the client experienced another brief pause in IO, the copy operation continued with no action required by the client or the administrator.

Fujitsu Scalable File Server Solution also allows administrators to add mirrors of a file system; this is in addition to the disk level protection in the ETERNUS array. File System mirrors can be used to enhance availability or facilitate migration of a file system to new back end storage non-disruptively. ESG Lab added a mirror of the ESG1 file system using a one step wizard. The wizard allowed the selection of disks for the mirror from all available storage. After clicking “OK,” the cluster began mirroring the file system to the new disks. Figure 19 shows the File System Details page after mirroring completed, confirming that there were now two mirrored copies of the file system.

Finally, ESG Lab tested the functionality of replication, including the ability to copy only changed blocks rather than whole files. A replication job was set up to replicate the file system cifs0, owned by a two-node cluster, to a file system on a four-node cluster. A total of 25 200 MB files were copied to cifs0 and a replication event was triggered. When the replication event was complete, the target file system, cif0t, was examined and the files were all transferred successfully, for a total of 5 GB of data moved. Next, ESG Lab renamed all 25 files in cifs0 and triggered a second replication event.

System logs showed minimal replication traffic to cif0t and the files at the target site, seen in Figure 20, reflected the new names less than a minute after triggering the replication, confirming that replication only transferred the changed metadata.

ESG Lab also tested the anti-virus feature built into Fujitsu File Services using two files with well known virus signatures. The files were copied into a share called virustest. ESG Lab then started a scan of the virustest share and the two infected files were discovered and moved to a quarantine area, shown in Figure 21.

The second anti-virus test ESG Lab performed reviewed autoprotect mode, where files are scanned on demand when read by clients. ESG Lab copied the same two infected files into the virustest share. When an attempt was made to read the files with a client, the files were automatically scanned and quarantined and access to the files was denied.

Dynamic Storage Tiering

Fujitsu Scalable File Server Solution can use multiple tiers of storage to meet a variety of application performance needs. SSD, Fibre Channel, or SAS drives are supported for performance-sensitive applications and workflows including high performance computing and rich media editing and delivery. Affordably dense Nearline SAS or SATA drives are supported for capacity-intensive applications including deep archives, backup to disk, and large scale consolidation of general purpose legacy file systems.

Fujitsu Scalable File Server Solution provides Dynamic Storage Tiering to provide automatic, hands-off movement of files between tiers based on access patterns. Policies can be set to determine how frequently the system checks for file access and what parameters are used to trigger movement between tiers.

ESG Lab Testing

ESG Lab tested the functionality of Dynamic Storage Tiering on a Fujitsu Scalable File Server Solution cluster that contained a primary tier of high speed SAS disks and a secondary tier of low speed Nearline SAS  drives.  A 100 GB file system, hscifs0, was set up with both tiers and access policies were set. The primary tier was seeded with files totaling 11 GB and the secondary tier was seeded with files totaling 54 GB. Figure 23 shows the utilization of each tier in the File System Details page of the management GUI.

ESG lab then modified 4 GB of data files residing in the secondary tier. After the scheduled scan, the changed data was moved from the secondary tier to the primary tier according to the configured policy. The two tiers were again examined in the File System Details page and primary utilization had increased to 15 GB while secondary utilization decreased to 50 GB, confirming that the 4 GB of data files that were modified had been automatically moved to the higher tier.

ESG Lab Validation Highlights

• Fujitsu Scalable File Server Solution was configured and serving files to Windows and Linux clients less than five minutes after sitting down at the console.
• The Web-based administration console can be accessed from any browser. No separate installation of an administration program is required.
• Snapshots were easy to configure and use to recover files.
• Growing the cluster by adding a new node was a straightforward, non-disruptive process as was growing and shrinking file systems.
• Audited FileStore performance was impressive, posting impressively high throughput with a very low overall response time.
• Advanced data protection and recovery capabilities typically associated with enterprise-class NAS appliances were tested. Local snapshots were used to recover files after common errors and the system stayed online through multiple simulated hardware errors, such as network outages and storage controller and power failures.
• Dynamic Storage Tiering provided automatic, hands-off movement of files between tiers based on access patterns.

Issues to Consider

• While Snapshots were easy to manage and use, administrators must manually share snapshots to allow users to recover files.  A feature to automatically share snapshots with pre-defined naming conventions and integrate with Microsoft’s Volume Shadow Copy Service (VSS) would be a useful enhancement.
• When prompting the administrator to select disks for mirroring a file system, the FileStore GUI did not exclude disks already in use by the source file system.  While the system would not create the mirror on disks already in use by the source, their appearance in the selection screen could be confusing to administrators.
• The version of SAMBA currently in FileStore can only share out a file system from one IP address, which can limit the total performance available to any single SMB share. This issue is corrected in the latest version of SAMBA, which is being integrated into FileStore as of this writing.

The Bigger Truth

The massive growth of file data flooding data centers today—and the wave that will be generated as more and more cloud storage and rich media applications come online—can easily overwhelm traditional scale-up NAS solutions. A scale-out NAS solution from Fujitsu offers scale beyond that which can be attained with traditional NAS solutions: users can start small with a two node system and affordable back-end storage and grow to a massively parallel system with Enterprise class ETERNUS arrays. The performance ceiling is raised by adding more processors and capacity is increased by adding more storage, enabling “just-in-time” scalability. And management is simple because the Fujitsu Scalable File Server Solution scale-out NAS solution is managed as a single entity—no matter how large it gets.

Legacy scale-up NAS solutions face a number of challenges. First, scale-up systems typically have capacity limits in the range of tens of terabytes, with individual file system limits between 2 and 16 TB. As capacity is scaled and limits are hit, more discrete systems are needed—and those systems need to be managed. Second, scale-up systems have fixed performance ratios; there is a fixed number of NAS heads that can be included in a single file system, typically one or two. Third, scale-up NAS has a relatively expensive price/performance ratio compared to scale-out.

A scale-out NAS solution from Fujitsu combines the field-proven performance and scalability of Symantec FileStore software, Fujitsu PRIMERGY industry standard servers, and Fujitsu ETERNUS DX disk storage system to cost-effectively address both scale-up and scale-out NAS challenges. What’s more, it’s surprisingly simple to deploy. ESG Lab was accessing files less than five minutes after getting started. It was also easy to manage via an intuitive graphical user interface.

Additionally, it supports enterprise-class NAS features that are often missing in scale-out NAS solutions; ESG Lab tested snapshots, file system mirroring, remote replication, and automated online migration between different tiers of storage. Last, but not least, it is fault tolerant and fast.

Three things in life are guaranteed: death, taxes, and information growth. Information growth will continue even in a down economy. This deluge of file-based information must be dealt with. Scale-out storage is the wave of the future—it is a path for IT managers to meet their number one storage challenge: keeping pace with overall data growth.

Because of these considerations, more and more enterprises are taking a serious look at scale-out NAS solutions like Fujitsu’s Scalable File Server Solution—clustered scale-out solutions are going mainstream. But commercial enterprises are not just interested in the increased bandwidth scale-out solutions bring to the table: users are expanding use cases for scale-out NAS thanks to the higher scalability and manageability of these systems. In short, scale-out makes economic sense.

Using an appliance-based approach that is fault tolerant and centrally managed, Fujitsu has harnessed the field proven power of ETERNUS and PRIMERGY hardware to Symantec’s FileStore software to create an enterprise-class scale-out NAS solution that is extremely scalable, extremely fast for a wide variety of applications, and extremely easy to deploy and manage.

Appendix

[2] http://www.spec.org/sfs2008/results/

Introduction

Organizations of all sizes are struggling to meet the conflicting challenges associated with information storage growth and complexity juxtaposed with global financial uncertainty. A growing number of IT managers are turning to virtualization and consolidation technologies to meet these challenges.

Background

ESG research indicates that a number of factors are driving IT decision makers toward more cost efficient storage solutions.  As shown in Figure 1, accelerating data growth, storage system costs, and increasing complexity are cited as significant challenges by IT managers.[1]

In addition to the storage challenges listed in Figure 1, ESG research indicates that reduced operational costs and reductions in capital expenditures are also top priorities when making purchasing decisions.[2] Put it all together and it’s clear that IT managers are looking for modular, cost effective storage solutions that are both efficient and scalable.

Coraid EtherDrive SAN

Coraid EtherDrive products combine commodity hardware, lightweight Ethernet networking, and a scale-out virtual storage architecture that can grow from a single appliance to multi-petabyte installations. As seen in Figure 2, Coraid provides both cost/capacity optimized and performance optimized storage appliances supporting SATA, SAS, and SSD drives. Coraid systems support all standard RAID types including RAID 0, 1, 5, 6, and 10.

To make the offering as turnkey and simple to deploy as possible, Coraid also offers HBAs, servers, and replication appliances. All Coraid products communicate using the lightweight AoE (ATA over Ethernet) protocol and standard Ethernet switches, which provides secure storage networking for industry standard x86 servers.

Coraid EtherDrive SAN promises an impressive list of capabilities, including:

• Price-performance: Higher performance than comparable Fibre Channel configurations, at approximately 20% of the cost.
• Massive throughput: More than 1200 MB/sec of throughput per Coraid EtherDrive SRX-Series storage array shelf for large-block sequential workloads.
• Simple scalability: Ease of implementation and management of Coraid EtherDrive storage compared to Fibre Channel and iSCSI.
• Optimized for virtualization: VMware and Hyper-v see Coraid storage as local-attached disks, with no need for switch configuration or multi-pathing software.

ESG Lab’s testing was designed to explore Coraid’s EtherDrive SAN and the AoE protocol, paying special attention to ease of use and management, capacity and performance scalability, and integration and operation in virtualized environments.

ESG Lab Validation

ESG Lab performed hands-on evaluation and testing of Coraid’s EtherDrive SAN at Coraid’s Redwood Shores, CA headquarters. Testing was designed to demonstrate the ease of installing and configuring an EtherDrive SAN as well as the cost-effective performance and capacity scalability of the platform.

Background: Ethernet SAN

Coraid’s EtherDrive SAN utilizes the AoE protocol to present disk storage to servers across a standard Ethernet network. AoE is an extremely simple method for sharing disk drives through a network. The communication that would normally take place between a motherboard and an IDE disk drive is arranged into data packets and sent across the Ethernet.  As can be seen in Figure 3, AoE is a simpler and more direct protocol than either iSCSI or Fibre Channel. AoE is not built on IP, TCP, or SCSI; packets are addressed to devices using their Ethernet MAC addresses and sent across the network with a minimum of overhead.

Fibre Channel and iSCSI are both based on SCSI, which is a complex protocol designed for a variety of devices (scanners, printers, etc.), in addition to disk drives. Because of this, they incur significant overhead when processing each packet. Both Fibre Channel and iSCSI run SCSI over high level networking protocols on top of a physical network infrastructure, consuming additional overhead and processing compared to AoE, which connects servers and storage directly across the physical Ethernet layer. The typical AoE packet contains just 48 bytes, plus the data payload, enabling “bare metal” performance and native Layer 2 multi-pathing. Fibre Channel and iSCSI first encapsulate the data in the SCSI command set and then wrap SCSI in a transport protocol.

Because they do not run over high level networking protocols like IP, AoE packets (like Fibre Channel) are non-routable. While they can travel across the switches that make up an Ethernet LAN, routers cannot send them to another network and devices outside of the AOE devices local network cannot communicate with them.  This makes AoE packets intrinsically secure. Coraid enables remote access to EtherDrive SANs for administration via AoE tunneling, which is similar to VPN access to a corporate network over the internet.

Getting Started

ESG Lab testing was conducted on a pre-wired, rack-mounted environment consisting of multiple SR2421 and SRX3500 EtherDrive SAN disk shelves. The ESG Lab test bed, as presented in Figure 4, consisted of multiple industry-standard x86 servers with both 20Gbps Coraid HBAs and 1Gbps Ethernet NICs installed. Servers were running VMware ESX server with Red Hat Linux and Windows 2008 installed as guest operating systems as well as physical Linux and Windows 2008 installations. An industry standard Ethernet switch was used for SAN connectivity.[3]

ESG Lab Testing

ESG Lab testing began by powering on an SRX3500 EtherDrive SAN shelf, then logging into a Linux server. Coraid’s cec utility was used to scan for the new chassis using the AoE protocol. In less than a minute, the shelf was visible.

The next step was to name the shelf to make it easier to identify it in a large deployment. Shelf 3 was chosen as the name for these tests. Next, using just three commands, RAID groups were created (Coraid automatically creates one LUN per RAID group), hot spares were assigned, and the LUNs were brought online, as seen in Figure 5.

LUN masking, the means by which servers are given exclusive access to volumes in a SAN environment, is done by Ethernet MAC address using the “mask” command. ESG Lab did not use LUN masking in these tests.

On the Linux server, ls /dev/etherd showed all AoE devices on the network. The storage administrator has nothing else to do—no iSCSI mount, no NFS mount.  The AoE LUNs look like local storage. Next, ESG Lab used mkfs to create and format a file system on each of the AoE LUNs.

Creating LUNs and presenting them for use on the network took less than one minute, while creating the file systems for use by the server took about another minute. In less than two minutes and just four simple commands, ESG Lab configured, provisioned, and was using Coraid EtherDrive storage.

Disruptive Price-Performance

Coraid EtherDrive SAN storage is a modular disk storage system providing massive scale-out capacity and performance with granular, just-in-time scalability to industry standard, open systems environments. The Coraid solution scales by simply installing additional disks and shelves, allowing organizations to start small and scale capacity to petabytes. Using 2 TB SATA drives, users can scale to a petabyte of capacity and 100 GB/sec of raw storage bandwidth in just two racks.

Performance in a storage environment is best measured with the metrics used by the applications organizations actually run.  For an e-mail application, that measurement is the number of users or mailboxes a given system can support.  For a streaming media application, the number of objects served concurrently that can be sustained during peak periods of activity is the measurement that matters most.

ESG Lab Testing

Performance was tested using the IOMETER workload generator via simulated application workloads based on Microsoft Exchange and streaming media services.  Tests were performed to verify a Coraid platform’s ability to deliver predictably scalable performance in a clustered scale-out environment over a standard Ethernet network. The Exchange workload is random in nature and very disk intensive.

Microsoft guidelines recommend a maximum of 1,000 Exchange users per core and less for a server performing multiple roles. This means that a quad-core server, doing nothing but Exchange, should support about 4,000 users.

Microsoft’s IOPS per mailbox guidance for Exchange 2007 is calculated based on the number of messages per mailbox, the user memory profile, in what Outlook mode the mailboxes are operating, and whether any third party mobile devices are used. The baseline value provided by Microsoft is .32 IOPS per mailbox.[4] This means that a quad core Exchange server with 4,000 exchange users will, on average, drive 1,280 IOPS to the Exchange Datastore.  As can be seen in Figure 6, a single SRX3500 LUN was able to support enough transactional IO to support more than 4,500 Exchange users using just 12 SAS drives and scaled linearly to just over 9,000 users with 24 SAS drives.

Next, streaming media performance was examined. This type of traffic is sequential in nature and uses larger block sizes than transactional workloads, putting more of a load on the storage network.

As Figure 7 shows, streaming media performance was excellent, delivering 826 MB/sec from just 6 SSD drives and more than 1,200 MB/sec from 24 SATA drives. Put into perspective, a single shelf was able to drive enough bandwidth to saturate a 10Gbps interface.

The maximum throughput recorded (1200+ MB/sec) was used to calculate the number of streams that could be delivered for a couple of well-known content types including standard definition and high definition broadcast video.  Bit stream rates of 3.75 Mbps for standard definition broadcast video and 80 Mbps for high definition video were used to determine that a single SRX3500 has the bandwidth required to simultaneously stream 120 high definition broadcast videos or 2,560 standard definition broadcast videos as shown in Figure 8.

What the Numbers Mean

• The system showed excellent disk response times for both random and sequential IO. The simulated Exchange disk IO response time was 20ms, while streaming media requests from SATA disk were satisfied in just 1ms.
• Microsoft stresses that, to ensure a positive user experience, the Exchange database LUN requires read and write response times of 20 milliseconds or less so that Exchange can service users’ client software quickly and efficiently. In this context, the SRX 3500’s performance is right on target.
• A single SRX3500 has the raw bandwidth required to service 2,560 concurrent standard definition, broadcast-quality video streams.

Next, ESG lab examined cost of acquisition for a petabyte of storage and SAN connectivity for various technologies. Each storage technology was configured to support the same class and quantities of storage, and SAN connectivity was calculated to support 200 physical servers with redundant connections. Table 2 summarizes the configuration built for each technology.

The cost of storage and SAN connectivity hardware was obtained from a combination of publically available sources, including reseller websites, GSA pricing schedules, and online pricing available directly from vendors.

The cost was calculated for modular dual controller Fibre Channel SAN arrays from three major vendors. The cost of dual controller multi-protocol arrays from two major vendors and the cost of direct attached storage (DAS) solutions from two major vendors were also calculated. The solution with the lowest overall price in each category was used for the comparisons presented in this report.

The bottom line results are summarized in Figure 9. Note that the costs of iSCSI, multi-protocol, and FC SAN solutions are significantly higher than a comparable Coraid EtherDrive SAN system and that the base costs of a Coraid SAN solution are lower even than DAS.

Calculated costs are detailed in Table 3.

What the Numbers Mean

• Coraid EtherDrive SAN has the lowest cost of acquisition, by a wide margin.
• The relative cost of acquisition of alternative technologies ranges from roughly 1.4x for DAS to more than 5x for FC SAN.
• The FC SAN solution is so much more expensive in part due to the cost of acquiring FC SAN connectivity.
• DAS technology has a number of limitations that were not considered in this analysis. First and foremost, it is a dead-end when it comes to server virtualization. SAN attached storage is needed to take full advantage of the benefits of server virtualization. Storage capacity held captive within, or directly attached to, a server can’t be moved non-disruptively to another server for maintenance or better quality of service. SAN attached storage is also needed to achieve valuable disaster recovery capabilities that have recently become available from server virtualization vendors (e.g., VMware Site Recovery Manager). And finally, islands of DAS capacity typically lead to poor storage utilization. Poor storage utilization dramatically increases the overall cost of ownership.
• In addition to CAPEX, ESG Lab believes it is likely that Coraid EtherDrive’s simplified architecture and management would also yield OPEX savings over alternate technologies.

ESG Lab also compared price-performance for the Coraid EtherDrive SAN systems tested to publically available results published for DAS and traditional Fibre Channel SAN systems. Price-performance was determined using a simple calculation of cost in dollars for a specific configuration divided by the number of MB/sec supported by that platform.

Virtualization Optimized

Coraid EtherDrive SAN storage systems integrate with VMware using a simple driver that enables VMware to mount EtherDrive storage arrays as if they were local drives. A VMware administrator can provision and manage virtual machine storage without the need for FC SAN administration or iSCSI client configuration.

ESG Lab Testing

ESG Lab performed virtualization tests on a VMware ESX 4.0 environment with two physical servers and six virtual machines.

First, ESG Lab logged into the vSphere client and clicked on server 192.168.0.214. As seen in Figure 10, the Coraid EtherDrive HBA was visible in the list of storage adapters and volume 10, created using the steps in Figure 5, was visible and ready for use.

The volume was formatted and made available to virtual machines using the Add Storage wizard, shown in Figure 11.

Next, the volume was assigned to a virtual machine using the native VMware Add Hardware wizard. Once the addition was complete, the volume was visible to the Windows operating system on the virtual machine. Figure 12 shows the Windows Disk Administrator tool with the new drive circled in green.

Finally, ESG lab examined availability, testing the synchronous mirroring capability of the Coraid EMX EtherDrive Mirror Appliance as well as the ability to physically move disk drives between chassis without disruption.

The availability test bed, depicted in Figure 13, consisted of three Coraid EtherDrive SR2421 shelves, one EMX Mirror Appliance, and one vSphere server, with one virtual machine running Windows Server 2008.

Two 12-disk RAID5 LUNs were created on two separate shelves and synchronously mirrored through the EMX appliance. Mirroring two volumes using the EMX appliance could not have been simpler. The mkmir command was used to select the source and target volumes to be mirrored. This single command pairs the volumes and starts the synchronization.

Next, the volume was assigned to a Windows server 2008 VM on the vSphere server. Once the volumes were fully synchronized, an IOmeter workload was started on the server, performing a mixed read/write workload against the volume, set to continue indefinitely. Power to the primary SR shelf hosting one side of the mirror was killed. Iometer continued reading and writing to the volume with no errors.

Finally, a single eight-disk RAID5 LUN in a single chassis was used to test the online drive relocation capability of the Coraid architecture. The LUN was assigned to a Windows 2008 VM and an IOmeter workload was started on the server, again performing a mixed read/write workload against the volume, set to continue indefinitely.

Power was killed to the chassis housing the eight-drive RAID 5 LUN. All eight disks were then physically relocated from the primary chassis to a spare chassis. The spare chassis was then renamed to have the same shelf number as the original chassis and the eight-disk LUN was placed online.

Total time for this physical failover was approximately three minutes. After the LUN was placed back online, the IOMeter transactions resumed successfully with no further service interruption. Most, if not all, other architectures, including highly available Fibre Channel and iSCSI SANs, simply cannot take LUNs offline in a VMware environment while machines are running without bringing the server to a crashing halt.

ESG Lab Validation Highlights

• ESG Lab configured, provisioned, and was utilizing Coraid storage in less than two minutes from power on.
• The SRX3500 demonstrated the ability to support thousands of Exchange users using just 12 SAS drives.
• Coraid EtherDrive SAN was able to drive more than 1200MB/sec from a single appliance, enough to stream 2,560 broadcast quality video streams simultaneously.
• Commodity hardware and cost-efficient AoE connectivity enable a cost of acquisition far less than Fibre Channel, iSCSI, and even DAS.
• Coraid proved well-suited to virtualized environments, providing simple to provision SAN storage that looks to a VMware cluster like direct attached disk.
• The EMX Mirroring appliance provided synchronous data protection for volumes across shelves with no disruption to service.
• ESG Lab was able to remove drives that were actively being accessed and move them to a different chassis with only a momentary pause in IO and no errors.

Issues to Consider

• Coraid’s EtherDrive SAN is currently managed through a command line with no GUI. The system is incredibly simple to use and manage, with all necessary functions controlled through a few simple commands and logical, human readable addressing of shelves, disks, and LUNs. Coraid indicated plans to ship an upgraded management system in Q3 2010 with a GUI and REST API support.
• The Coraid EtherDrive SAN solution does not yet offer advanced storage virtualization functionality such as thin provisioning or storage tiering. The driving factor behind these features, reducing the cost of storage, does not necessarily affect Coraid as it does traditional SAN architectures, which typically sell for many multiples of Coraid’s acquisition cost. In addition, these features are increasingly available in software at the hypervisor or file system layer, further obviating the need for them as array-based features.

The Bigger Truth

With storage costs consuming at least 28% of IT budgets,[6] companies are under constant pressure to find ways to reduce costs. Taking a long hard look at reducing capital and operational costs in the storage environment makes sense and so today, more than ever, IT is investing in new technology with a clear focus on reducing storage costs.

The high capacity and performance requirements of scale-out applications including backup to disk, content delivery, server and desktop virtualization, clustered computing, rich media, and un-structured bulk storage are taxing the budgets and infrastructure of IT organizations. Traditional storage network infrastructure can provide the capacity, agility, and performance these applications need, albeit at a high cost of entry and daunting complexity.  Rows of equipment are often needed to provide a petabyte of capacity and gigabytes per second of throughput. Data center managers are being pushed to the limit as administrators spend more and more time managing an ever-expanding SAN infrastructure.

ESG Lab found that the Coraid EtherDrive SAN storage system delivers shockingly simple deployment and management, with complete functionality delivered via a handful of easy to use commands and rock solid Ethernet SAN connectivity delivered via the extremely lightweight AoE protocol. The ease of management, deep scalability, and performance required for bandwidth-intensive scale-out applications are seamlessly extended to VMware environments as well.

ESG Lab testing has confirmed that Coraid’s architecture provides consistent levels of throughput—even during hardware faults.  Sustained throughput in excess of 1,200 MB/sec was observed for large block sequential reads. Cost-efficiency was impressive, with acquisition costs as low as 20% of the costs of traditional SAN attached storage. ESG Lab also verified a very interesting recoverability and resiliency feature, whereby drives can be moved to a spare chassis while an application is running.

With EtherDrive SAN storage, Coraid has dramatically simplified storage for consolidated and virtualized environments while enhancing performance and providing incredible cost efficiency. While the speeds and feeds are impressive, ESG Lab is most impressed by the shocking simplicity of both the AoE protocol and the Coraid architecture, making management of petabytes a reasonable task. If your organization is struggling to keep up with exponential data growth while providing ever higher levels of performance and availability, ESG Lab recommends that you consider Coraid EtherDrive SAN storage as the foundation for your virtualized data center.

Appendix

[2] Source: ESG Research Report, 2010 IT Spending Intentions Survey, January 2010.

[3] Configuration details can be found in the appendix.

[4] http://msexchangeteam.com/archive/2007/01/15/432207.aspx

[5] Source: ESG Research report, The Impact of Server Virtualization on Storage, December 2007.

[6] Source: ESG Research Report, Enterprise Storage Survey, November 2008.

Background

Server virtualization adoption is on the rise among organizations of all sizes and all industries around the world.  As a matter of fact, a recent ESG survey indicates that increasing the use of server virtualization is currently IT managers’ most important priority.[1] Given the impressive economic benefits of server virtualization, the glut of affordable and under-utilized processing power, and growing power and cooling issues in the data center, ESG predicts that the brisk adoption of server virtualization will continue for the foreseeable future.

As more and more applications are deployed within a consolidated virtual server infrastructure, IT managers are struggling to meet the challenges associated with storing and protecting a single pool of application data.   Exploding data growth, in part due to virtual server sprawl, is driving IT managers to carefully consider how they meet the storage needs of virtualized applications.  This is clearly shown in Figure 1, which indicates that improving data backup and recovery, managing data growth, and data center consolidation are top IT priorities.

This balance of this report explores how networked storage solutions from NetApp are ideally suited for the consolidation of popular Microsoft business applications deployed in a virtual server environment.

Unified Windows Storage Consolidation

NetApp storage solutions offer a Unified Storage Architecture that is ideally suited for server virtualization and Windows consolidation. As shown in Figure 2, a mix of business applications from Microsoft (Exchange, SQL Server, SharePoint, Active Directory) can be consolidated using a cluster of virtual servers (e.g., VMware vSphere) that are network attached to a NetApp  storage system  (e.g., NetApp FAS2040).

The NetApp Data ONTAP operating system that is built into each and every NetApp storage system supports a unified mix of host protocols (NFS, CIFS, iSCSI, FC, FCoE) and disks (FC, SAS, SATA) as it creates a unified platform which increases storage efficiency, simplifies data management, improves data protection, and reduces costs.

Unified storage efficiency is provided with powerful NetApp technologies including thin provisioning, FlexClone, primary storage deduplication, and space-efficient snapshot backups. Unified management simplicity is provided with an end-to-end family of management tools, including an application-aware family of SnapManager products (e.g., NetApp SnapManager for Microsoft Exchange), NetApp VSC (VMware plug-in), and a potent, yet intuitive, storage manager (NetApp System Manager).

ESG Lab Validation

ESG Lab performed hands-on testing of a unified Windows storage consolidation solution over the course of two days at a NetApp facility in Research Triangle Park, North Carolina.

Getting Started

An overview of the test bed is a good place to get started: a pair of servers running VMware vSphere server virtualization software was attached to a NetApp FAS2040 storage system over a 1Gb Ethernet network.[2] The NFS protocol was used to access virtual machine images and the iSCSI protocol was used to access Microsoft application data and additional virtual machines images stored on VMware VMFS. The VMware vSphere Client shown in Figure 3 was used to browse the pre-configured test bed. Note how a mix of Microsoft (e.g., Exchange, SharePoint, SQL Server) and Linux (e.g., Apache) applications was consolidated within a vSphere cluster sharing a consolidated pool of NetApp storage capacity.  Also note that a NetApp tab has been added so that common storage management tasks could be directly performed from within the VMware vSphere Client.

Manageability

The NetApp Unified Storage Architecture extends beyond the boundaries of the storage hardware with a unified collection of management software utilities—at the Microsoft application level, the VMware level, and the storage system level. Management at the application level is provided by software packages (e.g., SnapManager for Microsoft Exchange). Management at the VMware level is provided with plug-ins for the VMware vSphere Client management console (a.k.a., the NetApp VSC Plug-in for vSphere).  Management at the storage system level for routine tasks, including initial configuration and reporting, is provided by NetApp System Manager. Taken together, the unified management tools from NetApp use intuitive wizards and context-aware interfaces to simplify the management of virtualized Microsoft applications deployed within a consolidated infrastructure.

ESG Lab Testing

ESG Lab used VSC version 2.0 to demonstrate the power and simplicity of a unified management approach in a consolidated Microsoft environment. The VSC Plug-in was installed on a server running the VMware vSphere Client. As shown in Figure 4, VSC can be used to perform routine storage management tasks using an intuitive right mouse click from within the vSphere Client. In this example, a right mouse click was used to resize an iSCSI attached VMFS datastore hosting .vmdk files for a virtualized Microsoft Exchange application. Four mouse clicks and two minutes after getting started, the datastore capacity was increased from 500 GB to 600 GB.

A similar intuitive right mouse click from vCenter was used to create a new datastore for a consolidated mix of Microsoft applications.  Note how the wizard-driven interface shown in Figure 5 can be used to create a thin provisioned datastore that delivers just-in-time storage capacity. In this example, a 10 GB datastore has been configured to auto-grow in 1 GB increments to a maximum size of 200 GB. Six mouse clicks and two minutes after getting started, the new datastore was ready to serve the needs of a consolidated mix of Microsoft applications.

Nest, ESG Lab used the wizard shown in Figure 6 to automate the storage provisioning and cloning of ten virtual machines.

Seven mouse clicks and less than five minutes later, the VSC rapid cloning wizard had automated the creation of ten virtual machines sharing the same guest operating system (Windows Server 2008 R2). In this example, VMware virtual machine images (.vmdk files) were stored on the NetApp FAS2040 storage array using the iSCSI and NFS protocols accessed over an industry-standard 1 GbE network.

ESG Lab was particularly impressed with the best practices automation capabilities built into the VSC version 2.0.   As shown in Figure 7, the NetApp tab accessed from the vSphere Client was used to determine whether VMware ESX hosts were configured according to NetApp ESX host specific best practices. As shown at the top, the configuration settings that didn’t meet best practice guidelines were shown as red alerts. An intuitive right click was used to learn what needed to be fixed and to automatically correct the problem.  In this example, three recommended settings were changed with a single mouse click: an incorrectly configured host bus adapter, a multi-path driver, and NFS heartbeat settings.  As shown at the bottom, automatically setting values according to NetApp best practices eliminated the red alerts.

Recoverability

NetApp snapshots provide instant, space-efficient backup and recovery in consolidated Microsoft environments.  Point-in-time disk-based snapshot images can be used to quickly restore deleted or corrupt files. Snapshot images can also be used to dramatically shrink backup windows. The capacity consumed for each incremental snapshot is reduced to the capacity required to store the 4K chunks that have changed since the last snapshot was taken, making snapshots cost effective as system administrators keep multiple disk-based versions for quick and reliable restores.

The real power of snapshots is realized when they are used in conjunction with NetApp’s growing family of application-aware SnapManager software tools running on a server. With SnapManager, wizard-driven application consistent backup and granular recovery can be performed at the application level instead of at the volume level. For example, SnapManager for Exchange can be used to restore an individual user’s e-mail and SnapManager for SQL Server can restore an individual table within a database.  SnapManager tools are also available for restores at the operating system level (e.g., Microsoft Windows) and the virtual server level (e.g., VMware vSphere and Microsoft Hyper-V).  Sub-plug-ins for VSC can also be used to manage the protection of applications, virtual machines, and operating systems from the vSphere Client.

ESG Lab Testing

ESG Lab used NetApp SnapManager for Exchange and SQL Server to perform space-efficient disk-based backups and restores for Microsoft Exchange and SQL Server running in a consolidated VMware infrastructure. The schema for the SQL Server database used during ESG Lab testing is shown in Figure 8.  In this example, a 215 MB database full of baseball statistics was tested.

A wizard accessed from the NetApp SnapManager for SQL Server interface was used to perform a space-efficient disk-based backup.  Database corruption was simulated by deleting two of the tables (dbo.Batting and dbo.BattingPost). The wizard shown in Figure 9 was used to recover the database.  Less than three minutes after getting started with the intuitive NetApp SnapManager for SQL Server wizard, the restore had completed.   Microsoft SQL Server Management Studio was used to verify that the database had been restored correctly.

ESG Lab performed a similar set of tests with Microsoft Exchange. A 2 GB Exchange database with 40 pre-populated mailboxes was used during this stage of testing. The Exchange database was deleted to simulate a catastrophic operator error.  Less than two minutes after getting started with the restore wizard, Microsoft Exchange was up and running.

Efficiency

The NetApp Unified Storage Architecture uses a variety of technologies to increase storage efficiency in a consolidated Microsoft environment.  FlexVol with thin provisioning, FlexClones, and NetApp Deduplication are particularly powerful.

FlexVol with thin provisioning increases storage efficiency by providing just-in-time capacity for applications accessing storage using block-based or file-based protocol (e.g., iSCSI, FC, FCoE, CIFS, NFS). Instead of allocating the maximum amount of storage that an application might use over time, NetApp Thin Provisioning allocates capacity on demand from a shared pool of storage.

FlexClone reduces the capacity required to store clones of operating system and application images in a consolidated Microsoft environment. A cloned copy created with FlexClone through the Rapid Cloning Utility magnifies capacity savings as it stores differences instead of whole copies. NetApp Deduplication provides similar benefits as it reduces storage capacity by eliminating redundant chunks of data within a storage volume (block or file) via a background operation. Virtual server images are a great candidate for NetApp Deduplication because they tend to have a lot of data in common (e.g., operating system images and space reserved on disk for virtual server memory dumps).

The combined efficiency of NetApp FlexVol, Thin Provisioning, FlexClone, and Deduplication is illustrated in Figure 10. FlexVol with Thin Provisioning reduces storage capacity as it eliminates the differences between the capacity allocated to an application and what is actually consumed.  FlexClone and Deduplication eliminate duplicates to magnify the capacity savings.

The combined effect of NetApp capacity efficient technologies is backed by a 50% Virtualization Guarantee. As part of the program, NetApp is offering a guarantee that its customers will use 50% less storage compared to traditional storage in a VMware virtual environment, subject to terms and conditions.[3]

ESG Lab Testing

NetApp System Manager was leveraged to monitor the storage efficiency of a consolidated Windows environment during ESG Lab testing. Block-based volumes accessed via the iSCSI protocol (e.g., the iSCSI datastore hosting VMs) were configured to provide capacity on demand with FlexVol.  A NetApp Deduplication job was run on demand to eliminate duplicate data, including .vmdk files accessed via the iSCSI protocol. The Rapid Cloning Utility was used to realize additional capacity savings with NetApp FlexClone technology.  As shown in Figure 11, capacity requirements were reduced by 89% during ESG Lab testing.

A similar set of tests which measured the efficiency of NetApp Deduplication in a consolidated VMware environment was performed in a previous ESG Lab Validation.[4] During those tests, the capacity required to store 15 VMware virtual machine images was reduced by a factor of 97%.[5]

ESG Lab Validation Highlights

• The Unified Storage Architecture was used to consolidate a mix of file sharing and popular applications (Microsoft Exchange, SQL Server, SharePoint, Active Directory, Apache Web Server) in a virtual server environment powered by VMware vSphere.
• Unified management tools from NetApp were used to perform routine storage management tasks at the application level (e.g., NetApp SnapManager for Exchange), the virtual server level (NetApp VSC Plug-in for vSphere), and the storage system level (NetApp System Manager).
• The VSC 2.0 Plug-in was used to perform wizard-driven storage management tasks from the VMware management console.
• A Rapid Cloning wizard, accessed from the VSC-enabled VMware management interface, was used to perform a wizard-driven, space-efficient clone of ten VMware images in less than five minutes.
• FlexVol, FlexClone, and NetApp Deduplication reduced storage capacity requirements in a consolidated Microsoft environment by 89%.
• SnapManager for Microsoft SQL Server and Exchange were used to perform a wizard-driven, capacity-efficient restore of a database with simulated corruption.

Issues to Consider

• While ESG Lab tested a unified Windows consolidation solution with server virtualization technology from VMware, it should be noted that NetApp also supports tools and best practices for server virtualization technology from Microsoft (Hyper-V) and Citrix (XenServer).
• ESG Lab has confirmed that the combination of FlexVol, FlexClone, and Deduplication can be used to increase storage efficiency in a consolidated Windows environment by 89% or more, but storage administrators familiar with legacy storage systems need to change the way they’ve been managing storage capacity to take advantage of these capabilities.  Instead of waiting for an application or an operating system to signal that it is out of storage capacity, a FlexVol just-in-time storage pool must be monitored to make sure it never runs out of storage. The good news is that NetApp’s unified management approach can be used to simplify and automate these tasks using familiar management interfaces.
• While NetApp’s unified approach supports a broad variety of host interfaces, drive types, and management software packages, care should be taken when calculating the cost of a NetApp unified solution since many of the interfaces and software packages are separately licensed. This matters the most when comparing the price of a NetApp unified solution to a disk array from an emerging vendor that bundles a less exhaustive list of interface and software options into the acquisition price.

The Bigger Truth

IT managers are struggling to answer a number of questions as they look to meet the growing storage capacity needs of Microsoft applications.  As more and more Microsoft applications are classified as business critical, how will vital information assets be protected now and in the future? How will the organization keep pace with capacity growth of 50% or more annually while staying within budget?  How can more capacity be managed with existing service levels and existing staff?  Will storage investments complement—or complicate—a virtual server consolidation initiative? Can IT create a winning strategy that works for both the team and the business?

Windows file servers and islands of storage for Microsoft applications including Microsoft Exchange, SQL Server, and SharePoint are a waste of time and money.  Reliance on a mix of legacy servers and separate storage solutions to meet the needs of a mix of Microsoft applications increases capital equipment and ongoing operational costs. Costs are increased due to a variety of reasons, including inefficient server utilization, over-provisioned disk capacity, increased complexity for storage administrators, and the ever-rising cost of space and power.

For more than a decade, NetApp has been delivering unified storage systems with a single operating system that runs on all of its disk arrays.  Starting with a unified storage system that supports a unified mix of storage protocols and drive types, NetApp has evolved to create a Unified Storage Architecture which increases storage efficiency, simplifies data management, improves data protection, and reduces costs.   The storage efficiency gains in a consolidated Microsoft environment are particularly noteworthy. During ESG Lab testing, storage capacity requirements were reduced by 89%.

When we first tested a unified NetApp storage system with iSCSI support in 2003, ESG Lab was extremely impressed with the native Windows look and feel of the NetApp SnapDrive for Windows plug-in. Since then, NetApp has focused on the development of best practices and application-aware management interfaces for Microsoft applications.  During this most recent round of testing in 2010, ESG Lab was extremely impressed by the VSC version 2.0 Plug-in and the SnapManager interfaces for Microsoft Exchange and SQL server. As of the writing of this report, ESG Lab had tested more than a dozen storage systems in virtual server environments.   Based on our experience to date, ESG Lab believes that NetApp has the most powerful, intuitive, and consistent set of management tools for consolidated Windows environments.

If your organization is struggling to keep up with data growth, keep costs in check, and increase the availability of Microsoft business applications in a VMware environment, ESG Lab recommends that you start with an evaluation of the savings that can be realized with the NetApp Guarantee: NetApp guarantees that customers will use 50% less capacity compared to traditional storage in a VMware virtual environment. [6] With an integrated family of management capabilities, a common code base that supports all of the popular block and file-based storage interfaces, capacity efficiency that’s ideally suited for virtual server environments, and application-aware snapshots that are fast and efficient, ESG Lab has confirmed that the Unified Storage Architecture from NetApp is a sound foundation for Windows storage consolidation.

Appendix

[2] For more configuration details, please refer to the Appendix.

[3] For more information, visit www.netapp.com/guarantee

[4] Source: ESG Lab Validation Report, NetApp Deduplication for FAS, April 2008.

[5] The level of capacity reduction that can be expected in production environments will vary depending on the number of virtual machines sharing the same storage system and the degree of data commonality between virtual machines. In this example, there were relatively few virtual machines (15) with a high degree of data in common (each machine was running the same operating system and application).

[6] For more information, visit www.netapp.com/guarantee where you can access program details including FAQs, technical resources, best practices, and the new NetApp Storage Efficiency Calculator to evaluate your current environment and learn how NetApp storage efficiency technologies can lead to higher storage utilization.

Background

Server virtualization is a strategic initiative for IT organizations around the world.  ESG conducted an in-depth survey of senior IT professionals concerning their organizations’ most important IT priorities for the next 12-18 months;[1] as shown in Figure 1, server virtualization is clearly the number one priority.

It’s no surprise that server virtualization is a top priority given the dramatic benefits achieved by early adopters.  As more and more applications are deployed onto a consolidated pool of servers, better utilization is slashing the capital costs of equipment.  Operational costs are dropping as well. A consolidated pool of servers reduces the ongoing cost of power, cooling, and space in the data center.  The manpower costs associated with deploying a new application drops as IT managers take advantage of centralized and automated management tools.

Besides the obvious benefits associated with increasing the efficiency of servers in the data center, server virtualization can also be used to enhance the flexibility and availability of IT applications and services.   As applications are freed from the confines of a physical server infrastructure, server virtualization can be used to move those applications between servers and protect users from outages due to a physical server failure. In fact, ESG research indicates that virtual machine disaster recovery, backup, and mobility follow closely behind consolidating more physical servers in the list of the top ten server virtualization initiatives.[2]

Introducing Hyper-V R2

Microsoft Hyper-V R2 is a bare metal hypervisor that enables the hosting of multiple virtual machines on the same physical server. The supported virtual machines can be a mix of almost all Microsoft (server and desktop) platforms in addition to a couple different Linux platforms. Hyper-V R2 is available in the Windows 2008 Server R2 operating system as well as Microsoft Hyper-V Server 2008 R2. Using familiar interfaces and wizards, Hyper-V R2 lets companies take advantage of existing Microsoft skill sets, training programs, and certifications.

Hyper-V R2, which was released in September 2009, includes a number of valuable enhancements to Microsoft’s server virtualization platform.  Performance enhancements include improved performance for virtualized applications which rely on virtual hard disks.  New capabilities including Live Migration and Clustered Storage Volumes were added as well.   As shown in Figure 2, Live Migration enhances the mobility of virtual machines while clustered shared volumes enhance availability during failover clustering.

Live Migration, which was introduced in Hyper-V R2, differs from Quick Migration, introduced in Hyper-V R1.  A quick migration of a virtual machine from one physical server to another incurs a brief period of server unavailability. This gap, which typically last a few seconds, may be felt by end-users and in some cases may require an application restart. Live Migration eliminates this possibility—running virtual machines are moved between physical servers with no interruption.

A Clustered Storage Volume (CSV) is a highly available, shared storage volume that extends the capabilities of a Failover Cluster.  A Failover Cluster is a collection of computers (nodes) that work together to increase the availability of an application or service.  CSV allows multiple servers in a cluster to read and write at the same time while not having one node responsible for the volume’s management.   When a Live Migration is executed and its storage is provided by a CSV, the migration can occur very quickly since only the memory of the virtual machine needs to be migrated.

ESG Lab Validation

ESG Lab performed a hands-on evaluation of Hyper-V R2 server virtualization technology in a lab located on Microsoft’s Redmond, Washington campus.  Testing was performed with a goal of evaluating the flexibility, availability, and performance enhancements introduced in Hyper-V R2.

Getting Started

The test bed used during the ESG Lab validation was designed to emulate a physical to virtual server migration, as shown in Figure 3. Commonly deployed applications and services running on nine older physical servers were consolidated onto a cluster of new servers attached to a shared pool of SAS drives within a FC attached disk array.[3] The servers were configured to support a mix of services and applications including Active Directory and print services, network shared drives (Corp, Dept, Home), Intranet(IIS),  Red Hat Linux(RHEL), and databases (Microsoft SQL Server 2005 and 2008).  Microsoft Windows 2008 R2 Datacenter Edition, with built-in Hyper-V R2 support, was used to consolidate virtual servers onto a Failover Cluster using SAN-attached storage and Clustered Shared Volumes.  A fourth server was used to manage the physical and virtual environment with Microsoft System Center Virtual Machine Manager (SCVMM). [4]

ESG Lab Testing

ESG Lab testing began with a bare metal installation of Windows 2008 R2 on the first 64 bit server.  Six mouse clicks and 20 minutes later, the familiar Windows installation process was complete.  At this point, the server looked and behaved like any physical server running a Microsoft Windows operating system.

The wizard shown in Figure 4 was used to enable the Hyper-V role from the Server Manager console on the physical server.   The wizard-driven user interface used to enable the Hyper-V role was similar to the one used for configuring common server roles including Active Directory and print services.

Six mouse clicks and a reboot later, the Hyper-V R2 enabled server was ready for the configuration of the first virtual machine.

At a high level, three methods are commonly used to create a virtual machine (VM) in a Hyper-V environment:

1. Create a new VM from scratch
2. Clone an existing VM
3. Import an existing physical or virtual server

A wizard launched from the Hyper-V Manager console on the physical server was used to create the first VM from scratch.  Clone and import methods were used later in the ESG Lab validation to create additional virtual machines. As shown in Figure 5, a Hyper-V Manager wizard accessed from the Server Manager console was used to configure the first virtual machine with 512 MB of virtual memory, a virtual network connection, and a virtual hard disk (VHD).

Ten mouse clicks and 42 seconds after getting started with the wizard, the first virtual machine was running a network-based operating system install.

ESG Lab found that getting started with Hyper-V R2 is a straightforward, intuitive process.   At a high level, familiar wizard-driven graphical user interfaces built into Windows Server 2008 R2 were used to configure Hyper-V in three steps:

1. Install Windows Server 2008 R2
2. Enable Hyper-V role
3. Reboot and configure virtual machines

As shown in Figure 6, the first virtual machine was up and running 25 minutes and 22 mouse clicks after starting a bare metal installation of Windows Server 2008 R2.

Availability

Microsoft has improved the availability of applications running within a virtual server infrastructure with Hyper-V R2 enhancements, including a new feature called Clustered Storage Volume (CSV). A CSV can be created and mounted by servers that are attached to the same storage system through a storage area network. By simply turning on the failover and clustering feature on two or more nodes in a cluster, CSV provides automated and transparent failover of virtual machine from one physical server to another. CSV can also be used to accelerate the migration of virtual machines within a cluster.   While CSV creates a single name space which enhances the availability of a Microsoft-enabled virtual server infrastructure, it is recommended but not required for transparent failover or migration.

As shown in Figure 7, cluster shared volume support in Hyper-V R2 enables transparent failover. In this example, a virtual machine running on the failed server on the left has been configured to use a SAN-attached clustered shared volume, which is visible to both of the servers in the cluster. When the failure occurs, running applications fail over transparently to the healthy server on the right.

ESG Lab Testing

ESG Lab tested a CSV-enabled failover using a Microsoft SQL Server 2008 database running within a virtual machine hosted by Hyper-V R2. The database was installed on a clustered shared volume running within a three-node Hyper-V R2 cluster. A SQL select statement was run against a 952,576 row production database harvested from ESG’s internal IT operation.  The long running SQL query performed a join of three tables. The server running the SQL query was powered down as the SQL query was running.

After a few seconds, a remote desktop session connection with the virtual machine was lost.  The SCVMM console was updated to show that the server was no longer available. Forty seconds later, the remote desktop session reconnected automatically with the virtual machine running on a healthy virtual machine in the cluster.

Figure 8 provides an overview of the failover process. The “before” picture on the left shows the state of the cluster before the server was powered down.  At this point, the virtual machine was running on the third node in the cluster (WS08R2-N3). The “after” picture on the right shows the state of the cluster after the failover. Note that SCVMM has changed the status of the cluster and the third node to show a yellow alert.

The performance graph shown in the middle of Figure 8 was captured on the virtual machine running the SQL query during the failover. Note the spike in CPU activity and drop in network activity during the actual failover. Less than three minutes after injecting the error, the virtual machine had automatically failed over to a healthy node in the cluster.  The SQL query that was started before the server failure completed without error.

Flexibility

Live Migration is one of the highlights of Hyper-V R2.  As shown in Figure 9, Live Migration is used to move a running virtual machine from one physical server to another with zero downtime.

Live Migration differs from Quick Migration, introduced in Hyper-V R1, because Quick Migration causes a brief period of application and network interruption as the virtual machine is paused and restarted on a new server. This pause, which lasts less than a minute and can be tolerated by most applications, has been eliminated with Live Migration.

ESG Lab Testing

Selecting which virtual machine to migrate, and which server to move it to, was intuitively simple using Microsoft System Center Virtual Machine Manager (SCVMM). The SCVMM GUI was used to drag and drop virtual machines from one server to another. A right click on a running virtual machine provided yet another intuitive way to kick off a Live Migration. This method is shown in Figure 11.

ESG Lab noted that that SCVMM is not required to perform a live migration.  While the Failover Cluster Manager snap in that is built into Windows Server 2008 R2 can also be used to perform a Live Migration,  ESG Lab found that performing and monitoring a Live Migration from SCVMM  is slightly more intuitive and user friendly.

As shown in Figure 11, a Live Migration with an active SQL query running took one minute and eighteen seconds to complete. The SQL query took a bit longer to run than before the migration (17%), but continued to run without interruption. An active remote desktop connection to the virtual machine remained connected throughout.

Note that since the SQL database was installed on a clustered volume that was shared by all of the virtual machines in the cluster, the migration did not require the movement of the database which was stored on a virtual hard disk.  In this example, a copy of a 40 GB virtual C: drive was avoided as Hyper-V R2 moved a virtual machine in less than two minutes.

Scaling Out

As shown in Figure 12, ESG Lab continued to populate a failover cluster (R2-Cluster) deployed on three 64 bit servers to run nine virtual machines supporting a mix of 64 and 32 bit operating systems.

Note that the Microsoft SCVMM GUI can not only be used to manage a consolidated cluster of virtual machines hosted by Microsoft Hyper-V hypervisors, it can also be used to manage virtual machines hosted by VMware ESX.

Performance

Hyper-V R2 includes a number of performance enhancements that increase virtual machine performance.  Hyper-V R2 added support for Second Level Address Translation (SLAT), which uses hypervisor offload capabilities in modern CPUs to improve VM performance. The maximum number of logical processors has increased to 64 and supporting up to 384 virtual machines per server. Live Migration across different CPU versions within the same processor family has also been added (e.g., Intel Core 2-to-Intel Pentium 4 or AMD Opteron-to-AMD Athlon).

Hyper-V R2 also improves the performance of virtualized applications that store information on a Virtual Hard Disk (VHD).  A virtual hard disk is a file that encapsulates a hard disk image. VHDs first were created to be the storage media for virtual machines.  Today, VHDs are used to ship trial versions of software, in backup solutions, for bug triage (e.g., customers can convert a physical disk to virtual and share it), and even to store multiple boot environments.  VHD was integrated into Windows Server 2008 R2 which added native support.  Native support means the technology has been integrated into the operating system.

ESG Lab Testing

ESG Lab performed a series of tests designed to confirm Microsoft’s claim that fixed VHD performance is on par with native physical drive performance in Windows Server 2008 R2. The performance of real-world applications before and after virtualization on a modern CPU with SLAT support was tested as well.  Performance testing began with an evaluation of fixed VHD performance on a high performance 64 bit server with 110 GB of RAM and four AMD Opteron 8439 processors with Rapid Virtualization Indexing (RVI is AMD’s version of SLAT), each with six cores (24 cores total).

A fixed VHD is a file that is equal in size to a logical drive when it is created.  Because the physical storage required for a fixed size VHD is allocated when the file is created, there is a better chance at optimal placement and organization on-disk which yields the best performance.  The disadvantage is that space is committed even if it is not used. A dynamically expanding VHD allocates capacity on demand as data is written by applications. While the performance of dynamically expanding VHDs has improved significantly in Windows Server 2008 R2, their read/write performance is generally slower than fixed disks.[6] ESG Lab tested the performance of a fixed VHD.

The industry standard open source Iometer utility was used to generate workloads which mimic the IO profiles of real-world applications typically deployed in a virtual server environment.  Iometer version 2008.06.18 was used. [7] Four workloads were tested:  SQL log, OLTP, file server, and Exchange DB.[8]

Native testing on a server running Windows Server 2008 R2 Datacenter was performed using a 40 GB physical disk (a.k.a., raw disk or bare metal disk). Results were compared with a 40 GB fixed VHD running within a Hyper-V R2 VM configured on the same physical server.  Capacity was provided by a Fibre Channel attached HP MSA storage array with six 15K SAS drives configured in a RAID-5 group (5+1).  A two minute Iometer test run with a five minute ramp period was used. The average of three test runs was compared. The results are shown in Figure 13.

What the Numbers Mean

• ESG Lab testing indicates that native VHD performance is nearly the same as a native physical disk in Windows Server 2008 R2 (95% to 99%).
• Excellent VHD performance is due to a number of performance improvements in Windows Server 2008 R2 including integration of VHD support in the operating system.
• Prior to Hyper-V R2, customers often relied on pass through disks to optimize the performance of applications which store information on a SAN attached disk arrays. Pass through disks can also be used to take advantage of advanced data protection and disaster recovery capabilities built into disk arrays (e.g., snapshot sand remote mirroring).   R2 performance improvements make VHDs a more viable alternative to a pass through disk in a SAN attached environment.

During the next set of tests, ESG Lab evaluated the performance overhead of Hyper-V R2 in Microsoft Windows 2008 R2 Datacenter edition.  Physical and virtual performance was compared for the following application workloads:

1. Application install:  a timed installation of Microsoft SQL Server 2008 using a distribution image stored on a network attached shared drive within a private network.
2. Directory level copy:  a timed copy an 860 MB directory with 2,014 files to a temporary directory.  The c:\windows\win32 directory was copied to a temporary directory on the same C: drive.
3. Subsequent copies:  the directory level copy was repeated with much of the IO activity happening in cache. The average of three cached copy operations was recorded.
4. SQL query:  a long running SQL select statement using a 952,756 row production database from ESG’s internal IT operation was timed.  The SQL query performed a join of three tables. The average duration of three select statements was compared.

Comparing physical and virtual performance on the same server was accomplished after a reboot with Hyper-V role enabled. The physical server was tested with all of the available CPU cores (64) and RAM (110 GB). During virtual server testing, the server was configured with a single virtual machine which used four CPU cores and 64 GB of RAM.   Physical and virtual testing was performed with a 40 GB logical C: drive. The virtual configuration was tested with the C: drive configured as an NFTS volume on a fixed VHD. The results are shown in Figure 14.

What the Numbers Mean

• The difference in performance for real-world application workloads is relatively low (2% to 11%)
• Some of the performance difference may be due to the fact that native Windows Server 2008 R2 tests were performed with all of the available CPU and RAM in the physical server (64 cores, 110 GB of RAM). In contrast, the Hyper-V R2 VM tests were performed with the maximum amount supported CPU and RAM for a single Hyper-V R2 virtual machine (8 cores, 64 GB of RAM).
• In ESG Lab’s opinion, the manageably low performance impact of Hyper-V R2 won’t be detected by the vast majority of end-users and applications.

ESG Lab Validation Highlights

• Using familiar Windows-based tools, ESG Lab installed and configured a Windows Server 2008 R2 Datacenter server with bare metal Hyper-V R2 virtualization support in 25 minutes and 22 mouse clicks.
• ESG Lab has confirmed that Clustered Shared Volume and Failover Clustering support in Microsoft Hyper-V R2 can be used to increase the availability of IT services as it provides quick and transparent recovery of virtual machines after a server failure. A remote desktop connection to a VM running a SQL query was restored forty seconds after powering down a physical server in a cluster.  The SQL query completed without error.
• Live Migration was used to move running applications between physical servers with zero downtime.  A remote desktop connection to a VM running a SQL query remained available throughout the migration. The SQL query completed without error.
• Nine virtual machines running a mix of common business applications and a variety of 32 and 64 bit Microsoft Windows operating systems were consolidated onto a cluster of multi-core blade servers.
• Microsoft management tools with a familiar look and feel (Server Manager with the Hyper-V Manager MMC plug-in and SCVMM) were used to deploy, manage, and optimize the virtualized infrastructure.
• ESG Lab measured manageably low fixed VHD and virtualized application overhead in a Microsoft Windows Server 2008 R2 environment (2% to 11%).

Issues to Consider

• While not tested during this ESG Lab Validation, one of the more compelling capabilities of Microsoft’s integrated suite of virtualization management tools is the ability to manage physical and virtual servers— including virtual servers running over a VMware hypervisor—from a single pane of glass.
• Another core capability not tested by ESG Lab is the ability to Live Migrate VMs between servers in the same processor family (Intel Core 2 to Intel Pentium 4).
• Applications requiring more than the 8 CPU cores and 64 GB of RAM supported by Hyper-V R2 are not a good candidate for server virtualization.
• While a fixed VHD performs better than a dynamically allocated VHD, the disadvantage is that space is committed even if it is not used.

The Bigger Truth

Server virtualization is on a rapid and pervasive adoption path.  The cause—and effect—of this is the clear value proposition provided by server virtualization solutions. In a recent global research survey of 706 end-users, ESG found that the impact and benefits of implementing server virtualization are driving organizations to trust server virtualization with some of their most mission critical applications as they recognize that the benefits extend well beyond physical server consolidation and improved resource utilization.

Clearly, server virtualization provides a wide range of benefits, including better server resource utilization; increased consolidation of physical servers; and reduced consumption of floor space, power, and cooling.  In addition, ESG is finding that server virtualization is rapidly becoming an enabler of server management and data center automation. Virtual machines can be quickly provisioned, optimized, and tracked throughout their lifecycles and management tools are beginning to be able to manage physical servers and virtual machines as well as heterogeneous server virtualization solutions.  Disaster recovery and high availability are important—in some cases, they are the primary drivers for running production workloads on a virtualized infrastructure.

ESG Lab has confirmed that Microsoft’s growing family of server virtualization solutions have been built with each of these benefits in mind. Bare metal Hyper-V R2, which is built into Windows Server 2008 R2, provides the underlying virtualization technology that enables server consolidation. Wizard-driven configuration with a familiar Windows look and feel enables rapid server provisioning (less than 30 minutes during ESG Lab testing). A robust family of virtualization enabled management tools ensures that virtual—and physical—servers can be quickly provisioned, optimized, and tracked throughout their lifecycles.  Powerful flexibility and availability capabilities tested by ESG Lab—including Live Migration, Clustered Shared Volumes, and improved performance—are turning Microsoft server virtualization technology into an enabler of server management and data center automation.

Microsoft’s growing suite of management tools (e.g., SCOM and SCVMM) magnifies the value of Hyper-V R2.  With Microsoft System Center, you can manage at the application level regardless of whether applications are deployed on a physical server or a virtual server. You can manage virtual machines running on a mix of hypervisors from Microsoft, VMware, or Xen.

ESG Lab testing has proven that the performance overhead of Hyper-V R2 is manageably low compared to the outstanding benefits of server virtualization.  Moreover, ESG believes that a misdirected focus on the relative differences in performance between competitive hypervisor technologies is a disservice to IT managers. Given the dramatic benefits of server virtualization and the rapid advances in multi-core server technology, a relative difference between technologies that are 90% or more efficient matters little. The overall capabilities of the solution, including how well it works with existing processes and technologies,  matters much more. As an analogy, imagine you were trading in a car that gets 10 miles per gallon for one of two cars that each gets more than 100 miles to the gallon. Both of the new cars will be more than ten times more efficient that your old car. For most car buyers, an efficiency difference of a mile or two per gallon would be a minor consideration compared to price, performance, service, and options.

Speaking of price, ESG Lab believes that the cost of purchasing and owning a server virtualization solution is a vital consideration. Microsoft Hyper-V Server 2008 R2 is a free download, limited in terms of support for total memory, number of processors, and quick migration. Microsoft Windows Server 2008 R2 Standard, Enterprise, and Datacenter editions each include a Hyper-V R2 license for a single physical server as well as licensing for one, four, and unlimited virtual machines, respectively. Considering the additional savings that can be achieved when you leverage existing investments in Microsoft training, certification, and interoperability, the bottom line is simple: IT managers save time and money with the Hyper-V R2 support built into Microsoft Windows Server 2008 R2.

Appendix

[2] Source: ESG Research Report, 2009 Data Center Spending Intentions Survey, March 2009.

[3] Configuration details are listed in the Appendix.

[4] For simplicity, the management server is not shown in this diagram.

[5] Source: ESG Research Report, 2010 IT Spending Intentions Survey, January 2010.

[6] Microsoft indicates that the performance of a dynamic VHD support is approximately 87% of a native hard disk.  http://blogs.technet.com/b/virtualization/archive/2009/07/22/windows-server-2008-r2-hyper-v-server-2008-r2-rtm.aspx

[7] http://sourceforge.net/projects/iometer/

[8] To learn more about ESG Lab Iometer workloads, see http://www.enterprisestrategygroup.com/using-esg-lab-workloads/.

Background

ESG recently completed a survey of IT professionals with a goal of understanding the growing interest in virtual desktop infrastructure (VDI).[1] As seen in Figure 1, simplification tops the list of factors driving the adoption of VDI technology. Specifically, administrators are looking to simplify the repetitive, hands-on tasks of OS and application deployments, upgrades, patch management and provisioning.  Given the budgeting and manpower challenges being driven by world-wide economic concerns, it’s not surprising that more than half of respondents indicated that that reducing capital and operational expenses is driving an interest in VDI adoption.

In order to address these challenges, a VDI solution must be easy to deploy and manage, highly virtualized, highly available, and predictably scalable. N-way clustered storage architecture is ideally suited to address all of these issues. Clustered storage aggregates multiple storage controllers into a single storage cluster. Though these clusters may contain many storage controllers, they still appear to the applications and users as a single logical pool for easy management. In traditional dual node storage systems with fixed architectures, when a user’s environment outgrows their storage system, they may be forced to buy another system to achieve greater performance or capacity. Clustered storage systems allow users to add CPU, memory, and bandwidth transparently, enabling them to scale based on the needs of the business without purchasing a whole new storage system. Such clustered architectures allow for the aggregation and virtualization of all hardware resources, performance, and capacity in a linear fashion—just-in-time and as needed.

Virtual Desktop Infrastructure with Citrix XenDesktop

Citrix XenDesktop is a desktop virtualization system that centralizes and delivers desktops as a service (DaaS) to users—anywhere. The XenDesktop solution includes virtualization software for hosting desktops, user and session management, provisioning tools, and application delivery as well as service monitoring, reporting, and support.  Leveraging the Citrix HDX suite of technologies, the solution works to optimize the user experience for diverse user scenarios. XenDesktop supports multiple forms of hosted virtual desktops, including virtual machine (VM)-based and blade workstation based virtual desktops. XenDesktop also offers a desktop streaming feature that delivers a master desktop image directly to a physical endpoint.

The HP P4000 provides a true clustered architecture with advanced features and functions that make it a compelling solution for VDI. While DAS (direct attached storage) could be used for VDI, networked HP storage provides universal access while enabling high availability, desktop mobility, and online scalability that is impossible to achieve with DAS. Figure 2 illustrates a virtual desktop environment utilizing Citrix XenDesktop and HP P4000 SAN.

Users connect to the Desktop Delivery Controller via a secure web browser session (1). The Desktop Delivery Controller authenticates a user’s credentials against Active Directory (2), and then Citrix XenDesktop assembles the user’s virtualized desktop on demand (3), using volumes residing on the HP cluster. Virtual desktop delivery is optimized via Citrix HDX technologies for low bandwidth and high latency WAN connections (4). The user has access to their personalized desktop, applications, and resources from anywhere while still benefiting from centralized desktop management in the data center.

The HP Reference Configuration for Citrix XenDesktop on XenServer

HP has collaborated with Citrix to develop a reference configuration designed to provide robust, predictably scalable VDI solutions that are jointly tested and certified. HP and Citrix have developed a number of recommended configurations and services for Citrix applications; this report focuses on Citrix XenDesktop on XenServer.

Based on HP ProLiant BL460c G6 server blades and HP StorageWorks P4500 G2 SAN storage, the reference configuration shown in Figure 3 is intended to help end-users understand the components and interactions of a VDI architecture and provide a reference end-users could use as a starting point to build a virtual desktop solution capable of supporting approximately 1,000 Microsoft Office 2007 users on Microsoft Windows XP.

Specific configurations will vary based on users’ particular environments and needs. HP suggests users work with their HP Reseller or Sales Representative to help determine the best solution for individual environments.  Also, this testing focused solely on the virtual desktop (sometimes referred to as VDI) capability within XenDesktop; the hosted, shared desktop model of desktop virtualization enabled by the XenApp capability within XenDesktop has been tested and modeled separately by HP, and reference configurations have been developed specific to that operating mode.[2]

The HP StorageWorks P4000 SAN

The HP P4000 SAN is a clustered storage system that scales to meet the needs of VDI environments with ease. HP P4000 SANs are built on enterprise-class, industry-standard platforms configured as fully contained storage nodes that provide CPU, memory, bandwidth, and capacity. Each storage node is powered by SAN/iQ storage software, which provides intelligent storage system functionality. Customers can scale performance and capacity online as needed by adding additional storage nodes without disruption to the SAN, VM’s or physical server applications. The HP P4000 SAN remains a single logical system regardless of how many storage nodes are added to it, making it just as easy to manage a 16-node cluster as it is to manage a 2-node cluster. Additionally, adding nodes to the cluster is a transparent and non-disruptive process. HP advises ESG that the average cluster size sold is 4-6 nodes. The average cluster size deployed in the field is 15-20 TB (4 nodes) and 20% of the clusters deployed in production contain more than 50TB (10 nodes).

At the core of the HP P4000 SAN’s value is  the HP SAN/iQ storage software platform, which provides SAN management features such as  storage clustering,  application integrated snapshots, thin provisioning, remote copy (asynchronous replication), and SmartClone volumes.  In addition, SAN/iQ includes the unique Network RAID feature, which protects against component and environmental failures while keeping data volumes online and accessible.  The Network RAID feature provides a level of high availability usually found only in the most expensive SAN arrays, often as an optional software component. Network RAID is included with every P4000 SAN and can be enabled, modified, or disabled online. The ability to keep a volume online and accessible is a key benefit to the VDI environment as the loss of volume access could affect dozens, if not hundreds, of desktop users.  The P4000 comes with all management functionality built-in. There is no additional software to purchase.

HP BladeSystem Enclosures and Servers

HP BladeSystem is a converged infrastructure solution for data centers of all sizes. HP BladeSystem enclosures and servers minimize energy, cooling, and space requirements by consolidating powerful physical servers into a dense chassis, while simplifying administration through IO virtualization.

The enclosure selected for the reference configuration is the HP BladeSystem c7000. The c7000 enclosure consolidates the essential elements of a data center – power, cooling, management, connectivity, redundancy, and security, in a high-density, modular package.

The server blade selected for the Enterprise reference configuration is the HP ProLiant BL460c G6, which provides enterprise-class features for high performance and reliability along with energy efficiency.

ESG Lab’s testing was designed to validate the business value of deploying an HP P4000 SAN to support a Citrix XenDesktop VDI, including capacity, performance, and operational efficiencies uniquely enabled by the HP solution.

ESG Lab Validation

ESG Lab audited the HP reference configuration for Citrix XenDesktop on XenServer and conducted hands on testing of the HP P4000 SAN with Citrix XenDesktop VDI at a Hewlett-Packard facility in Houston, Texas.

Getting Started

The test bed, shown in Figure 4, consisted of a pre-installed, pre-configured four-node HP P4000 SAN supporting a four-server Citrix XenDesktop virtual desktop environment. Two SANs were configured using HP ProCurve switches. A Windows workstation running Internet Explorer was used as a virtual desktop endpoint.[3]

XenDesktop offers both “assigned” and “pooled” hosted virtual desktops. An assigned virtual desktop provides each user with a dedicated virtual machine. Users connect to the same machine each time and all changes and personalizations persist between sessions. The assignment can either be pre-determined by the administrator or pulled from a group of available desktops and assigned on first access.

Pooled desktops are a group of virtual desktops offering a standard configuration. Users are connected to any of the available desktops and when they log off, that desktop is returned to the pool. Backgrounds, bookmarks, application settings, and other personalization can be captured separately in the user’s profile. System changes, such as installed applications, are discarded and the desktop is reset to its pristine state. This ensures that the virtual desktop always is in a known good state and the next user that connects will get a “fresh” desktop configuration. ESG Lab tested using the “assigned” method for this report.

ESG Lab Testing

ESG Lab began testing with provisioning and configuration of a new virtual desktop. A new volume was created in two steps using the P4000 CMC (Centralized Management Console), seen in Figure 5 and Figure 6.

ESG Lab right clicked on the navigation tree, seen in Figure 5 and selected New Volume, which launched the new volume dialog box.

Next, a name for the volume was created and the desired capacity was entered, as shown in Figure 6.

Finally, Figure 7 shows how the new volume was assigned to the four Citrix XenDesktop servers.

Once the volume was visible to the XenServer Resource Pool, the Citrix Access Management Console was used to add the new storage (Figure 8 ) and import a previously exported virtual machine image (Figure 9).

Figure 10 shows the newly created virtual desktop.  ESG Lab booted the virtual desktop and confirmed that it was accessible from the endpoint machine. The entire process, including storage provisioning and allocation, took less than ten minutes.

Storage Efficiency

Traditionally, virtual desktop environments are built by creating volumes that will act as a remote user’s primary hard drive, holding their operating system and applications. An administrator will create the volume for the new virtual machine and either install the client OS or (more commonly) import a previously backed up image. This image is then managed as a physical desktop would be—application and OS patches must be applied to each VM individually and each image consumes as much storage as it would on a physical machine.

HP utilizes its P4000 SmartClone technology to optimize both the allocation process and the capacity consumption of virtual desktops. As shown in the top half of Figure 12, a single ‘gold image’ virtual desktop is built and then a snapshot is taken to create a base image. Thin provisioned SmartClones (volume copies) are created from the snapshot and presented to the Citrix XenDesktop server, which sees them as independent read-writable volumes. These volumes already have the OS image and applications installed on them, so the installation or import step is not needed. HP’s thin provisioning technology operates on a zero-reservation principle, meaning that no storage is pre-allocated to a SmartClone and data is only drawn from the allocation pool as new data is written.  Figure 11 illustrates how SmartClones would be leveraged to reduce storage requirements in a Citrix XenDesktop VDI.

Virtual desktops are typically very light with regard to the amount of data written as a percentage of the volume capacity, resulting in a very space efficient environment. Based on ESG’s experience in the lab and HP’s experience in the field, ESG Lab is confident that 70 to 90 percent capacity efficiency can be achieved over the life of a P4000 SmartClone in a VDI environment.

ESG Lab Testing

ESG Lab evaluated the storage efficiency of the HP P4000 SAN in a Citrix VDI environment by creating multiple virtual desktops using a single source volume.

First, ESG Lab accessed the Citrix Access Management Console and identified the volume to be used to create the new virtual desktops. As shown in Figure 12, the XP SysPrep Image VM was examined and the 8 GB storage repository volume sr1 was identified.

Next, a snapshot was taken of the volume sr1 and as seen in Figure 13, the New SmartClone Volume wizard was invoked by right clicking on the Snapshot.

Figure 14 shows the New SmartClone Volumes dialog box. Three SmartClones were created for this test (administrators can create up to 25 at a time). Thin provisioning was specified and permission was set to read-write. Finally, the SmartClones were assigned to the XenDesktop servers, as previously shown in Figure 7.

Before the SmartClone volumes could be added to the Citrix XenDesktop environment, the source volume sr1 needed to be detached and temporarily ‘forgotten’ by XenDesktop. This allows the new volumes to be added and modified so that their UUID (Universally Unique Identifier) and VG (Volume Group) could be changed to a unique value for each SmartClone. As seen in Figure 15, the process for adding the SmartClones is exactly the same as adding ordinary storage.

Finally, ESG Lab created a new Virtual Machine, pointed it to the SmartClone, and booted it up, as seen in Figure 16.

Storage utilization was confirmed using the P4000 CMC. The three cloned desktops, with a combined virtual capacity of 24 GB, consumed less than 100 MB of physical storage in addition to the 8GB of the source volume.

Citrix Essentials StorageLink for HP P4000 SANs

The Citrix Essentials product offering includes StorageLink API integration with HP SANs to enhance the scalability and agility of both Citrix XenServer and Microsoft Hyper-V virtualization environments; enabling simplified storage set-up and operation, VM lifecycle management, dynamic server provisioning and automated site recovery for DR sites. Leveraging StorageLink technologies, Citrix Essentials integrates storage management functions into the management console for the virtual infrastructure via wizards, enabling users to utilize the advanced services native within HP storage arrays from an easy to use GUI.

ESG Lab Testing

ESG Lab walked through the process of creating a template and cloning virtual machines using Citrix StorageLink manager. As Figure 17 shows, a storage profile was created first. A storage repository was selected which contained the volume to be used as the gold image for cloning.

Next, a virtual machine template was created using the just-created storage profile, as seen in Figure 18 and a predefined hardware profile for a power user’s desktop.  The template was named SL-XPSP2-A VM Template.

Finally, the template was cloned using the Create Virtual Machines wizard in the Citrix StorageLink Manager. The template created in the previous step was selected and finally, ESG Lab entered a name, selected the hypervisor host, selected clone as the copy type, and specified 10 clones, shown in Figure 19.

The cloning process took less than three minutes and when complete, all ten virtual machines were visible in XenCenter and ready for use.

Again, storage utilization was confirmed using the HP CMC. The ten cloned desktops, with a combined virtual capacity of 100 GB, consumed less than 100 MB of physical storage in addition to the 10GB of the source volume.

Performance and Scalability

In a virtual desktop environment, performance and scalability are determined more by the number and configuration of virtual desktop infrastructure servers than by any other factor. Storage performance requirements are less predictable than traditional IT applications and a storage solution in a VDI environment must be able to meet not only the average IO requirements, but the maximum load that will be generated—typically at the start of a shift when many users will all be logging on at once—while scaling to meet the capacity needs of a large user community.

ESG Lab Testing

ESG Lab used the Iometer workload characterization tool to simulate the type of IO generated by typical desktop operating systems and applications and audited scalability testing performed by HP.[5] ESG testing was focused solely on the storage back end and performed against two, four, and 12-node HP P4000 clusters. HP tested using a more real world, holistic approach that used a live XenDesktop environment, large numbers of vDesktops, and real world applications.

HP measured multiple characteristics during testing, including IOPS, server CPU utilization and user response time to determine the scalability of the various components of the XenDesktop environment.

HP observed a range of 15 to 20 Read IOPS during vDesktop creation and a range of 5-15 mixed read and write IOPS per vDesktop during VM Boot up.  After boot up, the sustained workload observed was 2.5 IOPS, consisting of nearly all writes. The Microsoft perfmon utility was used by ESG Lab to monitor the disk traffic for a physical Microsoft XP based knowledge worker’s desktop. An average of 20 IOPS was observed over multiple eight hour business days. With these data points in mind, a conservative value of 20 IOPS per virtual desktop user was used to estimate the number of virtual desktops that can be supported by an HP XenDesktop infrastructure.

Results recorded by ESG using the Iometer workload characterization utility, as well as results measured and projected by HP are detailed in Table 1. HP tested with one host running XenDesktop on XenServer against one two node P4000 cluster. Projections for larger configurations were based on the amount of IO that one server could drive. ESG tested with multiple virtual machines running Iometer, and were designed to determine the maximum amount of ‘desktop-like’ IO that a cluster could support. The results are summarized graphically in Figure 21.

ESG observed that the HP P4000 storage cluster scales IO nearly linearly. HP’s testing and projections for a real XenDesktop environment tracked closely to ESG’s testing. When sizing a complete VDI solution, care must be taken to follow vendors’ best practices for server sizing and configuration as well as storage.

What the Numbers Mean

• The number of virtual desktops that the infrastructure can support scales nearly linearly as servers and storage nodes are added to the HP P4000 cluster.
• HP’s reference configuration performance projections match closely with the observed performance results of ESG Lab HP P4000 testing.

High Availability

The HP P4000 architecture addresses availability at multiple levels. Hardware-based RAID technology is used within each server in a SAN/iQ storage cluster as a first line of defense against hard drive failures. In addition, SAN/iQ stripes data across all of the nodes in a cluster. In addition the Network RAID feature provides the option of spreading one or two extra copies of data throughout the cluster to protect against data loss due to the failure (or loss of connectivity) of a server participating in the cluster. A “stretched cluster” approach is also supported. For example, one half of a cluster could be located in a data center and the other half in a second location on a campus or in a building. In this manner, data loss can be avoided due to a localized facility error that affects half the nodes in the cluster (e.g., an overloaded power circuit or network failure).

ESG Lab Testing

Availability testing was performed against a stretched four node cluster on two separate gigabit Ethernet networks connected with a 10 gigabit Ethernet uplink. The Citrix XenDesktop servers had connectivity to all nodes in the stretched cluster.

ESG Lab introduced a variety of errors to validate fault tolerance. The stretched four-node SAN/IQ cluster shown in Figure 22 was used for hardware error injection testing. The following errors were injected as an Iometer workload was being run continuously on a virtual desktop running in the Citrix XenDesktop cluster as seen in Figure 23:

• Pulled a back-end Ethernet interface on node 1 at  Site 1
• Pulled an active disk drive
• Replaced the pulled drive
• Removed connectivity to both nodes at Site 1 from the cluster

Through all injected faults, Iometer continued to run on the virtual desktop without interruption. Next, ESG Lab transitioned the running virtual desktop between Citrix XenServers using Live Migration. Disk IO paused for a few seconds while the transition occurred, but the endpoint never lost connectivity and Iometer continued to run without error.

Finally, ESG Lab simulated a site failure by downing the Citrix XenDesktop server at Site 1. All VMs automatically transitioned to the other running XenDesktop servers in the cluster. Connectivity was lost to the running virtual desktop, but our simulated user was able to immediately log back in and bring up their desktop.

ESG Lab Validation Highlights

• ESG Lab found the HP P4000 SAN easy to configure, implement, and manage in combination with the Citrix XenDesktop environment.
• ESG Lab was able to use one ‘gold image’ virtual desktop to create and present multiple unique virtual desktops with minimal capacity overhead and no impact to users.
• The efficiency and cost effective scalability of the HP architecture was seen to meet the performance needs of real-world applications deployed in a distributed virtual desktop environment.
• The HP P4000 SAN was able to sustain continuous access for a Citrix XenDesktop user through disk, network, node, and site failures and a virtual desktop was able to automatically transition to a running XenDesktop server after a simulated server failure.

Issues to Consider

• While ESG tested back-end storage performance in a virtual desktop environment, other factors, including the CPU and memory configuration of the infrastructure servers and virtual machines, will have a much greater impact on the end-user experience. ESG Lab recommends that end-users leverage reference configurations and work with their virtual infrastructure vendor to determine the best practices and optimal configuration for each environment.
• While leveraging the P4000’s Snapshot and SmartClone technology for virtual desktop deployment and management is compelling and powerful, the process of making the cloned desktops unique after importing into XenDesktop was manual when ESG Lab first tested for this report. Subsequent demonstrations with HP confirmed that Citrix Essentials for XenServer now includes StorageLink Connect API integration with HP SANs to automate the process and make it more user-friendly. Leveraging StorageLink technologies, Essentials for XenServer integrates storage management functions into the management console for the virtual infrastructure via wizards enabling users to utilize the advanced services native within HP storage arrays.

ESG Lab’s View

Increasing numbers of clients and applications make desktop management a daunting task for IT. The number of applications supported increases with organization size, compounding desktop management challenges for large organizations. With increasing numbers of corporate applications to support, ongoing maintenance and management tasks directly translate into considerable IT staffing requirements and costs. Like server virtualization, desktop virtualization is establishing a foothold in the data center among IT staffs looking to optimize their current PC environments.

HP’s P4000 SAN has a highly scalable, clustered architecture that simplifies management and allows customers to start at the level of capacity and performance they require today and grow their environments on demand. Additionally, it is easy to use and manage, while providing advanced features such as Network RAID, Smart Clones, remote replication, and thin provisioning.

Customers can stretch their clusters to create multi-site SANs. We have seen storage systems that scale in this fashion with NAS and CAS products, but in our opinion, HP is one of the leaders in SAN attached true N-way clustered storage. ESG has long been a proponent of scalable clustered storage and we believe it will become the dominant approach due to the compelling value it brings.

ESG Lab was very impressed with HP’s performance, as it scaled nearly linearly with a challenging small-block random IO workload and latency actually decreased as the cluster and IO load grew. ESG Lab found that HP performed well in a virtual desktop environment, providing easy provisioning and powerful integration of Snapshot and SmartClone technology to optimize capacity utilization. High availability functionality was also impressive, sustaining multiple failures while providing continuous access to attached virtual desktop users.

HP’s storage systems powered by HP SAN/iQ Software delivered an easy-to-use, flexible, scalable, highly available, and highly efficient storage solution for Citrix XenDesktop customers. Matching in storage what Citrix provides for desktops, HP SAN/iQ Software supports volume cloning for creating large numbers of virtual desktops without the delay and cost of consuming actual storage for each clone. And because it is distributed by design, creating a disaster-resilient storage infrastructure is as easy as choosing which storage modules to configure in each separate location.

HP’s close collaboration and extensive testing with Citrix ensures that the HP XenDesktop reference architectures are jointly tested, certified, and tuned to deliver optimal availability and performance, while addressing desktop management challenges in the enterprise. Through hands-on testing, ESG Lab confirmed that HP provides a robust networked storage foundation with simple configuration, powerful desktop mobility, enterprise class availability, and near-linear scalability. The HP SAN enhances the intelligence and value of Virtual Desktop Infrastructure.

Appendix

[2] www.hp.com/solutions/activeanswers/xenserver

[3] Configuration details are listed in the Appendix.

[4] Source: ESG Research Report, Virtual Desktop Infrastructure Market Trends, February 2009.

[5] Test configuration and workload details can be found in the Appendix.

[6] Source: ESG Research Report, Virtual Desktop Infrastructure Market Trends, February 2009.

[7] Source: ESG Research Report, Virtual Desktop Infrastructure Market Trends, February 2009.

Background

ESG recently asked IT professionals to name the business initiatives which would have the greatest impact on IT spending decisions over the next 12-18 months. The number one response was cost reduction and simplification of business processes came in second. [1] Given the economic uncertainties of recent years, it’s no surprise that IT managers are looking to reduce cost and complexity.

Where are IT managers focusing their efforts toward this goal of reducing cost and complexity? As shown in Figure 1, server virtualization is at the top of the list. As a growing number of organizations use server virtualization to reduce the cost and complexity of a consolidated IT infrastructure, improving backup and recovery, and finding ways to cost-effectively manage data growth have become top priorities as well.

Nearly 50% of the IT managers within midmarket companies reported they have more than 10 TB of data that needs to be stored and protected.[2] And more than half of the respondents reported that data is growing at a rate of 20% or more per year.  For those organizations, data volume is doubling every two to five years. With these challenges in mind, a growing number of IT managers are turning to powerful new technologies which can drastically reduce the cost and complexity of storing and managing data.  The balance of this report examines an HP solution which uses innovative technologies to addresses several of the top issues noted in Figure 1 (increasing server virtualization, improving backup and recovery, managing data growth, and data center consolidation).

HP StorageWorks-enabled Converged Infrastructure for Mid-sized Organizations

The HP StorageWorks-enabled solution tested by ESG Lab is shown in Figure 2.  This solution can be used to cost-effectively consolidate server and storage resources using powerful new technologies including server virtualization, iSCSI, clustered storage, backup to disk, data deduplication, and WAN-optimized replication.

The key components of this end-to-end converged infrastructure are as follows:

• HP ProLiant DL380: The field-proven ProLiant family of HP servers provided the flexibility, performance, and enterprise-class uptime for a consolidated, virtual server environment.
• VMware vSphere: VMware vSphere software running on HP ProLiant servers was used to transform the IT infrastructure into a private cloud which enabled the automated delivery of IT infrastructure as a service.
• HP Data Protector: This data protection software package was used for simple and reliable backup and recovery operations in a virtual server environment.
• HP StorageWorks P4500: With a clustered storage architecture that leverages industry standard server and Ethernet technologies, the HP P4500 SAN was used as an iSCSI-attached primary storage solution that delivered cost-effective scalability, performance, and high availability.
• HP StorageWorks D2D2503: This disk-based backup system used data deduplication technology to reduce backup windows, provide fast and reliable restores, and reduce retained disk capacity up to 50 times.
• HP StorageWorks D2D4112: Working in concert with the StorageWorks D2D2503, the D2D4112 was used to store a WAN-optimized copy of backups at a second site for disaster recovery.
• HP StorageWorks MSL4048: This automated tape library, configured with a pair of SAS-attached LTO3 tape drives, was used to create portable copies of a subset of the backup data at the second site.

There are a number of benefits that can be achieved with this solution from HP. First and most obvious, is the fact that the solution is sold and supported by a single vendor.  A bit less obvious is the fact that all of the hardware, with the exception of the tape library, is built with industry standard HP ProLiant servers.  This reduces the cost of the solution as it avoids the use of proprietary controllers and disk enclosures.

As of the publication of this report, HP has refreshed the entire D2D product family and introduced a new high-end member, the D2D 4312. The D2D 4312 has more processing power and offers higher capacity than previous generation D2D models.  The new D2D product family is also running a new 64-bit data deduplication technology called HP StoreOnce.   HP’s vision for StoreOnce deduplication technology is that it will be ported to several HP platforms—including HP Data Protector. While ESG tested the previous generation D2D appliances, the scenarios depicted and conclusions drawn in this report still apply.

ESG Lab Validation

ESG Lab performed hands-on testing of a StorageWorks-enabled converged infrastructure at an HP facility in Fort Collins, Colorado. Testing was designed to demonstrate how virtual server, clustered storage, iSCSI, backup to disk, and deduplication technologies can be used to create a simple, cost-effective solution that is ideally suited for a typical mid-sized organization.

The ESG Lab Test Bed

A two-site solution was tested with an HP ProLiant DL380 server running VMware vSphere software in Site A. Virtual machines running Microsoft Windows Server 2008 as a guest operating system were used to simulate a number of typical applications (e.g., Microsoft Exchange).  HP Data Protector software was installed on one of the VMware vSphere-enabled virtual machines.

An iSCSI-attached HP StorageWorks P4500 was used to meet the primary storage needs of virtualized applications running on the HP ProLiant server.  HP Data Protector was used for backup and recovery of deduplicated data residing on an iSCSI-attached HP StorageWorks D2D2503 appliance in Site A.  Deduplicated backup data was replicated over a simulated WAN to a second D2D appliance in Site B. A SAS-attached HP StorageWorks MSL4048 autoloader with two LTO3 tape drives was used to create portable backup images at Site B.

Getting Started

ESG Lab testing began with a tour of a pre-wired test bed.  The VMware vCenter screen shot shown in Figure 4 shows how the primary and secondary sites were configured with the following virtual machines:  Microsoft Active Directory, Apache Web Server, Microsoft Exchange Server, Microsoft SharePoint Server, and Microsoft SQL Server.

HP StorageWorks P4500 (Virtualization SAN Solution)

The HP P4000 SAN is an iSCSI-attached primary storage solution that uses intelligent software running on a cluster of industry standard servers to provide cost-effective scalability, performance, and high availability.

ESG Lab Testing

ESG Lab examined the configuration of a two-node HP StorageWorks P4500 serving the storage needs of virtual machines on the iSCSI-attached HP ProLiant server.  A third P4500 node was added online to the existing cluster.  A new volume was configured and presented to a vSphere-enabled guest machine.

The wizard-driven user interface shown in Figure 5 was used to add the third node to the existing two-node cluster. Note that the intuitive navigation tree towards the left shows the two nodes in the cluster before the upgrade (hph32 and hph33).

Five mouse clicks and one minute later, a third node had been discovered and added to the cluster. SAN/iQ software began a low priority background operation which automatically redistributed existing capacity on the recently expanded cluster. There was no impact to a copy operation running on the existing cluster.

The wizard panel shown in Figure 6  was used to create a new volume for Microsoft Exchange log data. Note that a single mouse click was used to enable thin provisioning. Thin provisioning delivers capacity on demand, which increases efficiency and decreases costs.

Seven mouse clicks and one minute after getting started, the volume was configured and ready for use.  The Microsoft iSCSI initiator was used to discover and connect to the new volume from a Windows 2008 server running within a VMware-enabled virtual machine. The newly created volume was discovered and NTFS quick-formatted as an L: drive using the Microsoft disk administrator utility as shown in Figure 7.

In previous reports, ESG Lab has tested the valuable storage software capabilities built into a P4000 SAN at no additional charge, including thin provisioning, snapshots, remote mirroring, high availability, and extreme performance scalability (including a Microsoft-approved test bed supporting 152,000 e-mail users).[3]

HP Data Protector and HP StorageWorks D2D Deduplication (D2D)

HP Data Protector software and data deduplication algorithms running within an HP StorageWorks D2D appliance reduces the capacity needed to store backups that have already been written to disk.  As shown in Figure 8, Data Protector software and D2D hardware work together to create a cost-effective disk-to-disk backup solution.   Combining this solution with HP StorageWorks MSL tape library to create end-to-end data protection with a number of key advantages for small businesses:

• Backups and restores are faster and more reliable with disk.
• Data deduplication reduces the capacity required to store backups on disk.
• More backups can be retained on disk for quick and reliable restores.
• Replication to a central location minimizes complexity as it decreases risk.
• Centralized tape archival works with existing processes as it enables cost effective long term archival.

One of the key advantages of HP Data Protector software is its pricing model, in which license price is based on data stored as opposed to data protected.   As a result, D2D deduplication not only enables fast and reliable restores from disk, it also reduces licensing costs.

HP Data Protector software running on a backup server can be attached to a D2D Backup System through an Ethernet or a Fibre Channel network.  D2D capacity can be accessed as a network-attached drive (NAS) or a virtual tape library (VTL).

ESG Lab tested a D2D2503 as an iSCSI-attached VTL.   This configuration is ideally suited for small to medium-sized businesses as it reduces the cost of backing up to disk in a number of ways:

• Data deduplication reduces disk capacity requirements up to 50 times.
• iSCSI eliminates the cost and complexity of Fibre Channel host bus adapters and switches.
• The D2D Backup System performs like multiple tape drives operating in parallel.  A single tape drive can only perform one backup at a time.  To run more than one, more tape drives must be added and run in parallel.  A D2D Backup System can run many backup jobs simultaneously.

While ESG Lab tested HP backup software and hardware, it should be noted that HP Data Protector software can be used with a wide variety of backup hardware from HP and other vendors.  Similarly, HP D2D Backup Systems can be used with most popular backup applications.

ESG Lab Testing

ESG Lab used Data Protector software to back up the contents of the program files directory on the C: drive within a virtual machine running Windows Server 2008 as a guest operating system. Data Protector running within a virtual machine was configured to use the iSCSI-attached D2D2503 for disk-based backup and restore operations. The D2D Backup System eliminated duplicate data as backups were ingested and written to disk (a.k.a., inline deduplication).  The StorageWorks management console shown in Figure 9 was used to examine the configuration of the D2D Backup System.  The management console was also used to monitor the capacity savings that can be achieved with deduplication. In this screenshot, which was taken after the first full backup of the program file directory, an initial deduplication ratio of 2.8 was achieved as 11 GB of backup data consumed only 4 GB of disk capacity.

The space savings that can be achieved over a month of daily incremental and weekly full backups was documented in a previously published ESG Lab Validation report. [4] As shown in Figure 10, disk capacity was reduced by 92%.

WAN-optimized D2D-Series Replication and Remote Site Tape Archival (D2D2T)

HP backup and tape systems can be used to deploy a WAN-optimized disk-to-disk-to-tape (D2D2T) data protection strategy as shown in Figure 11.  Deduplication not only reduces disk capacity requirements in Site A, but also reduces the WAN bandwidth needed to replicate backup data to Site B.  With more backups retained safely on disk at Site B, disk-based retention periods can be increased and the amount of tape needed for off-site archival can be decreased.

While ESG Lab tested a two-site solution, it should be noted that many D2D Backup Systems deployed in remote offices can be configured for WAN-optimized replication to a central site.  As documented in a previous lab report,[5] a many to one deployment strategy can be used to eliminate the cost, complexity, and hassle of running backups in remote offices as it provides a centralized platform for tape archival.

ESG Lab Testing

ESG Lab configured and observed the  replication of deduplicated backup data from Site A to Site B and a copy to tape in Site B. Restores were tested using backup data storage on the D2D2503 in Site A, the D2D4112 in Site B, and the MSL4048 tape autoloader  in Site B.

The D2D management console was used to configure and monitor replication between Site A and Site B.  The screenshot shown in Figure 12 shows the status of a D2D replication job during a previous ESG Lab Validation. [6] In this example, WAN bandwidth requirements were reduced by 90% as 53 GB of Microsoft application data was replicated over an unrestricted Gigabit Ethernet connection in 10 minutes and 40 seconds.  Once the first full backup was completely replicated to the target D2D appliance, ESG Lab used a “Network Nightmare” WAN simulator to restrict WAN bandwidth to 2 megabits per second.  The first incremental backup was automatically replicated over the 2 Mbit/sec simulated WAN connection. Replication of the incremental backup transferred 5 GB of deduplicated data in 1 hour, 17 minutes, and 49 seconds.  As a result, D2D deduplication increased the effective bandwidth of the WAN connection from 2 Mbit/sec to 22.9 Mbit/sec.

The object copy capabilities of HP Data Protector software were used to archive a full backup image to a removable tape cartridge in a SAS-attached HP StorageWorks MSL tape autoloader in Site B.  The Command View MSL console shown in Figure 13 was used to monitor the status of two LTO3 tape drives in the MSL4048.

Data Protector was used to recover a single file from the D2D Backup System in Site A.  Browsing the backup history and finding the file to restore was intuitive and easy.   Restoring from disk using the D2D virtual tape interface felt the same as a restore from a real tape library, but completed quicker due to the random access of a disk-based backup.

Restores from the D2D Backup System at the remote site in Site B were tested next.  A Windows iSCSI initiator rescan and login were used to present the remote D2D appliance to the destination Data Protector cell.  After the Data Protector cell had discovered the remote virtual tape device, the restore operation felt the same as the restore from Site B.

The server and D2D Backup System in Site A was powered down to simulate a disaster. A recovery from tape in Site B began with a login to the active directory virtual machine at the remote site.  Data Protector was used to perform a barcode scan and an import of backup data on tape. A restore from tape of the program file directory completed without error in just under 10 minutes.

Cost-Efficient

Organizations of all sizes are struggling to meet the conflicting challenges associated with macro-level global financial uncertainty and micro-level information storage growth and complexity.  A growing number of IT managers are turning to virtualization and consolidation technologies to meet these challenges. With a focus on scalability, automated management, and capacity-efficient pricing, the HP StorageWorks-enabled solution tested by ESG Lab is an excellent example of an end-to-end solution that is purpose-built to address these issues.

ESG Lab created a total cost of ownership (TCO) model which compares the cost of a traditional physical server environment protected with tape to a virtual server environment protected with an HP StorageWorks-enabled D2D2T data protection strategy. Costs were broken down into categories including capital expenditures, administrative costs, tape costs, maintenance costs, power and cooling costs, and total floor space costs. The five year cost of ownership was compared.  A number of assumptions were made based on the type of equipment, WAN connectivity, backup and restore policies, and capacity and performance requirements a typical mid-sized organization might have in place. [7] The results are shown in Figure 14.

What the Numbers Mean

• The total cost of ownership of traditional physical servers protected with tape is 23% higher than the HP Converged Infrastructure.
• The bulk of the savings were realized with an iSCSI-attached clustered storage system (HP StorageWorks P4500), backup to disk with data deduplication (HP StorageWorks D2D Backup Systems), and the HP Data Protector licensing model which is based on capacity stored as opposed to data protected.

ESG Lab Validation Highlights

• VMware vSphere was used to simulate a consolidated mix of common business applications running on an HP ProLiant DL380 server (Microsoft Active Directory, Apache Web Server, Microsoft Exchange Server, Microsoft SharePoint Server, and Microsoft SQL Server).
• An HP P4500 SAS Virtualization SAN solution was attached via iSCSI to the HP ProLiant server. In less than ten minutes, the two-node storage cluster was upgraded online to three nodes and a new volume was created and accessed by a virtual machine.
• HP Data Protector software and a pair of HP StorageWorks D2D Backup Systems were used to provide deduplicated backup-to-disk in one site and WAN-optimized replication of backup data to a second site.
• HP Data Protector software was used to make copies of backup data on portable LTO3 tape cartridges within an HP StorageWorks MSL4048 autoloader.
• The HP StorageWorks converged infrastructure was used to test backup and restore operations within a D2D2T data protection strategy.
• A five year TCO analysis indicates that an HP StorageWorks enabled converged infrastructure is 23% more cost-effective than a traditional physical server infrastructure attached to a legacy modular FC disk array and a tape-only data protection strategy.

Issues to Consider

• ESG Lab tested an end-to-end HP solution in a controlled lab setting.   Due to the many variables in each production data center environment, capacity planning and application-level testing are recommended to ensure that the solution meets the performance and recovery objectives for your organization.
• While the HP DL380, P4500, D2D, and MSL systems all have easy-to-use management interfaces, a single “manager of managers” that provides an overall view of an entire environment would be of great value to administrators.
• While ESG is confident that one or more HP StorageWorks D2D Backup Systems can be used to meet the performance needs of a mid-sized organization, D2D systems with more capacity and horsepower could reduce cost and complexity within larger mid-sized organizations.  HP has advised ESG that the new line of D2D Backup Systems, released in June 2010, has been designed with these considerations in mind.

The Bigger Truth

As the global economy provides a constant reminder that costs have to be contained, a number of factors must be considered as IT managers examine the benefits of emerging technologies; specifically, data growth, consolidation initiatives, server virtualization, new application deployments, compliance mandates, and business requirements that necessitate more stringent SLAs make it difficult for IT organizations to keep up with growing demands.  IT managers within mid-sized organizations are especially challenged as they look for ways to consolidate, optimize, and automate the delivery of IT services.

Data protection is an integral part of IT operations—maintaining one or more local and/or remote copies of primary data provides an insurance policy for minimizing data loss and downtime. Data growth is wreaking havoc on data protection, making it difficult for IT organizations to keep pace with the capacity of data to protect, to meet backup windows and recovery objectives, and to manage spending.

Backup modernization has been underway over the last few years, with disk-based backup and recovery as a top initiative.  ESG’s research shows that organizations are more often leveraging disk and the increased adoption of deduplication is a contributing factor to the rise in the use of disk in backup.  Regardless, organizations are not releasing their grip on tape.  Tape remains an integral part of data protection strategies, especially as the go-to media of choice for long-term retention.

ESG Lab has validated that HP has created a converged infrastructure for mid-sized organizations.  ESG Lab tested HP ProLiant servers with HP Data Protector software running within VMware-enabled virtual machines.  An iSCSI-attached HP P4000 SAN that uses intelligent software running on a cluster of industry standard servers was used to create a primary storage solution that’s scalable, fast, and extremely easy to manage.  The data deduplication that’s built into HP StorageWorks D2D Backup Systems reduced disk capacity and WAN bandwidth requirements as a multi-site D2D2T strategy was implemented.

The deduplication technology at the core of a disk-based StorageWorks D2D Backup System was developed by HP Labs researchers working with engineers in the HP StorageWorks division. This valuable technology can be deployed in concert with HP Data Protector software and an HP StorageWorks MSL tape autoloader  to create an end-to-end data protection solution that is ideally suited for mid-sized organizations.

The bigger truth of this ESG Lab validation can be summed up in two words: trust and affordability. HP has been a trusted supplier of IT infrastructure for decades.  The solution tested by ESG Lab is a converged infrastructure with all of the hardware and all of the storage software supplied by HP.  An end-to-end solution from a single vendor that has earned your trust avoids the complexity and risk of building a solution on your own. A five-year cost analysis indicates that the solution presented in this report reduces costs by 23% over five years compared to a legacy solution that relies on physical servers attached to a Fibre Channel disk array and backed up with a traditional tape library. ESG Lab believes that IT managers in mid-sized organizations would be wise to consider the affordability of converged infrastructure powered by HP StorageWorks technology.

Appendix

[2] ESG Research Report, 2010 Data Protection Trends, April 2010.

[3] Source: ESG Lab Review, HP P4000 SAN: Affordable, Scalable, Reliable Storage, March 2010.

[4] ESG Lab Validation Report, HP Data Protector and Deduplication Solutions, June 2010.

[5] ESG Lab Validation Report, HP Data Protector and Deduplication Solutions, June 2010.

[6] Ibid.

[7] TCO assumptions are documented in the Appendix.

Background

While deduplication can reduce the cost of the raw storage required to store and replicate backup data on disk, integration with the existing ecosystem is crucial. As shown in Figure 1, recently completed ESG research indicates that ease of implementation, performance impacts, and integration with existing backup processes are key concerns.[1] Robust management, edge to core replication, tape integration, and deduplication options are important considerations as well, especially within large enterprise-class organizations. The diverse family of Hewlett Packard data protection solutions is ideally suited to address these, and other, concerns.

Deduplication

Choosing a deduplication strategy should include a discussion on how and where deduplication should occur. There are two basic locations deduplication can occur: at the source, typically accomplished via a software agent runing on the client machine, or at the target, which involves either writing directly to the device  or running a software agent on the media server to perform deduplication.  All deduplication includes some level of overhead. If it occurs at the source, that overhead occurs on the client machine or media server and may have an impact on backup performance due to the software required on client systems, which can consume processing, storage, and/or network resources to deduplicate data.  When deduplication is perfomed at the target, the overhead is incurred in the device where data is being written.

In addition, it is important to differentiate between a pure software approach — in which the software  runs on an industry standard platform, — and an appliance-based (or storage hardware-based) solution. With a  software solution users have flexibility in their choice of physical storage, but must ensure that the system has enough I/O bandwidth to support their performance needs, sufficient system resources to support desired de-duplication rates, and the right storage security to prevent potential data loss. An appliance-based  solution has the ability to address many of these concerns, but locks users into a particular hardware platform.

HP’s D2D and VLS are target-side deduplication appliances which are designed to provide cost-effective, easy-to-deploy deduplication and are tuned for optimal performance on each hardware platform.

HP Data Protection Solutions

Hewlett Packard offers diverse data protection hardware and software to address the concerns of enterprises from the core data center to the smallest remote office. Figure 2 shows the HP family of data protection solutions as they might be deployed in a typical distributed enterprise.

HP StorageWorks VLS virtual tape libraries provide a high performance primary backup target with optional deduplication in the FC SAN enterprise data center while HP StorageWorks D2D serves as an easy to manage data protection appliance in mid market data centers and remote offices with deduplication and WAN-efficient replication across sites. HP Data Protector software provides a solution to manage the D2D and the VLS not only in sngle-site backup environments but also in replicated solutions. While this report focuses on the benefits of a total HP solution, it’s important to note that HP products integrate well with third party products. Data Protector works with any inline de-duplication solution and the D2D and VLS can be used with third party backup software. However the end to end, single vendor solution including replication enablement and integration is a unique HP offering.

Benefits of HP StorageWorks Data Protection Solutions

VLS:

• Highly scalable capacity and performance: Up to 4800 MB/sec of throughput and 1280 TB of usable storage.
• The entire capacity of the system can be presented as a single virtual library target: The system can scale without time consuming reconfiguration and rebalancing of backup software and backup jobs.
• Accelerated deduplication: Fast hardware compression, in combination with post process deduplication, provides capacity efficiency without impact to backup windows.
• WAN efficient replication: Fast, cost-effective disaster recovery capability between data centers and remote sites.

D2D:

• Distributed capacity and performance: The D2D Backup Systems offer high performance multi-streaming backup speeds of up to 720 GB/hour.
• HP Dynamic deduplication: Enables longer term data retention on disk and WAN efficient replication.
• WAN efficient replication: Fast, cost-effective disaster recovery capability between multiple remote sites.
• Ease of use and deployment: HP’s D2D Backup Systems are designed for easy installation and deployment for mid-sized business environments—it is ready to deploy right out of the box.

As of the publication of this report, HP has refreshed the entire D2D product family and introduced a new high-end model, the D2D 4312. The D2D 4312 has more processing power and offers higher capacity than previous generation D2D models.  The new D2D product family runs a new 64bit data deduplication technology called HP StoreOnce.   HP’s vision is to port StoreOnce deduplication technology to several HP platforms – including HP Data Protector.   While ESG tested the previous generation D2D appliances, the scenarios depicted and conclusions drawn in this report still apply.

Data Protector:

• Advanced Backup to Disk – 24/7 information access and quick disaster recovery.
• Multiple Recovery Point/Recovery Time Objectives – Achieve business-driven recovery objectives.
• Manage data effectively – within existing budgets and infrastructure, even as the quantity of data grows.
• Centralized Data Protection – Protect data on distributed physical and virtual infrastructures.
• Broad Interoperability – Integrates with partner solutions including NetApp, Data Domain, IBM ProtecTIER, and supports any third party inline deduplication target appliance.

ESG Lab Validation

ESG Lab performed hands-on evaluation and testing of HP’s data protection solutions at an HP facility in Fort Collins, CO. Testing was designed to demonstrate the scalability, performance, and ease of management  of HP’s solutions from the point of view of a typical enterprise systems administrator integrating HP’s disk-based solutions into an existing tape environment.

Ease of Deployment and Integration

The test environment, shown in Figure 3, was used throughout testing. Testing began with a one-node HP StorageWorks VLS9000 array with deduplication, physically installed and powered up as it would be by HP professional services for an enterprise customer, in a data center environment with HP Data Protector software installed. Other elements typical of an enterprise environment, such as physical tape libraries and  HP StorageWorks D2D appliances, were also present in the test bed.[2]

ESG Lab Testing

ESG Lab initially logged in to the Command View VLS console by pointing Internet Explorer at the administrative IP address on the VLS9000 library and entering the administrator username and password. Next, the Create Virtual Library Wizard was launched. The Create Virtual Library Wizard walks the administrator through the steps to create a virtual tape library, asking for such details as library type,  and is shown in Figure 4.

ESG Lab selected an HP ESL E-Series to match the physical library type already installed in the test environment. Next, and more important, the virtual tape drive type and quantity, as well as the capacity and quantity of virtual tape cartridges, were set, as seen in Figure 5.

In less than two minutes, the Create Virtual Library Wizard was completed and the virtual tape library was configured and presented on the SAN.  Next, ESG Lab started the HP Data Protector Autoconfigure Wizard. The HP Data Protector Autoconfigure Wizard, as seen in Figure 6, discovers new backup target devices and prepares Data Protector to use them.

The Data Protector Autoconfigure Wizard took about two minutes to discover and add the Virtual Tape Library and its virtual tape drives to Data Protector. As Figure 7 shows, within five minutes of sitting down at the keyboard, ESG Lab was running a full backup of the first server to the HP VLS9000.

Scalability and Performance

One of the fundamental advantages of VTL backup is the ability to run many backup streams concurrently using multiple virtual tape drives. A single tape drive can only perform one backup at a time.  To get more than one backup job running at the same time, more tape drives must be added and run in parallel. A disk-based backup and recovery solution with many random access disk drives emulating many virtual tape drives can run many backup jobs simultaneously.  The random access nature of disk also provides improved performance when locating individual files to be restored.

ESG Lab Testing

ESG Lab performed backups using a single node VLS9000 as a target first and then repeated the test after upgrading the VLS to two nodes in order to examine its relative performance as storage capacity is scaled.  A full backup of mulitple servers was simulated using the HP tapeperf utility set to generate data with 2:1 compressibility.[4] The first iteration ran with two servers running 10 backup streams; the second iteration was performed with three servers running sixteen streams. Performance scaled linearly when the second node was added. The screen capture in Figure 8 shows the VLS Command View Console during the two node test, running at 1,164 MB/sec.

Figure 9 shows actual performance results obtained in ESG Lab testing projected out to a fully populated, eight-node system. Detailed results for each test run are shown in Table 1.

What the Numbers Mean

• When a node was added to the VLS, performance scaled nearly linearly with no degradation.
• An eight-node system should be able to acheive 4,656 MB/sec of sustained backup throughput, representing the ability to protect nearly 128 TB of data in an eight-hour backup window.

Remote Office Protection

The family of HP disk and tape data protection solutions, combined with HP Data Protector software, can be configured to create an automated, edge-to-core data protection topology (see Figure 10) that spans multiple sites and provides disk-to-disk-to-tape (D2D2T) functionality. HP StorageWorks D2D appliances provide WAN-efficient movement of data between sites and storage tiers while Data Protector provides the single point of management and catalog for backup data—regardless of where it resides (remote office or corporate data center), what type of media it is stored on (disk or tape), or its age (recent backup or long term archive). D2D disk-based backup and replication appliances support data deduplication to reduce the resources required to store backup images on disk and replicate backup images over a WAN.

ESG Lab Testing

ESG Lab used HP Data Protector software to configure, automate, and track the migration of backup data residing on HP D2D data protection appliances, as shown previously in Figure 3. An edge-to-core D2D2T data protection strategy was implemented using an HP D2D appliance located in a simulated remote office. Remote office backup data was replicated over a simulated WAN to an HP D2D in a corporate data center with data movement carried out by the D2D systems. The object copy capabilities of the Data Protector software were used to write a copy of the data to removable media in a Fibre Channel SAN-attached HP StorageWorks MSL tape library. First, ESG Lab logged in to the D2D web management console, shown in Figure 11.

The first backup was ‘seeded’ or replicated locally over a gigabit ethernet LAN connection. Seeding is often employed by users who have large data sets at remote offices as it allows the first bulk transfer of data to complete very quickly. The D2D device is then shipped to the central data center and from then on, updates require much less bandwidth thanks to deduplication.  The first bulk replication is illustrated in Figure 12.

Replication of the first backup transferred 53 GB of data in 10 minutes and 40 seconds over an unrestricted Gigabit Ethernet connection. Once the first full backup was completely replicated to the target D2D appliance, ESG Lab inserted the ‘Network Nightmare’ WAN simulator and restricted throughput to 2 megabits per second to simulate a nearby WAN connection between the remote office and data center.

The first incremental backup was performed and automatically replicated over the 2 Mbit/sec simulated WAN connection. Replication of the incremental backup transferred 5 GB of deduplicated data in 1 hour, 17 minutes and 49 seconds.  This means the D2D transferred at the equivalent of 11 Mbit/sec over the 2 Mbit/sec WAN connection. By comparison, the second full backup resulted in the equivalent of 22.9 Mbit/sec over the same link. The higher virtual throughput is due to the greater volume of duplicate data in a full backup.

Figure 14 shows actual and projected deduplication capacity savings over 30 days of backups on a weekly full, daily incremental backup schedule. The capacity savings over 30 days was projected at 92%—an 11.94:1 deduplication ratio.

Finally, ESG Lab copied the latest full backup to tape the copy to tape capabilities of Data Protector with the D2D, which can be used for archiving. D2D frees up space by expiring source media (virtual tapes) rather than deleting the source data (files). This ensures that data that exists in multiple backups is not deleted until all references to it are deleted.

In 20 minutes, the copy to tape was complete.

Cost-Efficient Protection

Organizations of all sizes are struggling to meet the conflicting challenges associated with macro-level global financial uncertainty and micro-level information storage growth and complexity. A growing number of IT managers are turning to virtualization and consolidation technologies to meet these challenges. With a focus on scalability, automated management using rich software tools, and capacity-efficient pricing, HP’s data protection solutions are an excellent example of solutions that are purpose-built to address these issues.

ESG Lab created a total cost of ownership (TCO) model based on a hypothetical backup environment with multiple remote offices, a major data center, and remote replication for disaster protection.  The scenario examines cost-savings associated with moving from a tape-based to a VLS and D2D-based backup and recovery strategy, although it still assumes use of tape for long-term archive of backups. Costs were broken down by category:  capital expenditures, administrative costs, tape costs, maintenance costs, power and cooling costs, and total floorspace costs.  The cumulative costs for both tape- and disk-based backups were calculated annually over a five year period.  A number of assumptions were made and included in the calculations based on what a current IT organization might have in place for equipment, WAN connectivity, backup and restore policies, and capacity and performance requirements.[6] Comparisons were made between the total cost of ownership of a traditional tape infrastructure with no replication or deduplication and an HP Data Protection environment with disk-based backup, deduplication, and replication.

Figure 17 shows the total cost a hypothetical end-user would incur over five years when comparing a traditional tape environment to a backup to disk environment with replication and deduplication. The inflection point, where the disk environment becomes less costly than the tape environment, occurs just before the end of year two.

What the Numbers Mean

• The total cost of ownership of tape alone is roughly 69% higher than HP disk-based data protection with deduplication and replication.
• Data Protector software provides significant savings due to licensing based on data stored as opposed to data protected. Deduplication reduces licensing costs.
• The tape solution is more expensive in part due to the cost of acquiring tape media and the added complexity of managing the distributed tape infrastructure.
• While eliminating daily off-siting of tapes represents significant savings, tape is still the most cost-efficient method for long term archive of backups, and most organizations will replicate deduplicated data to a remote site for copy to tape for this purpose.

ESG Lab Validation Highlights

• Within five minutes of sitting down at the keyboard, ESG Lab was running a full backup of the first server to the HP VLS9000.
• The VLS 9000 demonstrated near linear performance scalability, achieving an impressive 4 TB/hour with a two-node configuration.
• The HP D2D Backup System achieved 81% bandwidth efficiency, transferring a 5 GB incremental backup across a 2 Mbit/sec simulated WAN link in just under 1 hour and 18 minutes for an effective throughput of more than 11 Mbit/sec. Replicating a second full backup yielded an even more impressive 90% bandwidth efficiency, transferring a 54 GB incremental backup across the same 2 Mbit/sec simulated WAN link in just over 6 hours and 42 minutes, for an effective throughput of more than 22.9 Mbit/sec.
• The HP Data Protection solution suite demonstrated faster, more reliable backups and restores at a significantly lower total cost of ownership than a tape environment.

Issues to Consider

• As with all VTLs today, when a cartridge is deleted or expired in a backup application, space on the VLS is not reclaimed until the cartridge is deleted, expired, or overwritten via the VLS management application. Integration with backup applications to automatically trigger a delete or expire in the VLS when a cartridge is expired in the backup application would be a useful enhancement.
• While the VLS, D2D, and Data Protector all have easy to use management interfaces, a single “manager of managers” that integrated all three products and provided an overall view of an entire enterprise environment would be of great value to administrators.
• While ESG is confident that one or more HP StorageWorks D2D Backup Systems can be used to meet the performance needs of a mid-sized organization,  D2D systems with more capacity and horsepower could reduce cost and complexity within larger mid-sized organizations.  HP has advised ESG that the new line of D2D Backup Systems, announced in June 2010, has been designed with these considerations in mind.

The Bigger Truth

ESG Lab conducted its first hands-on testing of the Hewlett Packard’s enterprise VTL, the VLS 6000, in 2006 and then validated the VLS 9000 in 2008. Testing and discussions with end-users have confirmed that HP’s disk-based backup solutions fit seamlessly into existing backup environments while providing dramatic performance and capacity reduction benefits compared to legacy tape-based methods. The HP StorageWorks D2D, aimed at delivering deduplication and WAN efficient replication to smaller, remote offices, completes a comprehensive, enterprise wide edge-to-core data protection architecture, managed by HP Data Protector, that goes beyond disk-based backup.

During this independent lab validation, ESG Lab confirmed HP’s edge to core capabilities in support of large enterprises as well as deduplication support across disk, replication, and tape throughout the product line. HP’s comprehensive offering with capacity efficient pricing provides high performance data deduplication technology to deliver dramatic disk capacity savings while offering scalable, predictable performance.

A modest two-node VLS configuration tested by ESG Lab was able to back up at a sustained 4 TB/hour.  Based on the nearly linear scalability observed by ESG,  an eight-node system should be able to protect data at 16 TB/hour. Easy to navigate, web-based management enabled ESG Lab to manage backups for a remote office from creation, through replication, and finally to tape—all using HP Data Protector software—giving rise to an edge-to-core data protection strategy covering remote and branch offices as well as multiple data centers. Direct attach to tape capability enables enterprises to meet offsite and deep archive requirements using familiar tools and techniques while keeping tape copy traffic off the SAN.

HP Data Protector software exemplifies the company’s depth and breadth of end-to-end solutions for backup and recovery encompassing disk and tape. It should bring significant value to customers grappling with the challenges associated with cost-effective management of their data protection resources.

In essence, deduplication has become a crucial component of disk to disk backups, but when considering competing methods for implementation, customers should consider the tradeoffs and what’s best for their organization: ease of implementation, cost, and bandwidth all play an important role.

ESG Lab believes that the combination of enterprise class performance and scalability—along with comprehensive storage management software and services—provides a unique approach for optimizing data protection and recovery strategy in the enterprise. Hewlett Packard customers can now retain more data for fast and reliable restores and longer retention periods while minimizing impact on backups with accelerated deduplication. Combined with the consolidated data management provided by Data Protector, customers have a wide choice of configurations which can be used to dramatically increase the role of disk in the protection of critical data.

Appendix

[2] Detailed configuration information can be found in the Appendix.

[3] Source: ESG Research Report, Data Protection Survey, April 2010.

[4] Full testing configuration is described in detail in the Appendix.

[5] Source: ESG Research Report, Data Protection Survey, April 2010.

[6] Assumptions and parameters can be found in the Appendix.

Online Digital Archiving, the Market Backdrop

Challenges, Realities, and why Archive?

ESG estimates that organizations will retain nearly 63,000 petabytes of unstructured data in digital archives over the next three years in order to meet litigation and regulatory mandates (e.g., Federal Rules of Civil Procedure (FRCP), SOX, HIPAA, GLBA, and PCI DSS) and to support business intelligence initiatives.[1] To put this in perspective, it was only in 2009, that Eric Schmidt, Google’s CEO, estimated the total capacity of data on the internet at 5,000 petabytes. [2]

ESG research indicates that the vast majority of corporate digital assets are stored as unstructured data. Unstructured file data—which includes office documents, digital images, audio, and video files—accounted for 74% of global digital archive capacity in 2009 and is expected to constitute the bulk of digital assets for the foreseeable future.  Growing at a compound annual rate of 79%, the worldwide capacity for file-based unstructured archive data clearly dwarfs that of database and e-mail data, as shown in Figure 1. The situation cries out for organization, management, and easy access.

It’s no surprise then that the majority of organizations recognize the need for file-based archiving solutions: 65% of organizations surveyed by ESG currently have some form of formal process and/or technology in place to manage the archiving of file-based content. An additional 29% of organizations surveyed expect to deploy file archiving solutions over the next few years.  In the same survey, improved search, performance, and security were high on customers’ archive solution wish-lists. Indeed, full content index and search is the number one “must have” feature among current and prospective customers of archiving solutions.

Nexsan’s Assureon: Secure, Online Archive

Assureon is a self managing content addressable storage (CAS) platform that extends Nexsan’s family of highly efficient storage systems. Assureon is a disk-based online archive platform designed to be a simple, plug-and-play solution that enables organizations of all sizes to protect, retain, search, and retrieve unstructured digital assets. Applications producing fixed content can point to an Assureon located on the network as a shared drive and begin writing data to the device immediately. This simple process enables IT generalists in small enterprises and mid-market companies to quickly deploy the system with confidence.

Assureon is engineered with the goal of providing a secure, online archive for high value content in a high performance platform that is power-, space-, and cost-efficient. Windows 2003 and newer servers use a lightweight client that provides direct archiving to Assureon of any desired drives or folders while any platform that can mount an NFS or CIFS share can access Assureon through an integrated NAS gateway, completely transparently.

Assureon provides these benefits in a flexible, scalable package that allows customers to deploy their own self-managing archive solution or to engage a service provider to provide secure hosted archive services, as shown in Figure 3.

Hands-on: Assureon in Action

Assureon online disk storage archive provides numerous features to guarantee that files are safe from prying eyes, have not been tampered with, and are not corrupted over time. ESG Lab recently tested Assureon’s online archive capabilities in a simulated content management environment, focusing on data privacy, integrity, and longevity.

Privacy

Assureon uses the following capabilities to provide secured, private access to data, both within organizations and between multiple organizations sharing the same infrastructure.

• Data Separation and Security: Individual users, departments, or customers can store their files in their own virtual archive. Data is logically and physically separated, ensuring privacy and security.
• Access Audit Trail: Assureon establishes an unalterable audit trail for the life of an archived file. Every time a file is accessed, a record is kept of who accessed it and when it was accessed.
• Encryption: Assureon protects information privacy and security with AES256 encryption, ensuring the highest levels of file privacy and security.

To provide scalability and data separation, Assureon can be divided into separate CAS archives, providing physical data separation inside an enterprise or between customers of a cloud computing SaaS provider.  Each CAS archive can house multiple virtual archives, each consisting of a separate database and CAS object store, as seen in Figure 4.

Assureon’s support for multiple virtual archives enables Assureon to scale, have no CAS object limits, and to take advantage of Nexsan’s AutoMAID reductions in power and cooling costs for infrequently accessed virtual archives. Each classification within a virtual archive can be further divided into subclasses, such as departments, for more granular control. All CAS archives and virtual archives and data classifications can be managed from a single pane of glass as a single system.

ESG Lab Testing

ESG Lab used a standard x86 PC running Windows Server 2008 with the Assureon client installed.  Using Windows Explorer, a folder named DATA was created and its properties were accessed by right-clicking on the folder name.

As seen in Figure 5, the Assureon client adds a tab to the properties page to access and set Assureon archive policies. Administrators and users will only see the Assureon tab if they have permission to set policies.

Two folders were created: Finance and Marketing. For both folders, ESG Lab set the organization name, retention rule, and action to be performed on all files created in or copied to the folders.  Archiving can be set to real time and/or sync mode. Real time causes all files to be processed immediately upon creation and close or modification and close, while sync processes files manually or according to a schedule. For the purposes of this test, the action was set to “Replace with shortcuts” to immediately replace all archived files with shortcuts. Users have the option to leave the original files in place while still protecting the files in the archive, or set a threshold to have files replaced with shortcuts when space is needed. The Assureon Web GUI can be used to create more advanced or detailed archive policies, including setting flexible or compliant retention, setting the number of days before an archived file is replaced with a shortcut, or the specific file name pattern to match, if only certain files in a folder need to be archived.

ESG Lab tested data privacy by logging in as two different users, one from the Finance department, and one from the Marketing department, then examining their archived folders using Windows Explorer, Assureon Explorer, and the content search feature in the Assureon GUI. Only the folders and files each user had permission to see were visible.

Integrity

Assureon ensures data integrity using multiple integrated techniques and technologies.

• Fingerprint and Integrity: When data is archived, it is assigned its own unique fingerprint that stays with the file through its lifecycle. If a single bit changes, the fingerprint will change.
• Self-Auditing and Self-Healing: Assureon continually monitors files for fingerprint discrepancies protecting against tampering, viruses, corruption, accidental or deliberate deletion ,and theft. If discrepancies are discovered, Assureon notifies administrators and automatically self-heals the file.
• Independent Date and Time Stamp: Assureon’s independent time source prevents modifications to the system’s time clock ensuring the integrity of retention periods. With Assureon, tampering with the system clock or the date stamp on the file itself is prevented.

To test data integrity, a text file was created and copied into the folder created in the previous step. The copy operation completed instantly, with all the archiving and housekeeping functions, such as calculating the digital fingerprint, completing automatically and transparently. The file was examined and verified that the text was exactly as entered.  Next, the file was intentionally “corrupted” by changing the contents of only one copy. ESG Lab examined the integrity log, shown in Figure 7. Self Auditing and Self Healing after the system completed an automatic integrity audit. The integrity log shows that the changes to the object were detected, corrected, and the corrupted copy of the file was copied to a quarantine area to preserve it for investigative purposes.

ESG lab examined the contents of each version of the file using Assureon explorer, confirming that the contents of the file had been restored to the correct state.

It’s important to note that Assureon explorer enables administrators to restore shortcuts, or full files, to a production server by right-clicking on the file and selecting the appropriate action.

Longevity

Longevity is the ability to guarantee that all versions of a file will remain immutable and uncorrupted over time. Assureon uses multiple techniques to provide file longevity.

• File Availability Audit: Assureon serializes each file and continually checks to ensure all files are present in each store. If a file is missing, Assureon notifies administrators and automatically returns it to its original state.
• Automated Retention and Deletion: To satisfy regulatory compliance or corporate governance, organizations need to guarantee the retention and deletion of files. With automated integrity management and file immutability technology, Assureon protects against accidental or unauthorized file deletion and ensures files can be retained for compliant or flexible time periods as well as securely deleted when their retention period expires.
• Legal Holds: Legal holds can be placed on files to protect against file deletion when retention periods have expired. This ensures availability of files in cases of litigation or potential litigation.

ESG Lab examined multiple aspects of file longevity.  The retention rules page is shown in Figure 9. Assureon has the ability to create separate sets of retention rules for every organization created in the system. Each organization can create multiple rules within each organization to provide appropriate retention for any data type.

A retention rule was created with a retention period of seven years and data encryption enabled. Next, the audit logs were examined and it was confirmed that Assureon was running both file availability and integrity audits daily, restoring corrupted and lost files as necessary.

The Bigger Truth

ESG research has found that unstructured file data accounted for 74% of global digital archive capacity in 2009 and is expanding at a compound annual growth rate of 79%. The worldwide capacity for file-based unstructured data dwarfs that of database and e-mail data combined—and growth is accelerating. Organizations must deal with all this growth while meeting compliance and governance standards and controlling the cost of search and discovery. In this scenario especially, additional complexity is not desirable.

Nexsan’s Assureon is an excellent fit for businesses of any size that require intelligent archiving with CAS. The low cost of entry makes this solution available to a whole new set of IT departments that until now simply couldn’t afford to acquire or manage such technology. Companies can implement robust data security and protection with limited IT resources, and still utilize enhanced search capabilities to quickly produce pertinent data for internal investigation and litigation support without having to incur additional expenses from service providers culling through vast archives of data located on different media types.

Assureon delivers more business value by minimizing the power, space, and cost for given capacity and performance workloads. In addition to its powerful archiving tools, Assureon allows users to move away from a traditional tiered storage implementation and to gain the added IT and business benefits of Nexsan’s highly efficient storage.

ESG Lab was quite impressed with Assureon’s transparent and simple integration into a standard server environment, integrating archive policies, classification, search, reporting, and migration into one intuitive package.  ESG Lab testing revealed that Assureon provides excellent data privacy, integrity, and longevity in a package that is as easy for small businesses to deploy and manage as it is for large enterprises or service providers. The platform scales performance and capacity independently, leveraging Nexsan’s family of highly efficient RAID enclosures and industry standard servers.  It is clear that Nexsan is continuing to execute on its vision to provide users with a highly integrated storage infrastructure.

The company does face a couple of notable challenges. First, the CAS market has never exploded in quite the way that some imagined it would and second, the CAS market that does exist also has a very dominant player (EMC). However, like so much in life, the flip side to challenge is opportunity: EMC and others have shown that a decent market does exist and it’s all open to Assureon if its channel can find a way to the table; Assureon brings a level of simplicity and functionality that could well both expand the realistic market potential and certainly give prospective users more than a moment’s pause before leaping with the default solution. If Nexsan were a brand new player, its task might be very difficult indeed—however, it is a proven, relatively small, but high quality—provider with a base of “raving-fan” users. Nexsan already has several large medical OEMs supplying Assureon to medical and PACs archive end users.

ESG Lab found the system to be extremely easy to manage, providing policy based archiving while preserving privacy, guaranteeing data integrity, and protecting data longevity from a single, simple interface.  In ESG’s opinion, Assureon will enable organizations to reduce their costs and simplify their infrastructures while providing enterprise class archiving services at the speed of online disk.

[2] Chang, Fay Dean, Jeffrey Ghemawat, Sanjay Hsieh, Wilson C. Wallach, Deborah A. Burroughs, Mike Tushar, Chandra Fikes, Andrew and Gruber Robert E. “Bigtable: A Distributed Storage System for Structured Data”, 2006, Retrieved from http://labs.google.com/papers/bigtable.html on 10/7/2008

Introduction

The use of server virtualization to consolidate server infrastructure, reduce data center floor space, and maximize utilization of existing assets has seen phenomenal growth over the past decade, but server virtualization’s considerable success is dwarfed by the potential of desktop virtualization. Increasing  variety and numbers of client device types, the mobilization of the workforce, “always-on” expectations for corporate IT services, evolving regulatory compliance mandates, tightening security policies, and a driving need to increase operational efficiency all combine to make desktop management a daunting task for even the best IT organizations.

A growing number of organizations are using VDI (virtual desktop infrastructure) technology to reduce the cost, complexity, and risks associated with desktop management while providing a high-quality, predictable, and productive computing environment. This report documents hands-on testing of an HP StorageWorks P4000 SAN in a VMware View environment—paying special attention to ease of management, performance, storage efficiency, and availability.

Background

ESG surveyed IT professionals with a goal of understanding the growing interest in VDI.[1] As seen in Figure 1, simplification tops the list of factors driving the adoption of VDI technology. Specifically, administrators are looking to simplify the repetitive, hands-on tasks of OS and application deployments, upgrades, patch management, and provisioning while improving remote users’ computing experiences.  Given the budgeting and manpower challenges being driven by global economic concerns, it’s not surprising that more than half of respondents indicated that reducing capital and operational expenses is driving an interest in VDI.

In order to address these challenges, a VDI solution must be easy to deploy and manage, highly virtualized, highly available, and predictably scalable. N-way clustered storage architecture is ideally suited to address all of these issues. N-way clustered storage supports multiple storage controllers in a single cluster, which, though it may contain many storage controllers, appears to applications and users as a single logical system for easy management. In traditional dual controller storage systems with fixed architectures, when a user’s environment outgrows their storage system, they may be forced to buy another system to achieve greater performance or capacity. Clustered storage systems allow users to add CPU, memory, and bandwidth transparently, enabling them to scale based on the needs of the business without purchasing a whole new storage system. Such clustered architectures allow for the aggregation and virtualization of all hardware resources, performance, and capacity in a linear fashion—just-in-time and as needed.

The HP StorageWorks P4000 SAN

The HP P4000 SAN is a clustered storage system that scales to meet the needs of VDI environments with ease. HP P4000 SANs are built on enterprise-class, industry-standard platforms configured as fully contained storage nodes that provide CPU, memory, bandwidth, and capacity. Each storage node is powered by SAN/iQ storage software, which provides intelligent storage system functionality. Customers can scale performance and capacity online as needed by adding additional storage nodes without disruption to the SAN, VM’s or physical server applications. The HP P4000 SAN remains a single logical system regardless of how many storage nodes are added to it, making it just as easy to manage a 16-node cluster as it is to manage a 2-node cluster. Additionally, adding nodes to the cluster is a transparent and non-disruptive process. HP advises ESG that the average cluster size sold is 4-6 nodes. The average cluster size deployed in the field is 15-20 TB and 20% of the clusters deployed in production contain more than 10 nodes.

The economics of a clustered network storage system are compelling, with the potential to significantly reduce capital and operational costs. With HP’s P4000 SAN, the customer only has to add another storage node to increase performance and capacity—which costs far less than acquiring a whole new system—and cover its associated software, licensing, and maintenance charges. Most midrange storage systems support, at best, a dual-controller configuration, which limits scalability and flexibility.

At the core of the HP P4000 SAN’s value is  the HP SAN/iQ storage software platform, which provides SAN management features such as  storage clustering,  application integrated snapshots, thin provisioning, remote copy (asynchronous replication), and SmartClone volumes.  In addition, SAN/iQ includes the unique Network RAID feature, which protects against component and environmental failures while keeping data volumes online and accessible.  The Network RAID feature provides a level of high availability usually found only in the most expensive SAN arrays, often as an optional software component. Network RAID is included with every P4000 SAN and can be enabled, modified, or disabled online. The ability to keep a volume online and accessible is a key benefit to the VDI environment as the loss of volume access could affect dozens, if not hundreds, of desktop users.  The P4000 comes with all software functionality built-in. There is no additional software to purchase.

It is important to note that in a VDI environment, the other benefits of a P4000 highly virtualized storage cluster in combination with VMware, such as thin provisioning and Linked Clones, can be leveraged to significant effect. Thin provisioning offers a simple solution to the problem of stranded capacity—it is a storage system technology that allows users to safely allocate as much logical capacity as needed to a desktop volume while physical capacity is drawn from a common pool of storage on an as-needed basis; only when a desktop application performs writes is physical capacity drawn from the storage pool. Additionally, physical capacity can be added to the storage pool non-disruptively at any time.

All virtual desktops share common data blocks. Using the VMware Linked Clones feature along with P4000 Thin Provisioning produces a highly efficient way of storing desktop data within the SAN.  Through conversations with end-users and HP, ESG Lab has observed that a range of 3% to 15% of a desktop system volume is typically consumed by unique data. Thin provisioned Linked Clones would be 90% efficient in a VDI environment where the unique data written in each desktop is equal to 10%. A 250-desktop environment where each desktop was allocated 30 GB would normally require over 7.5 TB of usable storage. Linked Clones could reduce the storage requirement in this example to less than 1 TB, a reduction of 85%.

Virtual Desktop Infrastructure with VMware View and VMware vSphere

VMware View is a desktop virtualization system that centralizes and delivers desktops as a managed service to users—anywhere. The VMware solution includes VMware vSphere virtualization software and VMware View for hosting desktops, user and session management, provisioning tools, and application delivery as well as service monitoring, reporting, and support.  VMware desktop virtualization leverages PCoIP technology, a server-centric protocol that does the majority of graphics rendering and processing on the VMware servers, transmitting only compressed bitmaps or frames to the remote clients over the LAN or WAN. Through VMware View Manager, VMware View provides a single management tool to provision new desktops or groups of desktops and a simple interface for setting desktop policies. View Composer, based on VMware Linked Clone technology, enables the rapid creation of desktop images from a master image. When updates are implemented on the parent image, they are pushed out to any number of virtual desktops in minutes, simplifying deployment and patches without affecting user settings, data, or applications.

Figure 2 illustrates a virtual desktop environment utilizing VMware View and the HP P4000 SAN.

Users connect to the VMware View Manager using either the VMware View client or any one of numerous certified third party hardware and software clients.[2] VMware View Manager authenticates a user’s credentials and then uses those credentials to automatically authenticate users as they log into their virtual desktops, using volumes residing on the HP P4000 cluster. The virtual desktop is delivered via the PCoIP optimized delivery protocol. The user has access to their personalized desktop, applications, and resources from anywhere while still benefiting from centralized desktop management in the data center.

ESG Lab’s testing was designed to validate the business value of deploying an HP P4000 SAN to support a VMware View VDI, including capacity, performance, and operational efficiencies uniquely enabled by the HP P4000 SAN .

ESG Lab Validation

ESG Lab conducted hands-on testing of the HP P4000 Virtualization SAN with VMware View VDI at a Hewlett-Packard facility in Houston, Texas.

Reducing Complexity

The test bed, shown in Figure 3, consisted of a pre-installed, pre-configured four-node HP P4000 Virtualization SAN supporting a VMware View virtual desktop environment. Two P4000 SANs with two nodes each were configured using HP ProLiant DL385 G6 servers and HP ProCurve switches for the IP traffic between the SANs. Windows workstations running Windows XP were used as virtual desktop endpoints.[3]

VMware View offers both persistent and non-persistent hosted virtual desktops. A persistent virtual desktop provides each user with a dedicated virtual machine. Users connect to the same machine each time and all changes and personalization persists between sessions. The assignment can either be pre-determined by the administrator or pulled from a group of available desktops and assigned on first access.

Non-persistent desktops are a group of virtual desktops offering a standard configuration. Users are connected to any of the available desktops and when they log off, that desktop is returned to the pool. Backgrounds, bookmarks, application settings, and other personalization can be captured separately in the user’s profile. System changes, such as installed applications, are discarded and the desktop is reset to its pristine state. This ensures that the virtual desktop is always in a known good state and the next user that connects will get a “fresh” desktop configuration. ESG Lab tested using the persistent method for this report.

ESG Lab Testing

ESG Lab began testing with provisioning and configuration of a new virtual desktop. A new volume was created in two steps using the HP P4000 Centralized Management Console, seen in Figure 4 and Figure 5.

ESG Lab right clicked on the navigation tree, seen in Figure 4 and selected New Volume, which launched the new volume dialog box.

Next, a name for the volume (SH-L001) was created and the desired capacity (500 GB) was entered, as shown in Figure 5. Finally, Figure 6 shows how the new volume was assigned to the four VMware View servers.

Once the volume was visible to the VMware cluster, the vSphere Client management console was used to rescan the adapter for the new storage and add the volume to the server, shown in Figure 7.

Next, a virtual machine was deployed from a template to provide a master image for cloning large numbers of virtual desktops, shown in Figure 8.

ESG Lab booted the virtual desktop and confirmed that it was accessible from the endpoint machine. The entire process, including storage provisioning and allocation, took just over seven minutes.

Storage Efficiency

One way of provisioning virtual desktop environments  is to build fully provisioned volumes that will act as a remote user’s primary  computing environment,  including their operating system and applications. An administrator will create the volume for the new virtual machine and either install the client OS or (more commonly) import a previously backed up image. This image is then managed as a physical desktop would be—application and OS patches must be applied to each VM individually and each image consumes as much storage as it would on a physical machine.

HP P4000 utilizes its thin provisioning technology in combination with VMware Linked Clones to optimize both the allocation process and the capacity consumption of virtual desktops. Figure 9 compares traditional allocation and provisioning with thin provisioned Linked Clones. When using thin provisioning and Linked Clones, a virtual desktop is built and then a snapshot is taken to create a parent image of a virtual machine. Linked Clones are created from the snapshot using VMware View, which sees them as independent read-writable volumes. These volumes already have the OS image and applications installed on them, so the installation or import step is not needed. HP P4000’s thin provisioning technology operates on a zero-reservation principle, meaning that no storage is pre-allocated to a parent image or Linked Clone and data is only drawn from the allocation pool as new data is written.

Virtual desktops are typically very light in terms of the amount of data written as a percentage of the volume capacity, resulting in an environment that is an excellent candidate for thin provisioning. Based on ESG’s experience in the lab and HP’s experience in the field, ESG Lab is confident that 70% to 90% capacity efficiency can be achieved over the life of a VMware Linked Clone in a VDI environment.

ESG Lab Testing

ESG Lab evaluated the storage efficiency of the HP P4500 Virtualization SAN in a VMware VDI environment by creating 72 virtual desktops using a single parent image.

First, ESG Lab accessed the VMware vSphere client and identified the virtual machine to be used as the parent to create the new virtual desktops. As shown in Figure 10, the virtual machine master snapshot (parent image) was created and named SS-XPM-01.

Once the snapshot was created, VMware View manager was used to deploy a pool of desktops based on the parent image via a deployment wizard, as detailed in the next few screenshots. First, the wizard asks what type of desktop configuration is being deployed; ESG Lab selected “automated pool.” An automated pool contains desktops that are automatically created and customized by View Manager based on a VMware vCenter virtual machine template (the snapshot created in the earlier step).

Next, the desktops being created were specified to be persistent. Persistent virtual desktops are assigned to their user on the first use, so the user returns each time to the same virtual desktop. This type of pool is used when users want to customize their desktops by installing additional applications and storing local data. Non-persistent desktops are used in environments where desktops are only required temporarily and can be deleted after every use to give users a clean desktop every time they log in.

Figure 11 shows the next step, specifying the VMware vCenter servers that will be utilized for this desktop pool and the use of Linked Clones.

Next, a unique ID and desktop settings were specified for the pool, followed by provisioning settings, where the number of desktops was specified as 24 for this pool. Several steps followed where the parent VM was identified as well as the default image snapshot and the cluster on which to run the virtual desktops was selected as well as a location to store users’ data.

Figure 12 shows the datastore selection screen. Storage overcommit, which determines how aggressively the system assigns new VMs to the space available on a datastore, was left at the default (conservative).

After specifying the datastore, QuickPrep settings were specified, which are used to configure desktops after they have been created, including joining them to a domain, if necessary.

Figure 13 shows the summary screen of the Add Desktop wizard. When ESG Lab clicked “Finish,” the 24 virtual desktops were created and configured.

The entire process to create 24 desktops, from the first click of the wizard to the desktops being ready for use, took 35 minutes to complete. ESG Lab repeated the Add Desktop wizard twice to create a total of 72 desktops. The entire process of creating 72 new persistent desktops from start to finish took 101 minutes, or less than one and a half minutes per desktop.

Storage utilization was confirmed using the HP P4000 CMC. The 72 cloned desktops, with a combined virtual capacity of 239 GB, consumed less than 10% of the projected capacity of 2.88 TB based on the 40 GB volume size of the template VM.

Performance and Scalability

In a virtual desktop environment, performance and scalability are determined more by the number and configuration of virtual desktop infrastructure servers than by any other factor. Storage performance requirements are less predictable than traditional IT applications and a storage solution in a VDI environment must be able to meet not only the average IO requirements, but the maximum load that will be generated—typically at the start of a shift when many users will all be logging on at once—while scaling to meet the capacity needs of a large user community.

To reduce potential complexity for users, HP uses the phrase “POD” in a VDI environment to describe the aggregated resources required to host a given number of users. A POD contains any number of virtual infrastructure servers and an HP P4000 SAN system consisting of a specific number of storage nodes as well as software and thin clients. The servers provide the processing power to run additional virtual desktops while the storage nodes provide both additional capacity and storage performance to the cluster.

ESG Lab Testing

To test storage performance in a VDI environment, ESG Lab used a proprietary HP load generation tool, which captures actual desktop IO and plays it back to simulate as many desktop sessions as desired. The HP tool was run against storage clusters of three, four, and five nodes. Previously collected Iometer workload characterization results were audited for 10-, 15-, 20-, 25-, and 30-node environments. ESG Lab used the Iometer workload characterization tool to simulate the type of IO generated by typical desktop operating systems and applications.[4]

HP indicates that a broad range of IOPS has been observed for virtual desktop workers in the field.  Additional internet research reveals 5 to 6 IOPS is frequently indicated as a typical value found for Windows XP workstations, but applications and workloads run by users may alter this considerably. ESG Lab ran the Microsoft perfmon utility to monitor the disk traffic on a physical Microsoft XP desktop for a knowledge worker with heavy IO usage. An average of 20 IOPS was observed over multiple eight hour business days. With these data points in mind, a conservative value of 20 IOPs per virtual desktop user and an optimistic value of 5 IOPS per desktop were used to estimate the number of virtual desktops that can be supported as HP P4000 storage nodes are added to the HP P4000 SAN system.

IOPS results recorded by the HP tool and the Iometer workload characterization utility, detailed in Table 1, were used to estimate the number of virtual desktops that might be supported for each of the configurations.  Both the conservative estimate of 20 IOPS and the optimistic value of five IOPS per virtual desktop were used for these calculations. The results are summarized graphically in Figure 15.

What the Numbers Mean

• The number of virtual desktops that the infrastructure can support scales nearly linearly as storage nodes were added to the HP P4500 Virtualization SAN.
• As the maximum performance rose, response time got shorter due to the larger pool of storage nodes and drives which were available to respond to IO requests.

Availability and Recovery

The HP P4000 SAN architecture addresses availability at multiple levels. Disk based RAID technology is used within each storage node in a P4000 SAN as a first line of defense against hard drive failures. In addition, the P4000 cluster stripes data across all of the nodes in a cluster. The Network RAID feature provides the option, volume-by-volume, to create multiple mirrors of data throughout the cluster to protect against data loss due to the failure (or loss of connectivity) of a storage node participating in the cluster. A “stretched cluster” approach is also supported, where one half of a cluster could be located in a data center and the other half in a second location within a building or on a campus. In this manner, data loss can be avoided due to a localized facility error that affects an entire data center (e.g., an overloaded power circuit or network failure).

ESG Lab Testing

Availability testing was performed against a  six-node P4000 Multi-site SAN comprised of two separate gigabit Ethernet networks connected with a 10 gigabit Ethernet uplink, as shown in Figure 16 . The VMware vSphere servers had connectivity to all storage nodes in the stretched cluster.

In the first phase of testing, ESG Lab introduced a variety of hardware errors to validate fault tolerance. The following errors were injected as an Iometer workload was being run continuously on a virtual desktop running in the VMware View cluster as seen in Figure 17:

• Pulled a back-end Ethernet interface on node 1 at site 1.
• Pulled an active disk drive.
• Replaced the pulled drive.
• Removed connectivity to all nodes at site 1 from the cluster.

Through all injected faults, Iometer continued to run on the virtual desktop without interruption. Finally, ESG Lab simulated a site failure by removing network connectivity to all storage nodes at site 1. There was a pause of approximately 20 seconds, then user access resumed. Users were not disconnected and could continue working after the pause in IO.

ESG Lab Validation Highlights

• ESG Lab found the HP P4000 SAN easy to configure, implement, and manage in combination with the VMware View environment.
• ESG Lab was able to use one “parent image” virtual machine to create and present multiple unique virtual desktops with minimal capacity overhead and no impact to users.
• The efficiency and scalability of the HP P4000 SAN architecture was seen to meet the performance needs of real-world applications deployed in a distributed virtual desktop environment.
• The HP P4000 SAN was able to sustain continuous access for a VMware View user through disk, network, node, and site failures. Each simulated failure immediately triggered an alert that was sent to a systems administrator.

Issues to Consider

• While ESG tested back-end storage performance in a virtual desktop environment, other factors, including the CPU and memory configuration of the infrastructure servers and virtual machines, will have a much greater impact on the end-user experience. ESG Lab recommends that end-users work with their virtual infrastructure vendor to determine the best practices and optimal configuration for each environment.
• While leveraging VMware’s snapshot and Linked Clone technology for virtual desktop deployment and management is compelling and powerful, it is a value-added feature that is licensed per user in a VMware environment. A bundling option which leverages the Snapshot and SmartClone technology built into the P4000 SAN and avoids the VMware license fee would make this an even more cost-effective solution.
• It is important to note that the Iometer utility, used for some of the performance testing in his report is optimized for generating IO and does not place great demands on server memory or CPU. The performance tests conducted for this validation focused on storage scaling and did not include server sizing considerations. When sizing a complete VDI solution, care must be taken to follow vendors’ best practices for server sizing and configuration as well as storage.

ESG Lab’s View

Increasing numbers of clients and applications make desktop management a daunting task for IT. The number of applications supported increases with organization size, compounding desktop management challenges for large organizations. With increasing numbers of corporate applications to support, ongoing maintenance and management tasks directly translate into considerable IT staffing requirements and costs. Like server virtualization, desktop virtualization is establishing a foothold in the data center among IT staffs looking to optimize their current PC environments.

HP’s P4000 SAN has a highly scalable, clustered architecture that simplifies management and allows customers to start at the level of capacity and performance they require today and grow their environments on demand. Additionally, it is easy to use and manage, while providing advanced features such as Network RAID, remote replication, and thin provisioning.

Customers can stretch their clusters to create multi-site SANs. We have seen storage systems that scale in this fashion with NAS and CAS products, but in ESG Lab’s opinion, HP P4000 is a leader in SAN attached true N-way clustered storage. ESG has long been a proponent of scalable clustered storage and believes it will become the dominant approach due to the compelling value it brings.

ESG Lab found that HP P4000 SAN performed well in a virtual desktop environment, providing easy provisioning and powerful integration of thin provisioning and Linked Clone technology to optimize capacity utilization. High availability functionality was also impressive, sustaining multiple failures while providing continuous access to attached virtual desktop users.

HP’s P4000 SAN systems delivered an easy-to-use, flexible, scalable, highly available, and highly efficient storage solution for VMware View customers. Matching in storage what VMware provides for desktops, HP P4000 supports thin provisioning and Linked Clones for creating large numbers of virtual desktops without the delay and cost of consuming actual storage for each clone. And because it is distributed by design, creating a disaster-resilient storage infrastructure is as easy as choosing which storage nodes to configure in each separate location.

Through hands-on testing, ESG Lab confirmed that the HP P4000 SAN provides a robust networked storage foundation for a virtual desktop architecture. With simple configuration, powerful desktop mobility, enterprise class availability, and near-linear scalability, the HP P4000 SAN enhances the intelligence and value of the VMware View virtual desktop infrastructure.

Appendix

[2] http://www.vmware.com/resources/compatibility/search.php?action=base&deviceCategory=vdm

[3] Configuration details are listed in the Appendix.

[4] Iometer configuration details can be found in the appendix.