Background

While IT priorities and challenges are often considered with data centers and other centralized corporate resources in mind, it is important to remember that organizations often have distributed locations that have significant and complex computing requirements. In fact, typical IT challenges are often exacerbated in these remote/branch offices due to distance and lack of onsite IT staff. ESG research found that companies face significant challenges when it comes to delivering applications over the WAN from a central location to employees at branch office locations. According to a recent ESG survey,^[1] nearly half of respondents identified improving application performance for end-users as a key initiative (see Figure 1). Improvements to application accessibility and better collaboration capabilities were also high on the list.

Riverbed Steelhead EX + Granite Overview

Riverbed Technology’s Steelhead product family is designed to provide increased application performance and data transfer speeds over the WAN. Steelhead products address four main solution areas.

• Application Acceleration—Steelhead optimizes both TCP and UDP traffic, addresses application-specific latency, delivers LAN-like performance and availability over the WAN, and enables improved collaboration, file sharing, and productivity for distributed enterprises.
• Bandwidth Optimization—enables network managers to achieve better utilization of existing WAN bandwidth by eliminating redundant WAN traffic.
• IT Infrastructure Consolidation—enables consolidation of IT infrastructure from remote offices to a centrally located facility, maintaining performance, availability, and security, as well as reducing capital expenditure and management costs.
• Backup & Replication Acceleration—enables quick and secure backup and replication from branch locations.

The Riverbed Steelhead family of products is designed to optimize WAN traffic between distributed remote and branch office locations and a central data center. Steelhead appliances run the Riverbed Optimization System (RiOS), which is the software platform that enables data-, protocol-, and application-level WAN optimization and allows a central office to consolidate the majority of its remote office server infrastructure, taking the first step toward true infrastructure consolidation. Riverbed Steelhead EX combines WAN optimization capabilities with VMware on the appliance, enabling a branch to virtualize local servers and minimize the bandwidth required by users and applications.

Granite was developed by Riverbed to deliver edge virtual server infrastructure that extends an enterprise storage area network (SAN) out to remote offices. It enables organizations to centralize and consolidate branch office storage at a primary data center. Steelhead EX + Granite combines Riverbed Steelhead and Granite software capabilities with the goal of serving write-intensive and custom applications in the branch with a global storage infrastructure projected from the data center, eliminating storage at remote branch offices previously considered too difficult to consolidate.

ESG Lab Validation

ESG Lab performed hands-on evaluation and testing of the Riverbed Steelhead EX + Granite WAN optimization and storage consolidation appliance at a Riverbed facility in San Francisco, California. Testing was designed to demonstrate how Steelhead EX + Granite improves application performance and availability, as well as delivers infrastructure consolidation and data security for branch locations in a distributed enterprise.

Getting Started with WAN Optimization

Steelhead appliances at remote locations, along with Steelhead Mobile software on remote user laptops, work together with one or more Steelhead appliances in the corporate data center to optimize traffic flowing over the WAN. Riverbed addresses three areas that affect WAN efficiency: application chattiness, data redundancy, and transport protocol inefficiency.

To speed application performance, application-specific optimizations complete transactions locally in the branch on behalf of servers in the data center, eliminating the need to wait for application responses over a WAN connection. To reduce the amount of data sent over a WAN, Steelhead appliances and software intercept and inspect WAN data to determine whether the data or a portion of it has been seen before. When a user attempts to access data already encountered by the local Steelhead datastore, the data is served locally, eliminating the delay of pulling data over the WAN. With this capability, Steelhead appliances allow users and applications to read and manipulate data, while only requesting or sending unique blocks across the WAN. To overcome transport protocol limitations, Steelhead appliances more intelligently scale and pack TCP payloads, significantly reducing round trips and more efficiently transmitting data across the WAN.

Figure 4 shows the test bed used by ESG Lab, which consisted of a simulated data center and remote office. A Network Nightmare WAN simulation device was used to limit bandwidth and inject latency, simulating a transcontinental T1 link. The data center was configured with one NetApp FAS 2050 connected to both Steelhead and Granite core appliances. The remote office had a Steelhead EX + Granite appliance running Steelhead WAN optimization software, Granite block-storage acceleration, and the Riverbed VSP. The connection between the two environments was limited to 1.5 Mbps (T1 equivalent) and had 100 milliseconds of round-trip latency injected to simulate a remote office connecting to a data center up to 3,000 miles away.

ESG Lab Testing

For the first round of tests, Steelhead WAN optimization was used to optimize typical knowledge worker operating tasks. Data reduction and optimization-related metrics were captured using the Steelhead management console as well as wall-clock timing of certain operations. As shown in Figure 5, simple file transfers, Microsoft Exchange messages with attachments, and Microsoft SharePoint transaction performance were measured with and without Riverbed Steelhead WAN optimization enabled. The corresponding results show performance improvements of anywhere from 5X to 50X, depending on the type of transaction.

The data in Table 1 includes the sizes of the objects used in performance testing and the number of seconds to completely execute each operation. The largest time reduction was seen with the transfer of a 65.3 MB file from a remote client to the corporate file server.

Consolidation of Branch Services

Organizations are using server virtualization to simplify their IT infrastructures while reducing costs in their data centers through consolidation. For services required in branch offices, consolidation (minimizing infrastructure in the branches) is also a key goal. However, organizations are faced with a challenge in that certain applications require local compute and storage resources to meet performance requirements for end-users. Steelhead EX features the Riverbed Virtual Services Platform (VSP) that incorporates VMware virtualization technology to consolidate branch servers and applications onto the Steelhead EX appliance.

ESG Lab Testing

ESG Lab began testing the ease of consolidating branch office services and applications using Riverbed Steelhead + Granite technology by simulating the user experience of moving from a traditional branch office with dedicated servers, applications, and local storage to a virtualized Riverbed WAN-optimized and consolidated model, as illustrated in Figure 6.

ESG Lab first virtualized a Microsoft Windows Server 2008 system in the remote location by installing the VM onto the Riverbed VSP. This is accomplished via the Steelhead management interface. The Steelhead EX + Granite appliance can host up to five end-user virtual machines. ESG Lab connected to the Steelhead EX + Granite appliance through the Riverbed Steelhead Management Console to install the virtual server as shown in Figure 7.

Once the virtual machine was loaded and running, ESG Lab simulated a Microsoft SQL server workload using the Iometer load generation utility. SQL Server is an application that, due to the response-time sensitivity of transactional databases, is often hosted on physical servers with local storage in a branch office. In this test, a 4 KB block size was used with a 67% read, 100% random access pattern.

Figure 8 shows the Iometer results displayed during the test. The most important metric to note here is “Average I/O Response Time (ms).” While the back-end connection to the storage array was over a simulated T1 connection with 100 ms of round-trip latency, Iometer reports only 31 milliseconds of latency to disk because data is being written to the local Steelhead blockstore.

It’s important to note here that without Steelhead EX + Granite, ESG Lab was unable to obtain a usable result due to the restricted bandwidth and high latency of the WAN link. In fact, the connection to the LUN in the data center timed out, and the mount failed.

Consolidating Branch Office Storage

Riverbed Granite extends iSCSI block storage from the data center to the remote site in a way that is transparent to users and applications, and that takes advantage of Riverbed Steelhead WAN optimization technology. Granite enables organizations to maintain local servers at branch offices while actually storing and protecting their data within their data centers. Riverbed VSP provides the ability to host those servers directly on the Steelhead EX appliance.

ESG Lab Testing

ESG Lab tested Granite by mounting iSCSI LUNs from the data center on physical branch server nodes as well as within the virtual machines hosted in the Steelhead EX + Granite appliance. To verify the challenge of accessing “unoptimized” iSCSI storage over the WAN, ESG Lab first attempted to mount an iSCSI LUN directly from a traditional branch server to a data center SAN without Granite, and observed that the connection timed out and the mount failed.

Next, ESG Lab tested whether Riverbed Granite could allow the use of iSCSI over the WAN by configuring Granite appliances in both a data center and a remote office location. Figure 9 shows the basic functional design of storage extended with Granite. In effect, there are two iSCSI connections (working from right to left):

• Within the data center (right), between the actual iSCSI SAN target to the Granite core iSCSI initiator.
• Within the remote site (left), between the production server iSCSI initiator to the Steelhead EX + Granite iSCSI target.

Using the same branch server, ESG Lab was able to successfully mount with no errors or timeouts the same iSCSI LUN that had previously failed to mount in the “unoptimized” test.

The combination of Granite with Steelhead WAN optimization makes it possible for a data center LUN to be successfully mounted by a remote office production server—either a physical server located in the branch or one that is virtually hosted within the Riverbed VSP hypervisor.

Next, ESG Lab examined performance and usability. For this test, ESG Lab (working right to left in Figure 9):

1. Configured multiple LUNs within the data center SAN, which is completely unaware of the Steelhead EX + Granite extended storage scenario.
2. Configured the Granite core iSCSI initiator to mount the LUNs that are to be extended.
3. Assigned a specific Steelhead EX + Granite edge device from the Granite core at the data center to extend each LUN. By doing so, the Steelhead EX + Granite device became an iSCSI target and offered the LUN to devices within the remote site.
4. Connected the LUN to the production Windows server using its iSCSI initiator, with the server being completely unaware that the LUN is not within the remote site but actually extended from the data center.

From there, the LUN behaved like any other iSCSI-attached device and could be mounted and utilized. A common concern about remote storage is that the latency associated with initial use or access requests will have an impact on the end-user experience. To observe the behavior, ESG Lab configured an extended LUN within Steelhead EX + Granite that contained a known data set and requested various randomly selected files.

Figure 10 shows the Steelhead EX + Granite Blockstore Metrics report, which measures the amount of “hits” (requested blocks that were already available at the branch) and “misses” (blocks that needed to be sent from the data center) in megabytes. This, in effect, measures the effectiveness of the Granite technology to pre-fetch and/or quickly transmit the necessary blocks, such that the branch server experiences the storage as though it is local.

ESG Lab used AutoCAD software to open a 33.5 MB file (cifs.dwg). The first time the file was accessed across the WAN, the file-open process took 27.7 seconds. The file was closed and the workstation was rebooted to clear client cache. When the workstation was restarted, the same file was opened using AutoCAD, and the file opened in 5 seconds, the same as baseline testing over the LAN.

While Figure 10 shows the Granite technology’s effectiveness in pulling data from the data center to the branch, Figure 11 shows the behavior of data written at the branch being committed to the data center.

• The dark blue line tracks the amount of data being written to the Granite blockstore.
• The light blue areas show data received but not yet written back to the data center storage array, implying minimal latency in transmitting those blocks to the data center.
• The grey areas show data that has been committed to the data center.

Overall, ESG Lab found the Riverbed Granite extended storage solution to be surprisingly easy to configure and utilize. Neither the production server at the remote site nor the data center SAN felt any impact from the extended distance. The iSCSI implementation was intuitive and performed well over a simulated 3,000-mile WAN connection. The combination of Granite with Steelhead technology dramatically accelerated data transfers over low-bandwidth, high-latency connections.

Remote Office Scenarios and Implications

After the servers were virtualized and the SAN storage was extended from the central data center, ESG Lab was ready to explore the performance and resilience of a Riverbed-enhanced remote office when a WAN link goes down and is restored.

ESG Lab Testing

First, ESG Lab set a baseline by opening several large files that resided on an extended iSCSI SAN volume shared on the remote office LAN by a virtual server hosted in the Steelhead EX + Granite appliance without Granite acceleration. ESG Lab opened a 39.4 MB AutoCAD file named “http.dwg” using a client on the remote office LAN. Without Steelhead and Granite optimization, the file opened in 721 seconds, or just over 12 minutes. The AutoCAD application was completely unresponsive while the file was pulled across the WAN.

Next, the same file-open test was performed with Steelhead WAN optimization and Granite active. The first time the file was accessed, the operation completed in 39.1 seconds. After closing the file and opening it a second time, the file opened in 5.0 seconds. As expected, the second open was much faster, being serviced by the blockstore cache on the Steelhead EX + Granite appliance. Saving the file to a new name took 3.1 seconds.

As seen in Figure 12, at 18:07, ESG Lab disconnected the simulated WAN between the remote office and data center, and attempted to open the same file. The file opened successfully, in 5.6 seconds. Next, the file was saved to a new name, which completed in 3.1 seconds. This is comparable to the performance observed when the WAN was connected.

ESG Lab repeated these procedures multiple times, opening files and saving them to new names. Performance was consistent. Figure 12 shows the data writes/commits report from the Steelhead EX appliance.

As shown in Figure 12, as data was written to the network share, the uncommitted bytes that had been accepted by the Steelhead appliance but not yet transmitted to the data center increased. Note that the entire time that the volume was disconnected, the client and server at the remote site remained connected, and the volume remained online.

After approximately 30 minutes, more than 40 MB had been written to the shared volume. ESG Lab then reconnected the WAN and monitored the data writes/commits report. As can be seen in Figure 13, the Steelhead appliance committed the 43.6 MB of data to the NetApp FAS in the data center in about 45 seconds.

The resynchronization was automatic and completely transparent. Users and applications saw no change in connectivity or access when the WAN link was down, nor when it came back up. Figure 14 shows a network traffic summary report for the time period that the resynchronization was executing.

The port number indicates the type of traffic: Port 7951 is traffic flowing between the Steelhead EX + Granite edge device and Steelhead and Granite core devices in the data center. The actual data transmitted across the WAN link was highly optimized, and, of the 43.9 MB of iSCSI data transmitted by the virtual server, only 3 MB was actually transmitted across the WAN, a reduction of 93%.

Data Protection Scenarios and Implications

When extending storage from the data center to the remote office, data protection becomes multifaceted, incorporating not only backup and recovery of production data, but also protection of remote office data from loss or theft.

Securing Data in the Appliance

Riverbed appliances utilize integrated storage to hold cached data in remote locations, designed to enhance the remote user’s experience by providing local access to frequently used data. The Riverbed Steelhead + Granite appliance offers AES encryption (up to and including AES-256) to securely encrypt the data on disk. The AES key for the Granite blockstore is kept in a secure vault area, which is also encrypted using AES-256.

The default key to each appliance vault is unique, derived from a unique identifier of each appliance. The vault key can be changed by organizations to comply with their own security standards. When an appliance boots, the vault key must be provided, or the contents of the blockstore are not accessible. A visual representation of Riverbed encryption is shown in Figure 15.

Backup and Recovery

When considering backup and recovery, multiple data protection scenarios are either enabled or enhanced, including:

• File/application-based protection of the remote data, from the data center
• Block-based protection of the LUNs used by the branch platforms, from the data center

In principle, because server-centric storage utilized at the branch is in fact extended from the data center via Granite (and user-centric data from the branch is stored locally on Granite-extended volumes), Riverbed suggests that data protection can be done entirely from the data center instance of the data.

ESG Lab audited the operating methods that Riverbed uses for storing its data to understand the viability for customers to use their current data-protection methods within a Steelhead EX + Granite deployed configuration.

Traditional File/Application Backups from Guest-VM Branch Servers

For production servers running at the branch, presumably as virtual machines within the Steelhead EX + Granite (VMware Virtual Server) host environment, traditional file- and/or application-centric backups are still achievable.

ESG Lab looked at how a typical backup agent can be installed within a virtualized production OS to send backup data to the requesting backup server located at the data center, as shown in Figure 16. In this configuration, as files are queued to be sent from the production VM to the backup server, Steelhead WAN optimization is designed to recognize the data that already exists at the data center from previous synchronizations. In this case, while both the remote backup agent and the data center backup server believe that the data is being sent across the WAN, only truly unique data segments and reference “pointers” to previously encountered data actually traverse the network.

Based on how ESG Lab tested file transfers with Steelhead EX + Granite, file-centric data movement during backups should be nearly eliminated. Similarly, application-centric backups that generate storage IO as part of the backup process (such as SQL Server log files) will benefit. In those cases, as the data files are prepared for backup, their corresponding blocks within Granite will be committed to the data center and therefore may not need to traverse the WAN during the actual backup.

SAN-based Backup of the Branch from the Data Center

Perhaps the most intuitive and yet subtle method ESG Lab observed was the ability to back up the extended LUN from within the data center. Because the SAN is unaware of the Steelhead/Granite solution, LUNs can be backed up directly using storage-based snapshots and clones—traditional “serverless” backup solutions.

As seen in Figure 17, ESG Lab observed that by backing up the original LUNs from the SAN, all of the production data could be protected in the data center, including virtualized server-centric data and client-specific data that are Granite-extended.

As shown in Figure 17, ESG Lab found serverless backups of remote office volumes to be potentially ideal choices for those customers who already utilize them within their data centers. The IO burden is removed not only from the production resources, but also from the Steelhead and Granite appliances, freeing them up for production IO exclusively. Like any serverless backup, an understanding of the applications in use and the need for consistency and post-backup processing are keys to success.

Riverbed SAN Hardware Snapshot Integration

Along with extending and potentially enhancing customers’ existing backup methodologies, Riverbed has also developed a Riverbed Hardware Snapshot Provider (RHSP) mechanism to directly integrate its storage-extending capabilities with both the Microsoft Volume Shadow Copy Service (VSS) and SAN arrays from EMC, Dell EqualLogic, and NetApp.

Although RHSP was not tested by ESG Lab, a discussion with Riverbed highlighted RHSP capabilities that directly address the need for application consistency with backup. RHSP installs as a plug-in on Windows clients in the branch office. It is used within the VSS process to place a point-in-time marker into the Granite blockstore. This enables a backup agent to quiesce an application to indicate an application-consistent restore point. In turn, this indicator triggers a snapshot on the data center SAN storage array that can then be used for any required restores or subsequent secondary backups to disk or tape in the data center.

ESG Lab Validation Highlights

• ESG Lab used Steelhead appliances to optimize WAN performance and reduced data by up to 50 times, enabling more productive collaboration between remote and central offices.
• When running an OLTP database workload in a Riverbed appliance-hosted virtual machine, ESG Lab observed latency to storage over the WAN decreased by more than 67%, enabling a remote server to mount an iSCSI volume hosted in a distant data center, and making it possible to consolidate business-critical branch services over the WAN.
• ESG Lab found Granite-extended block-storage performance over a simulated transcontinental WAN link to be remarkably viable, rivaling local-attached storage in both throughput and latency.
• ESG Lab validated that Riverbed improved performance by a factor of 18 to 26 times when opening and editing large CAD files across a high-latency, low-bandwidth T1 link, providing WAN access to centralized project files at LAN-like speeds.
• ESG found that customers’ existing backup methodologies were all potentially viable options for remote offices—in ways not achievable without the combination of WAN optimization and storage extension. Without changing their backup mechanisms, customers may find their solutions enhanced because of how Steelhead optimizes the data streams that Granite has already synchronized between sites.

Issues to Consider

• It should be noted that one key to the performance of the extended storage is the built-in blockstore within the Steelhead EX + Granite appliance itself. When designing the storage to be used at the branch, customers should be aware that the Granite solution does not change normal design considerations around capacity or IO performance. Those aspects should still be considered when determining the size of the Granite edge device to place at a particular branch location.
• While many data-protection scenarios are enhanced through this configuration, for SAN-based backup of the branch from the data center, a minimal amount of effort is still required to ensure the boot volumes of the virtualized VMs on VSP within each branch appliance are protected and recoverable. ESG Lab hopes that Riverbed will address this in future releases, so that even in the VSP scenario, an entire VM can be protected at the data center.
• While many backup processes may potentially gain benefit from a Riverbed Steelhead solution with Granite technology, they do so without any awareness of Riverbed’s changes to infrastructure or topology.

The Bigger Truth

Riverbed provides comprehensive WAN optimization solutions, helping organizations share applications and data across global wide-area networks. Riverbed WAN optimization solutions have been proven in the field to give businesses order-of-magnitude increases in the performance and value of their existing IT infrastructure and mission-critical applications, including file sharing, e-mail, backup, document management systems, IT tools, and ERP and CRM solutions.

Riverbed has applied its field-proven WAN optimization technology to provide similar performance gains for SAN-based block data extended to remote offices. Achieving a data reduction of more than 26 to 1 in ESG Lab testing, Steelhead Granite technology not only reduces the amount of bandwidth needed to connect to data center SAN storage, but also provides access to remote users at local speeds, maximizing the productivity of those remote workers. Steelhead EX + Granite also enables organizations to utilize existing investments in data protection hardware and software and secures that data in the data center.

In a truly fluid enterprise, all data and storage resources will be centralized in the data center. When that occurs, organizations will gain the ability to provide desired performance in the branch and the ability to quickly provision systems and storage wherever or whenever they wish. In addition, data protection becomes much easier and more secure for remote offices—it is executed centrally along with all valuable corporate data in the data center.

ESG Lab confirmed, through hands-on testing, that Riverbed’s Steelhead EX + Granite solution is able to reduce remote office network traffic while extending data center SANs with little impact on remote office servers and clients. The solution integrated well with Microsoft Exchange and SharePoint business applications as well as basic file system services and iSCSI block storage, long considered all but impossible to extend over long-distance, low-bandwidth WAN links. Organizations interested in improving the remote user experience while bringing data-center-class performance and protection to their remote offices should seriously consider Riverbed Steelhead EX + Granite.

Appendix

[2] Source: ESG Research Report, 2011 IT Spending Intentions Survey, January 2011.

[3] Source: ESG Research Report, 2011 Remote Office/Branch Office Technology Trends, July 2011.

[4] ESG Research Report, Remote Office/Branch Office Technology Trends, July 2011.

[5] Source: ESG Research Report, 2011 Remote Office/Branch Office Technology Trends, July 2011.

[6] ESG Research Report, Remote Office/Branch Office Technology Trends, July 2011.

The Challenges

Companies of all sizes continue to struggle with the various aspects of data protection. A great deal of attention is paid to solving not only traditional backup/restore, but also adding archiving and storage resource management to their infrastructures. Along with improving backups of virtualization platforms, laptops, and key workloads, ESG research[1] found that IT end-users planning to implement new data protection initiatives had other goals as well:

• 19% plan to implement data archiving
• 19% plan to implement data deduplication
• 18% plan to re-architect their backup processes
• 13% plan to implement reporting of backup/storage

Users attempting to address diverse backup, archive, and reporting needs often employ technologies from multiple vendors—each with their own agent technologies on individual production servers, as well as their own server back-ends and management interfaces. Each point solution performs its own operations on every production server, including traversing the disk, consuming memory/CPU cycles, and contributing to network traffic.

The Solution: CommVault Simpana 9.0 with “OnePass”

CommVault customers running Simpana software have already learned to appreciate something better than a myriad of point solutions. Simpana software’s common platform delivers backup, archive, search and storage resource management administered from a single console. While built on a single software code base, Simpana software modules have previously utilized separate processes and index databases to run archive jobs, followed by backup and, finally, reporting.

Throughout 2011, CommVault regularly added incremental features to its Simpana 9.0 platform—one of which is a new operating methodology referred to as “OnePass,” which enables backup, archiving, and analytical reporting from a single traversal of the file system. By only reading and/or moving data once, redundant backup, archive, and reporting processes are eliminated to speed operations, reduce storage costs, and simplify management.

ESG Lab Testing

ESG Lab tested the new OnePass functionality at a shared CommVault and HP test facility located in Denver, Colorado. The ESG Lab test bed consisted of a typical Simpana software configuration of one CommServe and two MediaAgents, each configured to protect three HP X9720 scale-out NAS nodes sharing a single file system, as seen in Figure 1.

The test bed was provided by HP to assess Simpana 9.0’s ability to protect a high-volume of unstructured data.

ESG Lab investigated how CommVault consolidated data protection methodologies using the OnePass architecture. The left side of Figure 2 shows the typical IO patterns of three related data management workflows, including traditional backup, file-archival for reducing disk consumption, and reporting services. The right side of Figure 2 shows the combined workflow of the OnePass-enabled agent in Simpana 9.0.

Figure 2 shows how “OnePass” traverses the production storage only once, thereby eliminating significant IO redundancies on the primary server, which should dramatically reduce backup windows and the IO penalties associated with data protection and management tasks.

In a traditional environment using three data management tools, ideally with some level of integration or at least reporting, one might:

1. Perform a traditional backup for data recoverability using traditional incremental methods.
2. After the backup is complete and therefore recoverable (just in case), determine if any files are candidates for archive (hierarchical) management. These files should be “stubbed” to save space, meaning that the original file is replaced with a “stub” pointer referring back to the original file held in near-line storage. This ensures that the actual contents are able to be retrieved transparently when the file is accessed.
3. With the backup finished and the appropriate files migrated to near-line storage, update the reporting system for usage and capacity.

In the case of Simpana OnePass functionality, the operating methodology is similar … yet optimized:

1. The agent conducts a backup of changed files.
2. With the backup changes successfully committed on the media server, the same agent then assesses the files as candidates for archival, and, if so, stubs the file.

• No additional file system traversal is necessary because it was done during the backup.
• No additional disk “read” or network “send” operations are performed during stubbing, as would be required by a separate archival product. The archival process knows that the backup process already read the file and sent it during the backup operation—so it already exists within the Simpana unified storage pool.
• Either way, the archival routines within the OnePass agent simply perform the stubbing operation of replacing the actual file with a stub—after which the file-system driver will handle retrieval requests in case the file is accessed.

1. With the backup complete and the appropriate files archived, the reporting mechanism updates its information.  Again, this occurs without any incremental disk traversal or network operations because Simpana OnePass uses a common index and reporting mechanism from a single collection.

ESG Lab tested the unified OnePass operating model by first conducting separate backups, archives, and report generation using Simpana 9.0 without the OnePass methodology at work. The files were spread across six nodes of an HP NAS and were backed up in parallel by one of the two Simpana media server nodes seen in Figure 1. After the initial testing, ESG Lab audited the results of a similar prolonged test provided by CommVault.

ESG Lab found that the overall backup time was reduced anywhere from 30% to 200% based on three key factors: data types and sizes, amount of redundancy among stored files (e.g., versioning), and archival retention settings that will vary by company. At the low end, even a 30% time savings may mean the difference between compliance with backup window SLAs or not. At the high end, the incremental nature of these backup processes, coupled with nearly transparent archival and SRM functionality, may make the entire backup tax nearly vanish for some production environments.

While less quantifiable, ESG Lab noted that by 1) only traversing the file system once, and 2) offloading the analysis processes to the Simpana MediaServer seen in Figure 2, an appreciable amount of disk IO and CPU processing should be relieved from the production server(s). This means that the production platforms should spend far fewer resources on data protection/management, reserving resulting in more IO and CPU for production purposes.

ESG Lab was impressed by how simple the process was to enable OnePass for Simpana customers. As is typical, the actual agent software components are upgradable through either a push from the Simpana administration console or an .MSI through the customer’s typical software deployment tool. The software can be deployed at any point even if the OnePass functionality is not immediately enabled.

Figure 3 shows how enabling the Archive or SRM reporting functions within the unified agent (i.e., enabling “OnePass”) is simply a matter of two checkboxes within the backup configuration in the Simpana administration console.

Simpana Archive Integration with Scale-out NAS

“OnePass” is not the only innovation recently delivered for the Simpana 9.0 customer base. Along with backing up large file systems, CommVault now also offers its archival capabilities as the near-line extension of scale-out NAS platforms, including the HP X9000 (IBRIX) product family.

By integrating the Simpana software’s archival ability with scale-out NAS, CommVault software is able to offer an additional tier of near‑line storage, enabling organizations to leverage a wider range of storage options at a better price point.

ESG Lab Testing

ESG Lab initially treated the HP X9720 platforms as the production server farm being backed up by Simpana. By reconfiguring the test environment, ESG Lab was also able to test Simpana archival storage as a near-line expansion of a scale-out NAS appliance.

Figure 4 shows the reconfigured test bed with the production NAS being archived by the recent enhancements in Simpana 9.0, using the HP X9000 (IBRIX) platforms as a recent example.

In Figure 4, files accessed from the X9000 platforms can be either taken from their own storage pool or transparently retrieved from the Simpana Archive. While the user “sees” all files just as they would expect to within the NAS, those files may be within the primary storage of the scale-out file system or within the Simpana archival storage pool (using any storage that Simpana software supports).

While some NAS vendors provide their own “archival” capabilities through storage tiering and near-line capacity, it doesn’t always align with the “unified” data protection benefits described above unless 1) backup and reporting are also performed within the NAS/SAN and 2) the NAS/SAN platform is common across the entire corporate environment.  Using a software-based approach, customers may be able to leverage the unified data protection/management capabilities of CommVault software across a wide variety of production servers and NAS platforms consistently—and as a complement to any data management functions that may be offered by the NAS itself.

ESG Lab tested this by enabling the Simpana Linux file server agent on each of the HP X9000 NAS nodes. While many data management products purportedly present challenges when integrating with IBRIX platforms, CommVault is able to use its standard agent with the addition of a registry key on each IBRIX node.

After enabling the agent, ESG Lab tested the user experience by defining archival policies within Simpana software for various files and then retrieving them from an NFS client workstation.

Figure 5 shows two files used during testing of the archive integration with scale-out NAS:

• The top file listing shows the files were originally 100 MB.
• The left statistic reveals each file consumes 102,512 KB.
• The right statistic reports each file’s size as 104,857,600 bytes in the directory listing.
• The middle of the screen reveals that the files were stubbed after archive—consuming only 20 KB each within the NAS, while still displaying 100 MB in the directory listing.
• The last file listing shows that after accessing one of the files, it has been retrieved and thus consumes its regular capacity within the NAS while the other file remains archived until first access.

Note, that while Figure 5 shows the attributes from an NFS perspective, Windows (CIFS) users would have a similar experience where the actual consumption size is masked and the user perception is all files being offered and stored on the HP NAS.

After enabling archival, ESG Lab configured recurring jobs to enable migration of data from the shared file system within the six IBRIX nodes to the Simpana ContentStore. Files that have been migrated will be returned to make file requests from a client workstation accessing the NFS shares on the X9000. ESG Lab observed no appreciable lag in performance or changes in the users’ experience as file requests were routed to the Simpana platform and transparently retrieved from the CommVault software-powered archive.

ESG Lab Validation Highlights

• ESG Lab examined and tested the combined methodology of “OnePass” with appreciably reduced overall data protection jobs, as well as reduced impact to the production servers due to the consolidated network and disk operations of “OnePass.”
• ESG Lab observed how easy it was to enable Simpana software as an archive to a scale-out NAS, without perceivable changes to the end-users’ experience.

Issues to Consider

• ESG Lab found that while it would be easy for an experienced Simpana operator to add the OnePass functionality to their environment, the Simpana administration console may appear complex to someone new. This is a reasonable result of a very mature ninth-generation codebase that continually adds new features and options based on feedback from over 15,000 customers.[3] Those considering converting to Simpana for its OnePass functionality, its other workload-specific capabilities, or its ability to provide an archival store for scale-out NAS should be prepared for a learning curve which can be offset by training.
• While the HP X9000 is just one of the scale-out NAS platforms supported by the Simpana software archival function, customers will want to ensure that their specific platform is currently covered. With CommVault routinely producing updates and incremental functionality, those not directly supported today may be supported later in 2012.

The Bigger Truth

Most environments struggle with a myriad of data protection and management technologies, perhaps because of workload-specific requirements, data center solutions that are less ideal in remote offices, or simply different data management goals (e.g., backup, archive, and reporting). For many, the sentiment has often been “If there was a unified solution that did everything well, then we would all own it already.” For others, the potential interoperability of suite-based software or simply complementary products from the same vendor have left customers disappointed as they discovered that each product operates as if it were the only tool that matters.

By simply enabling the “OnePass” capabilities within Simpana 9.0, CommVault customers can enjoy something that many others should find very enviable: a single agent that backs up, archives, and reports on each production server, with only one network stream and significantly optimized disk-IO impact. The result is something that appears so intuitive that it should be the measure by which other unified products aspire—where functions/technologies may have originally been developed or even acquired separately, but eventually become folded into a single agent talking to a unified back end.

Along with observing the before and after effects of “OnePass,” ESG Lab also tested integration of the archival capabilities of Simpana software with scale-out NAS, showing an appreciable benefit to customers with applicable platforms. Without changing the client experience or installing client-side software, even the most advanced NAS platforms can take advantage of an additional tier of storage through the near-line capabilities of Simpana.

If you are currently using a variety of data and management technologies for different purposes and have been disappointed by the lack of integration or coexistence supportability, then Simpana may be exactly what you have been looking for. While individual test results will vary, the fact that common disk reads and network operations are unified should be a valuable optimization method that all environments can take advantage of. Looking at the unified workflow of Simpana software’s OnePass methodology should make you ask, “Why doesn’t everyone do it like that?”

[2] See: ESG Research Report, Scale-Out Storage Market Forecast 2010-2015, December 2010.

[3] CommVault press release, November 2011

Background

A growing number of organizations are using solid-state storage solutions in the data center. As shown in Figure 1, 34% of respondents to a recent ESG survey are currently using solid-state storage technology in either servers or external storage systems, and another 35% are currently evaluating it or have plans to do so in the next 12 months.[1] While early adopters report that improved performance was their primary reason for deploying a solid-state storage solution, they’ve achieved a number of additional benefits, including improved power and cooling efficiency, increased environmental tolerance, enhanced longevity, and improved reliability.

The first wave of widespread solid-state storage adoption began about four years ago, when flash memory became available as a solid-state disk drive tier in enterprise-class disk arrays. More recently, a growing number of organizations have installed PCIe-attached flash storage in servers to create a low-latency pool of primary disk or an extended disk cache. As matter of fact, 21% of respondents to a recent ESG survey indicate that they are currently using flash storage solutions in servers, and 15% plan on doing so in the next 12 months.

Early adopters report that an increasing number of performance-critical applications are accelerated with solid-state storage, including OLTP database, ERP financial, OLAP business intelligence, supply chain management, and high-performance computing (HPC) applications. Solid-state storage is also accelerating the performance of consolidated virtual server and virtual desktop infrastructures.

Virident FlashMAX

FlashMAX is a small form factor PCIe drive that delivers high-speed, flash-based storage in capacities ranging from 300 GB to 1.4 TB. FlashMAX is designed to bridge the ever-growing performance gap between server CPU cores and traditional storage solutions. Typical performance issues common to real-world workloads are eliminated as FlashMAX delivers extremely high levels of predictably fast and sustained performance for mixed-application workloads.

Virident uses unique software and hardware to leverage the benefits of flash technology inside the FlashMAX SCM drive. The SCM architecture provides storage-like capacity and resilience while delivering memory-like performance in a small, universal form factor. The software layer, called vFAS (Virident Flash management with Adaptive Scheduling), serves as a gatekeeper, granting access to the flash media as efficiently as possible at all times. Without the need for slower, legacy storage protocols or interconnects, major improvements in application performance occur. These improvements occur due to vFAS’s virtualization of the primary flash media, which is accessible to applications via a standard block device interface. vFAS also intelligently and efficiently manages the asymmetric read/write/erase latencies of flash media to deliver consistent, predictable performance to the applications.

vFAS maximizes flash lifetime with global wear-leveling techniques. When necessary, data is relocated to less-used parts of the flash media to prevent hot-spots and overuse. Concerns regarding reliability and data availability are put to rest by the support of built-in flash-aware RAID. Data is spread across a RAID group that spans multiple flash chips and is protected by a RAID-5-like scheme, which prevents disruption from media failures while maintaining application data access and operational continuity. The RAID implementation is flash-aware and is tied tightly into the garbage collection and wear-leveling mechanisms.

FlashMAX provides a high level of consistency across all application workloads, whether the drive is brand new or fully utilized. The challenges associated with many first-generation PCIe flash adapters have been addressed with the multi-dimensional performance capabilities of FlashMAX, which offers:

• High throughput for small, medium, and large IO block sizes.
• Similar levels of performance for random and sequential access patterns.
• High levels of performance for reads, writes, and a mix of reads and writes.
• Sustained consistent performance over time.
• Extreme performance scalability with multiple FlashMAX adapters in a single server.
• Exceptionally low latencies and fast response times for real-world application workloads.

ESG Lab Validation

ESG Lab performed hands-on evaluation and testing of the FlashMAX at Virident corporate headquarters in Milpitas, California. Testing was designed to demonstrate the multi-dimensional performance capabilities using the industry-standard FIO utility.[2]

First Dimension: Small, Medium, and Large Block Sizes

Performance-sensitive applications that benefit from solid-state storage often have high throughput requirements (e.g., an HPC application processing a large machine-generated data set with 512 KB IOs). Others require high performance for relatively small IO requests (e.g., an OLTP database application with 4 KB IOs or a financial application writing logs with 1 KB block size). A third class of applications requires high performance for a mix of block sizes, with large IOs being used for data requests and small IOs used for metadata requests. The FlashMAX optimizes performance for each of these workloads, delivering predictably fast performance for a mix of IO block sizes.

ESG Lab Testing

ESG Lab used the FIO utility to test the sequential read throughput capabilities of a single FlashMAX adapter as it processed IO requests with block sizes ranging from 512 bytes to 512 kilobytes. The results are shown in Figure 3.

What the Numbers Mean

• Much like the horsepower rating of a car, the aggregate throughput of a storage solution is a good indicator of the underlying power of that storage solution’s engine.
• Storage throughput is a measure of the bandwidth available to the system. Throughput can be measured on a stream or aggregate basis. A stream is represented by one application or user communicating through one IO interface to one device. Aggregate throughput is a measure of how much data the storage solution can move, as a whole, for all applications and users.
• Aggregate FlashMAX throughput scaled in a near-linear fashion for the smaller block sizes, shown toward the left side of Figure 3.
• Sequential read throughput reached a peak of 1,394 MB/sec at a 4 KB block size.
• A peak aggregate throughput of 1.394 GB/sec (1,394 MB/sec) is an excellent result for a single PCI flash drive.
• Performance remained predictably high as IO block sizes increased from 512 bytes up to 512 KB, shown toward the right side of Figure 3.

Second Dimension: Sequential and Random Access Patterns

HPC applications can have a variety of workload access patterns including random, sequential, and a mix of random and sequential access. Regardless of the IO access pattern, one key advantage of PCIe flash storage solutions is the ability to perform thousands of times more IOs per second (IOPS) than traditional spinning disk drives or ten times more IOPS than drive form-factor (SATA or SAS) SSDs.

While early adopters of flash storage solutions in the HPC market were initially focused on throughput-intensive sequential workloads, broader adoption in the wider (more horizontal) database, server virtualization, and desktop virtualization markets has begun to take off. Multi-user database and virtualization applications tend to have more random IO access patterns. First-generation flash solutions tended to have different performance characteristics for random and sequential access patterns. FlashMAX SCM provides similar levels of high performance for random and sequential workloads.

ESG Lab Testing

As shown in Figure 4, ESG Lab ran both random and sequential 8 KB reads at four different queue depth sizes to show not only how the access patterns perform, but also how they scale.

What the Numbers Mean

• Online database applications, including those that rely on the latest version of Microsoft SQL Server, are typically composed of a mix of random and sequential IO access patterns, with a block size of 8 KB as tested during this phase of the ESG Lab validation.
• Extremely fast sub-millisecond response times of 38 and 64 microseconds were recorded for 4 KB random write and read workloads respectively.
• The extremely fast response times recorded during ESG Lab testing are significantly faster than a drive form factor SSD. SSDs are slower due to the additional overhead of an IO protocol (e.g., SAS) vs. the low latency of a PCIe bus and the vFAS software advantage with the FlashMAX.
• The extremely fast response times recorded during ESG Lab testing are 25 to 200 times faster than a traditional disk drive.
• The total number of IOPS processed at a single queue depth was slightly higher for the sequential access pattern (14,286 IOPS) compared with the random access pattern (13,716 IOPS).
• At a queue depth size of 16, the random access slightly outperformed the sequential access.
• As queue depth continued to increase, performance eventually leveled out at a little more than 170,000 IOPS for random 8 KB reads and 165,000 IOPS for sequential 8 KB reads. It would take more than 1,000 power-hungry disk drives to deliver 170,000 random 8 KB IOPS.

Third Dimension: Sustained Performance over Time

Solid-state storage solutions have historically had problems maintaining performance over time. Because space is needed to service incoming write requests, a “garbage collection” process needs to run in the background. The goal of the garbage collection process is to free-up necessary space by consolidating written data that was fragmented due to the erase-before-write nature of flash media. Because this process happens along with all of the other user requests, enterprise application performance can be greatly affected. This can be seen by a severe drop in throughput and large response-time spikes. The performance drop is even more severe when flash devices are filled to capacity. The phenomenon is often referred to as a “write cliff,” which aptly describes how write performance seems to fall off a cliff over time. FlashMAX SCM was designed with a goal of providing sustained performance over time and avoiding the write cliff problem.

ESG Lab Testing

ESG Lab tested the ability of a FlashMAX to sustain performance over time. An online database workload was emulated using a mixed 8 KB random workload with a mix of 70% reads and 30% writes. The database was sized to utilize the full capacity of the drive. The duration of the test was set for more than three hours (one hour warm-up followed by two hours of recorded runtime) to allow ample time for the flash device to reach full capacity and potentially be affected by the garbage collection process. The results are shown in Figure 5.

Performance varied minimally over the entire test, and the sustainability is clear. The FlashMAX was able to deliver consistent performance of just under 100,000 IOPS throughout the full duration of the test.

The Fourth Dimension: Scaling Performance with Additional FlashMAX

Scalability, put very simply, is the ability to elegantly handle more work or to physically grow to accommodate that additional work. In this case, scalability applies specifically to the near-linear performance increase that can be achieved with more than one FlashMAX drive installed in a single server. Illustrating this concept, Figure 6 shows how performance and capacity increase in a near-linear fashion with each additional FlashMAX drive that is installed in a server.

ESG Lab Testing

Eight FlashMAX SCM drives were installed in a powerful NEC GX server with fourteen PCI slots during this phase of testing. The Lab used the FIO utility to test 4 KB random read and write workloads. Results are shown in Figure 7.

What the Numbers Mean

• Performance scaled in a near-linear fashion for random read and write workloads as up to eight FlashMAX drives were installed in a single server.
• Performance peaked at an extremely high level of 2.2 million IOPS and 10.6 GB/sec of throughput on a single server.

Fifth Dimension: Real-World, Mixed-Application Workloads

Having looked at the throughput, IOPS, and response-time ratings of the turbo-charged FlashMAX engine, here’s where ESG Lab found “the rubber meets the road” when examining FlashMAX performance with real-world application workloads.

ESG Lab Testing

ESG Lab used the FIO utility to measure the performance of a single FlashMAX card for three application workloads:

• Virtual Desktop Infrastructure (VDI): Designed to emulate a virtual desktop environment composed of heavy knowledge-worker users sharing a common gold image (a.k.a., a linked clone). This workload is composed of 80% 16 KB random writes and 20% 16 KB random reads.
• Online Transaction Processing (OLTP): Order entry and reservation systems are two examples of OLTP applications. Oracle and Microsoft SQL Server are two examples of database applications used to create such OLTP applications. OLTP applications are characterized by a number of users accessing a shared system in parallel. This workload was composed of mostly random reads (70%) with relatively fewer writes (30%).
• Decision Support System (DSS): This workload, also referred to as data mining, emulates a database application that is doing a large-scale random query with a block size of 4 KB. An end-of-month analysis of the effect of a coupon-redemption program on same-store sales is an example of a decision support application.

In order to test the worst-case scenario, the drive was filled to capacity in all three application workload scenarios. Figure 8 shows the throughput scalability of these applications as queue depth increased.

What the Numbers Mean

• Having one outstanding request at a time, the VDI workload was able to achieve 162 MB/sec throughput, and it eventually scaled up to more than 400 MB/sec, with as little as 16 outstanding requests.
• The OLTP simulation reached a maximum of 794 MB/sec throughput at a high queue depth of 256.
• Showing the largest scaling factor, the DSS workload scaled from 117 to 1,352 MB/sec as queue depth increased.
• The performance of a single FlashMAX SCM drive that was recorded during simulated VDI workload testing can be used to support more than 1,000 heavy desktop users.[3]
• A traditional disk array with more than 1,000 disk drives would be needed to deliver the performance recorded during the OLTP testing with a single FlashMAX drive.

ESG Lab Validation Highlights

• Predictable performance scalability with a variety of IO block sizes that peaked at 1.39 GB/sec from a single FlashMAX drive (4 KB sequential reads).
• Predictably fast performance for reads, writes, and a mix of reads and writes for workloads simulating real-world OLTP, VDI, and DSS applications.
• Nearly identical levels of high performance for sequential and random read workloads.
• Sustained performance of 96,825 IOPS over two hours for an 8 KB OLTP workload and drive filled to capacity.
• Up to 345,046 IOPS from a single FlashMAX drive for a 4 KB random read workload.
• Extremely fast sub-millisecond response times (38 and 64 microseconds for 4 KB random write and read workloads, respectively).
• Near-linear performance scalability as up to eight FlashMAX cards delivered up to 2.2 million IOPS and 10.6 GB/sec of throughput on a single server.

Issues to Consider

• The high cost of solid-state capacity compared with traditional hard drive capacity has focused early adoption mainly among businesses whose revenue depends strictly on application performance (e.g., trading applications within the financial industry). As a matter of fact, ESG research indicates that the high cost of solid-state capacity is the number-one objection by organizations that have not yet deployed a flash-based storage solution. As the cost of flash capacity decreases over the next three to five years and performance needs increase, ESG expects that the adoption of PCIe flash drives will grow in the broader horizontal enterprise IT market. This is especially true within server virtualization and desktop virtualization environments with high performance needs. In this case, PCIe flash drives have an economic advantage ($/IOP) compared with traditional hard drives.
• Early adopters considering using a flash-based PCI drive in a server to solve a performance problem with a business-critical, high-performance application should consider the extra costs of installing FlashMAX drives in multiple clustered servers for high availability and failover. Solid-state disk drives installed at the other end of the wire (in a SAN-attached storage array) are a viable alternative for more cost-effective sharing and failover. However, they are typically much slower than a PCI flash drive. A FlashMAX drive that’s installed inside of a SAN-attached disk array could be used to create a simply elegant, highly available alternative that cost-effectively accelerates the performance of tier-0 applications and consolidated virtual server environments.
• ESG Lab ran performance scalability tests on a high-end server that could support multiple FlashMAX drives in a single server. The server being tested needed to have enough PCIe slots to support eight FlashMAX drives, and enough processing power to support the necessary requests to drive each FlashMAX to its limit. To achieve the performance scalability results documented in this report, a high-end server with similar specifications is recommended.[4]

The Bigger Truth

The growing gap between the speed of servers and traditional disk-based storage solutions is causing a number of problems in the data center. Though flash memory solutions have filled this void by serving as an answer for some of the most performance-critical application workloads in recent years, issues still exist. First-generation flash-based storage solutions often have challenges maintaining predictably high levels of performance, over time, for real-world applications with mixed-IO patterns. Sustainable performance over the life of a drive is a particularly vexing challenge due to the impact of background garbage collection processes.

ESG Lab has confirmed that Virident FlashMAX is a next-generation PCIe flash drive that leverages intelligent algorithms to provide high levels of sustained, multi-dimensional performance. Extremely low latencies and high levels of performance were recorded with a variety of workloads. ESG Lab was most impressed with the paucity of “saw-tooth” and “drop-off” performance patterns associated with first-generation PCI flash drives. Scalability testing with up to eight FlashMAX drives delivered extremely high levels of near-linear performance scalability that sustained more than 10 GB/sec of throughput and two million IOPS with a single server.

With a proven ability to deliver predictably fast real-world application performance over the life of the drive, FlashMAX is well suited for the growing number of performance-sensitive OLTP, OLAP, DSS, HPC, and VDI workloads that are migrating from traditional disk drives to high-speed flash memory. ESG Lab believes that Virident, with the FlashMAX family of storage-class memory solutions, has unlocked the potential for affordable, large-scale deployment of flash technology in the modern data center.

Appendix

[2] http://freecode.com/projects/fio

[3] Using a conservatively high estimate of 20 IOPS per user

[4] The specifications for the server used during ESG Lab testing are listed in the Appendix.

Background

IT managers are always on the lookout for ways to improve backup and recovery performance, streamline protection management, and reduce costs. ESG research proves the point (see Figure 1), as backup and recovery remain top spending priorities year after year. In 2011, when asked where they would make significant storage infrastructure investments, more than one-third (36%) of survey respondents planned to invest in backup and recovery solutions.[1] Other top-ten protection priorities include data replication for offsite disaster recovery (24%) and tape replacement (15%).

Improved storage management software tools (21%) and data reduction technologies (18%) are also expected to receive investment money. While these technologies can improve many aspects of IT, they can be applied to data protection tasks specifically to reduce costs.

EMC NetWorker

EMC NetWorker is a well-known and trusted backup and recovery solution that centralizes, automates, and accelerates data protection across the IT environment. It enables organizations to leverage a common platform for backup and recovery of heterogeneous data while keeping business applications online. NetWorker operates in diverse computing environments including multiple operating systems; SAN, NAS, and DAS disk storage environments; tape drives and libraries; and cloud storage. It protects critical business applications including databases, messaging environments, ERP systems, content management systems, and virtual server environments.

Integration with VMware vStorage APIs for Data Protection (VADP). NetWorker supports VADP, VMware’s recommended off-host protection mechanism that replaces VMware Consolidated Backup (VCB). VADP improves performance by eliminating temporary storage of snapshots and enabling support for Change Block Tracking (CBT) as well as improving network utilization and reducing management overhead. NetWorker communicates with VMware vCenter to auto-discover and display a visual map of the virtual environment, streamlining administrative tasks dramatically.

EMC Data Domain. Data Domain systems deduplicate data inline during the backup process. Deduplication reduces the amount of disk storage needed to retain and protect data by ratios of 10-30x and greater, making disk a cost-effective alternative to tape.  Deduplicated data can be stored onsite for immediate restores enabling longer-term retention on disk. NetWorker not only can use Data Domain systems as disk targets, but also can leverage Data Domain Boost (DD Boost) software to achieve faster and more efficient data protection. DD Boost increases performance by distributing portions of the deduplication process to the NetWorker storage nodes and/or application modules so that only unique, compressed data segments are sent to the Data Domain system.

DD Boost also provides visibility into Data Domain system information, and it enables NetWorker to control replication between multiple Data Domain systems while maintaining a single point of management for tracking all backups and duplicate copies.

EMC NetWorker Clone-controlled replication with Data Domain Boost. A key feature available with the integration of NetWorker and DD Boost is clone-controlled replication. Through the NetWorker GUI, administrators can create, control, monitor, and catalog backup clones using network-efficient Data Domain Replicator software. NetWorker also enables administrators to move backup images to a central location where they can be cloned to tape, consolidating tape operations. With NetWorker wizard-based clone-controlled replication, administrators can schedule Data Domain Replicator operations, track save sets, set retention policies, monitor the local and remote replicas available for recovery, and schedule cloning automatically. It also takes advantage of Data Domain’s deduplication, compression, and high-speed replication to reduce data amounts and speed cloning resulting in improved performance and reduced network bandwidth requirements.

EMC Data Protection Advisor. DPA provides unified monitoring, analysis, alerting, and reporting across the data protection environment. It collects information about data protection automatically to inform IT decisions and help administrators correct problems and meet SLAs. The software’s single, integrated view brings simplicity to a complex environment, reduces risk, and helps IT work more effectively. DPA takes volumes of disparate data and turns it into actionable knowledge, enabling organizations to reduce costs by more efficiently managing people, processes, and equipment.

ESG Lab recently tested NetWorker and confirmed that enhancements such as greater integration with Data Domain systems, VMware, and DPA result in faster backup performance, better replication control, and simpler administration across heterogeneous environments. Its enhanced versatility helps reduce both the cost and the complexity of a task that is typically considered to be complicated and expensive.

As a complement to our hands-on testing in the laboratory setting, we also recently spoke with three IT managers using NetWorker, Data Domain, and DPA in heterogeneous physical and virtual environments. Their organizations, which vary in size and in the types of data being protected, include a high-tech research and development organization, a diversified media company, and a government agency.

In the next sections, we let these customers speak for themselves. All customer quotes appear in italics.

Customer #1: A High-tech R&D Organization

ESG spoke with the backup and recovery architect for the worldwide R&D organization of a global technology leader. This architect is responsible for protecting file system and SQL database data, test-build source code, and user-experience test media files for all the R&D teams and departments across the company in all geographies.

Situation

Our organization was established specifically to support the product groups and their mission. A great many spots had not been covered by backups, so we wanted to provide backup services that would have a global span of reach and be able to scale. Our goal is to be 100% virtualized, which means we could be looking at 20,000 to 30,000 discrete virtualized servers in a single environment with a single data zone. We have to scale to meet those needs. Also, we aim to be 100% tapeless.

NetWorker Environment

We have 20 TB of data now, but we are just turning on the environment for the general population. We sized the environment to scale and have 128 TB of committed use for about 2,000 clients. We back up physical and Hyper-V virtual servers. This is just the tip of the iceberg. We plan to expand to other data zones in the future, but we built the NetWorker Server to handle 7,000 to 8,000 clients before we extend it to other zones. Next we will expand to a large lab with about 200 TB across 4,000 clients. We’ll probably be backing up multi-petabytes within the next two to three years.

Right now, we’re a single data zone with a single NetWorker server running at the central site. We built a high-speed environment with 10 Gb connections, tiered storage, fast drives, and flash cache. Storage won’t be a bottleneck as far as getting NetWorker performance. Four storage nodes are attached and run within that data zone. Two of those storage nodes are in a remote location using the 10 Gb connection between our central and remote sites. At the central site, we have a DD880, and DPA runs there as well. Primary data is in the lab environment with two storage nodes. Those are using a DD670 target, so all clients in the labs backup to those two storage nodes.

The entire environment is controlled in the central site 10 miles away. We use DD Boost from the storage node to the DD670, and NetWorker to manage replication from the DD670 to the DD880. So DD Boost clones create our offsite components. The DD670 provides 30-day retention, while the DD880 provides 90-day retention. Data is expired from there. Longer retention will be offered later but with a goal of remaining tapeless. We have multiple VLANS, including a dedicated replication and metadata VLAN.

Why Select NetWorker

We looked at other solutions from Symantec and CommVault. One was too complex to deploy with the number of services we run. With the other, the size of the company was too small: I’m not going to bet our source code on a small company, and their view of support and customer response were not up to my enterprise standards. Also their interface is very complex, while NetWorker’s is intuitive. NetWorker’s footprint is also lighter-weight.

Implementation

Our NetWorker environment (see Figure 3) has been in place for about a year. It took 12 hours to implement NetWorker, Data Domain, and DPA for reporting and chargeback.

Results

We have expanded the data that is protected, incorporating global reach and central management. We are now able to do cost recovery and chargeback. We no longer have to write scripts because we can schedule cloning and pick save sets. The cloning mechanisms are working very well for us.

Customer #2: Diversified Global Media Conglomerate

ESG spoke with the director for international infrastructure and projects at this large global media company. The organization provides services to all international offices—60 locations across all divisions, from film to consumer products, retail stores, and entertainment locations. This customer’s major IT hubs are located in Hong Kong and Tokyo (covering Asia); Paris and London (covering Continental Europe, the U.K., South Africa, the Middle East, and Russia); and Buenos Aires and Mexico City (covering Latin America).

Situation

We currently provide services for more than 15,000 employees. That number will grow over the next few years as we add about 13,000 employees in 160 locations including China. We have focused on consolidating sprawl and reducing costs—because these improvements will drive the agility that enables us to expand at a very high rate. Our offices are not running the same things—they are doing film, theatrical, distribution, consumer products, retail, etc.

The way we are growing, we just cannot have individual backup servers in every location—but we can grow organically using the major hubs in each area, making us nimble enough, for instance, to have a 30-day cycle from signing a lease to opening a retail store.

Just two-and-a-half years ago, every individual office had local tape silos running Backup Exec or NetBackup. We tried to get a harmonized platform with automated reporting, but it didn’t work. Each location had a different offsite backup vendor partner who would rotate tapes and take them offsite. We had no centralization or reporting.

NetWorker Environment

In each [Tier 1] hub [Hong Kong, Tokyo, London, Paris, Mexico City, and Buenos Aires], we have servers, storage, NetWorker nodes, and a Data Domain DD890 or DD880. NetWorker supports everything under the sun: Oracle 10g/11 and Informix databases, plus HP-UX, Red Hat Linux, Windows virtual machines, Solaris, and AIX.

The retail end makes it different, so we have AS400 as well. NetWorker plays well for Wintel, Red Hat Linux, HP-UX, and some Unix platforms.

So we operate in pairs. Anything that is backed up to Data Domain in Hong Kong is replicated to Tokyo and vice versa. Paris/London and Mexico City/Buenos Aires work the same way. All of our Tier 2 and Tier 3 locations are basically fully virtualized and run on Celerra iSCSI for file- and block-based data. That’s all replicated back to the Tier 1 site using Celerra Replicator. At those hubs, we backup and write out to Data Domain. We still use the Data Domain systems as VTLs because it’s the only way to get NDMP (Network Data Management Protocol) out. The Data Domain systems have a split personality: They support the advanced file type devices (AFTD) as well as VTL.

Any OS-based level services we drive through NetWorker. So, we still back up the HP-UX boxes running Oracle 10g RAC onto NetWorker, and with or without DD Boost, ultimately, it all goes to Data Domain. We give DBAs the option of writing SQL and Oracle RMAN directly into Data Domain; we leave Data Domain to age-out the database jobs and provide cloning to make sure that the backups are sent offsite. DPA reports on it all.

Retention policies are anywhere from one week to one year, and they are handled globally. Everything expires either on virtual tape or spinning disk.

We are live on DD Boost too. For all virtualized environments, it comes in from VADP. Then, basically, we write to the storage node, and DD Boost spins it back over. We get the VADP nodes to function as network storage nodes, have DD Boost [running] on it, and then poof, we write back out to the Data Domain systems. That’s [representing] significant savings.

Operationally, we drive all work through DPA, including reporting for Data Domain. That part is still in its infancy. We would like to see deeper reports for Data Domain, but it’s pretty good. So we have a DPA point of presence for Europe, one in Asia, and one in Latin America. They handle all their countries and locations—not just regarding NetWorker, but also Celerra Replicator and RecoverPoint job status.

Why Select NetWorker

EMC came in, did a competitive swap of the sprawling stand alone Symantec-based tape configurations, and consolidated [us] onto a single NetWorker platform. I think the secret sauce really was the reporting functionality that came from DPA.

We looked at IBM Tivoli Storage Manager (TSM) too. All [backup applications] have their pros and cons, but the orchestration from DPA really sealed it together. [It offers us] standardization and automation of reports and the ability to actually blend not just typical NetWorker backup, but also replication through RecoverPoint or Celerra Replication and Data Domain integration. It all flowed into a single DPA screen, and that is the secret sauce for us. NetWorker also has a cleaner user interface.

Results

We are extremely satisfied with the NetWorker and DPA solution. We could not be happier. We have been able to free-up resources within the infrastructure team because we’re not troubleshooting failing backup jobs or failing tapes, and [we are not] having to deal with two different consoles. Our team can focus on more value-added items because [the environment] just works (see Figure 4).

We’ve drawn a line in the sand. We are a Data Domain back-end shop. With this implementation and DPA, we don’t have to log into the Data Domain box just to figure out what’s going on in terms of job successes, failures, high-water marks, times for backup, replication, and cloning between hubs.

We did some DD Boost tests. We saw an immediate 40% reduction in backup times compared to our current AFTD backup and very significant deduplication from the source storage node back to the Data Domain device. Network traffic dropped from 50 megabits per second to 7 Mbps. NDMP is really our problem: We really want NDMP DD Boost functionality!

With NetWorker clone controlled replication, we can actually manage Data Domain replication from within NetWorker. Once we’re done with the setup of the job, we never really have to go back to the NetWorker screen. Everything goes through DPA.

We’ve consolidated from 50 sites down to six. And we have gotten rid of all tape except for legacy systems like AS400. The NDMP backups go to virtual tape, so they are also on the Data Domain system. It’s been a year and a half since we had to do any tape out at all. There’s a very real chance that once we are done with the tape silos’ depreciation cycles we’ll take them out completely.

Our goal is not backup. It’s to provide a better automated DR-based service. We want to automate more and provide better DR and HA capabilities. NetWorker backup happens to be one of the vehicles to actually bring that about.

Customer #3: International Government Agency

ESG spoke with the storage virtualization and data protection team leader for an Asia/Pacific government agency responsible for community services such as child protection and safety, disability services, housing, sports, and recreation.

Situation

There’s a big consolidation process at the moment and a refinement of our tiering and classification of business systems. About 12 months ago, we had a single data center and commissioned a new one. So there is a huge project [underway] at the moment to identify the priority of applications, [determine] how the business wants them protected, better define SLAs, etc.

Originally, we had a single data zone and clustered backup server. Then, we split our data center into DC1 and DC2, and we moved our backup environment into a third small data center (DC3). We had two storage nodes attached to an EMC CLARiiON CX4-480 with about 70 TB of disk, and a Quantum i6000 tape library with 10 RTO-5 drives. The architecture was simple, and the roughly 600 clients were balanced across the storage nodes. They backed up to the AFTDs in the storage nodes, and we used scripts to do cloning. A very small number of clients backed up directly to tape—to preserve PST archives and things like that. We also archive to EMC Centera, and that backs up to tape as well.

NetWorker Environment

We have 90% to 95% of our infrastructure virtualized with about 1,500 virtual servers. We have been running NetWorker for close to 15 years now. We are running the NetWorker server in a non-network zone, so effectively, we have a manual failover for the cluster.

Today, we have three DD890s, one for each data center. The two active data center systems are configured with 64 TB, while the larger one has 128 TB. We split the data zone in two because we grew by acquisition and by the merging of departments, which added about 150 more clients. A new pair of VMware virtual centers became active about nine months ago too, adding another 200 Citrix clients. We were using VCB for that stuff, and have moved to VADP. We may clone that data to tape or just continue to use client-based backups and clone to tape. With our Data Domains, it’s great: We can turn around and do VADP backup as well as client-based [backup], and it doesn’t consume any more storage.

In DC1 and DC2, we deployed a pair of storage nodes for each zone and attached 16 DD Boost devices to those. So there was a single storage node for each data zone in each active data center—four virtual storage nodes in the active data centers. The nodes in DC1 have devices off the Data Domain system at DC2. We back up to our local storage node and send a full data set between the client and the storage node, and then DD Boost sends the data off to the opposing data center. It is deduping the data at that point. DD Boost helps [reduce the bandwidth consumed] on that LAN.

We use NetWorker clone-controlled replication, going from a DD Boost device to a DD Boost device. This gets data from the DC1 and DC2 Data Domain systems cloned to the DC3 Data Domain system. We clone it to tape as well. VSS [monitoring] is pretty much built into anything, so we do use that; we’ve got a couple of instances of the Oracle module in use, and the Exchange and SQL modules too.

In terms of retention policies, daily backups are retained for four weeks on the DC1 and DC2 Data Domain systems; when we clone them to the DC3 Data Domain, we change the retention and browse policies. Weeklies are retained for 12 weeks, and quarterlies for 12 months, all on Data Domain. DC3 is a central place to tape out if we need to, and tapes are stored offsite.

Why Select NetWorker

There is a government push to standardize technologies, so NetWorker and Data Domain were the main [candidates for standardization].

Implementation

Getting it up and running and configuring devices took us an hour. In terms of actually working out how these systems were going to work within our organization and [support] the way we do things, we had to tweak things a little bit—like getting 16 devices to each storage node, and our pools, etc. In terms of deploying devices, it’s easy. We probably had a week’s worth of planning to do, but the actual implementation (see Figure 5) took only a couple of days.

Results

Full backups of our Exchange data used to take 36 to 48 hours, and they’ve come down substantially to about 12 hours with DD Boost. We used to split our full backups into four chunks and stagger them across each weekend of the month. With the Data Domain systems, we’ve collapsed that [process] down to doing full backups on the first weekend only.

We used to run out of space constantly. We had 70 TB of disk available with the CLARiiON and advanced file device. But because we also had media database issues, we were waiting for the network to clear out devices.

Because we don’t have to do that anymore, and because the Data Domain systems are sized appropriately, the backups finish in a timely manner. That means the network can do better database consistency checks, so our whole environment has become more stable.

The other big benefit is that we now have all of our backups online. Our data set is in DC1, initial backup is in DC2, and a clone is in DC3—almost instantaneously. So we have the added comfort of quickly having multiple copies in multiple locations. We still write cloning scripts because the GUI-based tool only filters save-sets by group, client, or pool—not by bus or anything. With small changes, I would start to use it.

When we moved to the new backup environment, we also moved to capacity-based licensing. That has meant we can experiment with virtual storage nodes and other modules, configurations, and features [without having to request a temporary key, with its associated delay and subsequent sales call]. It’s just brilliant in that sense.

The Bigger Truth

Backup/recovery gets a bad rap from most organizations because of the pain and expense it can cause. But whatever its reputation, no one disputes its importance. As a result, backup and recovery are perpetually among IT organizations’ top spending priorities.

Those organizations today face not only continual data growth, but also new backup and recovery challenges related to server virtualization. IT managers look to storage solution vendors to help them:

• Reduce the amount of backup data.
• Find easier and faster ways to complete backups.
• Speed recovery to keep the business in operation.
• Enhance management capabilities.
• And always, reduce data-protection costs. Backups must be done, but no CIO likes spending money on data that is really only there for emergencies.

ESG has been familiar with EMC NetWorker for many years. We completed our most recent NetWorker ESG Lab Validation in July 2011. NetWorker remains a viable solution for diverse computing and storage shops, and recent improvements are designed to enhance data protection for VMware environments (via VADP integration), take advantage of EMC Data Domain Boost (to speed backup performance while leveraging deduplication), consolidate management, and integrate DPA (for reporting and chargeback).

It is worth noting that few vendors would be able to offer such an extensive suite of integrated products as EMC. NetWorker, Data Domain, DD Boost, DPA, Avamar, and RecoverPoint can all be easily combined to deliver a multi-functional data-protection environment.

The diverse set of customers with whom we spoke demonstrates the range of benefits that EMC NetWorker delivers:

• The high-tech R&D organization needed to provide backup services to labs across the globe and scale massively to accommodate a goal of 100% virtualized servers and 100% tapeless backup. NetWorker was chosen over other competitive solutions and is expected to back up multiple petabytes of research data to Data Domain systems in the next few years.
• The media company with 60 worldwide locations could no longer make do with individual backup solutions in each office. With a new installation of NetWorker, Data Domain, DD Boost, and DPA, the company has freed-up staff resources, consolidated sites, virtually eliminated tape, and provided higher availability and automated disaster recovery to its lab teams.
• The government agency and long time NetWorker customer drastically reduced backup times while simultaneously improving its sense of data security.

And in all three cases, these customers reported that deploying the NetWorker solution was quick and easy.

EMC has a history of buying technologies that complement what it already has and further integrating these solutions. Its backup and recovery products today work together to provide an end-to-end solution, from backup software to target devices, leveraging deduplication to reduce data volumes and software such as DD Boost to improve performance and manageability. DPA rolls-up details of backup and replication activities across multiple products so that administrators can keep track of progress in a single location as well as enable departmental chargeback.

You might say that backup administrators have a friend in EMC.

The Stages of Server Virtualization

Many organizations are currently reaping benefits of server virtualization that include lower IT capital and operating costs and greater IT efficiency. Organizations with server virtualization experience are moving beyond these benefits to improve application provisioning, maintenance, availability, and backup/recovery processes. Server virtualization’s benefits appear to be closely correlated to users’ experience and confidence in the technology.

Over time, organizations tend to move through three distinct stages as they deploy server virtualization technology, beginning by virtualizing IT-owned infrastructure applications and utilities like file and print services. The key objectives during this first wave of virtualization are consolidation, manageability, and cost reduction. As organizations gain confidence in virtualization technology, more intensive applications like Exchange, SQL, and SharePoint are consolidated during the second wave of adoption. In this stage, the agility and availability of IT services are often enhanced as IT managers take advantage of virtualization to migrate applications from physical servers to the increased fault tolerance provided within the virtualization layer. As organizations continue on their virtualization journey, ensuring the performance and scalability of tier-1 applications like SAP is a key objective. ESG asked IT professionals what strategies they have implemented to address virtual server performance concerns. As seen in Figure 1, server architecture plays a huge role, with increased memory density and additional CPU capacity being key considerations.

Server Virtualization is a Top IT Priority

Server virtualization is becoming ubiquitous as a strategic initiative for IT organizations around the world. Of the 1,602 respondents to a recent ESG research survey, nearly three-quarters (74%) said their organization currently uses server virtualization.[1] In addition, as part of its annual IT spending intentions research, ESG surveyed senior IT professionals concerning their organizations’ most important IT priorities, and increasing the use of server virtualization was the number one response. Server virtualization is clearly their most important priority for the coming year and beyond, making it the third year in a row that it has ranked at the top of the priority list.^[2] Businesses recognize the value that server virtualization delivers and will continue to make investments in the technology in order to drive further efficiencies in their IT environments.

ESG’s data has confirmed that a massive wave of server virtualization expansion is well underway. For example, while 73% of organizations have virtualized 40% or less of their total population of servers today, 58% expect to virtualize more than 40% of all of their servers 24 months from now. The data also indicates that more of these new virtual servers will be run in production environments—on average, the percentage of VMs run in production will increase from 39% today to 58% within two years.[3]

Given the fact that 59% of organizations surveyed by ESG have not yet virtualized tier-1 applications and that the greatest benefits to be gained with virtualization come with the virtualization of these data-intensive applications and workloads, ESG expects to see an increasing number of organizations tackling tier-1 virtualization.

Overcoming Tier-1 Virtualization Concerns

While server virtualization adoption continues to gain momentum, IT organizations still have numerous hurdles to overcome in order to move closer to a 100% virtualized data center. ESG’s data indicates that many organizations struggle with concerns over performance, technology complexity, integration, security, organizational confusion, and a basic lack of knowledge and skills that they believe will be vitally important when virtualizing tier-1 application environments.

When it comes to virtualizing multi-user tier-1 applications essential to the business, in addition to the general concerns already cited, organizations have additional worries:

• Will the virtualization layer add significant performance overhead?
• Can the virtualized infrastructure scale to continue to meet their needs?
• Can performance SLAs for virtualized tier-1 applications be met with confidence?

Despite the challenges, experienced organizations with more mature virtualization deployments are rapidly moving beyond the initial benefits of consolidation, finding that more extensive use of virtualization can help improve application backup/recovery, bolster application availability, and automate IT processes. They have come to realize that the real metrics that matter in a virtual environment are those focused on availability and performance, and measure the success of their virtualization efforts not only by their ability to reduce costs and increase efficiency, but also by their ability to meet application performance requirements.

Of these more mature organizations surveyed by ESG, 52% cited application uptime and 48% indicated performance among the metrics used to gauge the success of their virtualization deployments, as can be seen in Figure 2.

Many of these early adopter organizations have now virtualized their entire infrastructures, including tier-1 applications like SAP. In other words, application performance is a top criterion for virtualization success that is being addressed by the early adopters that have fully embraced server virtualization.

The HP ProLiant DL 980, Powered by Intel Xeon Processors with VMware vSphere 5

New technologies and offerings from HP, Intel, and VMware seek to address these performance and scalability concerns. The HP ProLiant DL980 G7 is designed to reduce bottlenecks and improve throughput and performance as well as deliver enhanced reliability in an x86 environment.

Scaling up to 80 total cores with Intel Xeon E7 4800 and 7500 Series processors, up to 4 TB of  memory, 16 IO slots, and HP Integrated Lights-Out 3 (iLO 3) remote server management software, the DL980 G7 server is a platform designed for organizations looking for balanced scalability and self-healing resiliency for today’s enterprise compute environments.

VMware vSphere 5 is designed to take advantage of these massive scale-up servers and new processor technology, with support for up to 2048 vCPUs and 2 TB of RAM per host, and support for up to 32 vCPUs and 1 TB of RAM per virtual machine. In addition, VMware has introduced support for multi-core virtual CPUs which allows VMs to use advanced memory management features in server hardware as if it were on a physical machine.

Figure 3 shows how HP, Intel, and VMware offerings can be leveraged to provide a robust, massively scalable virtualized environment for the most critical tier-1 applications running in enterprises today.

A growing number of businesses are looking beyond the initial benefits of increased consolidation and manageability that can be achieved when virtualizing tier-1 application workloads. And yet, as previously referenced, 59% of organizations have not yet virtualized tier-1 applications. Performance concerns are among the reasons most cited as preventing companies from using virtualization more pervasively. The balance of this report summarizes the results of ESG Lab testing designed to evaluate the scalability and performance of a tier-1 SAP ERP application workload running on an Intel Xeon-powered HP ProLiant DL980 server fully virtualized with VMware vSphere 5.

ESG Lab Validation

ESG Lab performed hands-on evaluation and testing of virtualized tier-1 application workloads in VMware’s facilities in Palo Alto, California utilizing HP’s ProLiant DL980 server with VMware’s vSphere 5 running SAP ERP 6.0.  The workload used was designed to emulate a number of different user activities in a sales and distribution environment executing several different transactions.

Getting Started

Figure 4 illustrates the test bed used by ESG Lab. SAP ERP 6.0 was installed in a VMware vSphere 5 virtual machine running on an HP ProLiant DL980 server configured with eight Intel X7560 8-core CPUs and 512 GB of DDR3 system RAM. While the DL980 can support up to 2 TB of RAM today and 4 TB with 32 GB DIMM modules in the near future, 512 GB was more than sufficient for the tests described in this report. Storage services were supplied by an HP P4500 SAN over a 1Gbps iSCSI SAN.

Testing was executed in a repeating loop against an SAP installation where all components (application and database servers) were installed and running on one virtual machine. While the workload was running, different aspects of virtual machine hardware utilization were measured as well as application response time. The testing simulated a sell-from-stock business model and included the creation of a customer order with multiple line items, delivery of the order, shipment, and invoicing.

This workload is resource-intensive and is intended to gauge the expected performance of the tested server platform and configuration in a virtualized SAP environment.

ESG Lab Testing

Figure 5 shows the HP ProLiant DL980 as seen in the vSphere 5 console, with eight 8-way Intel Xeon processors installed and hyperthreading enabled.

Testing was performed against a single virtual machine with 4, 8, 16, and 32 vCPUs. Each virtual machine had RAM allocated in proportion to the number of vCPUs assigned, from 16 GB for the 4 vCPU virtual machine to 128 GB for the 32 vCPU machine, as seen in Figure 6. It’s important to note that vSphere 5 allows administrators to configure the number of virtual sockets, enabling optimized VM memory management. It is also important to note that unlike physical machines, virtual machines can easily have CPU and RAM resources added or subtracted at any time, making the VM easier to dynamically scale to meet increasing or decreasing performance demands.

ESG Lab ran a CPU-intensive portion of the test workload and captured the screenshot shown in Figure 7.

This workload was run in a loop for several minutes. The virtual machine distributed the load quite evenly across all 32 vCPUs. It’s important to keep in mind that while most workloads don’t require the maximum VM configuration of 32 vCPUs and 1 TB of RAM, vSphere 5 was able to manage large memory and vCPU allocations in a VM quite well, and it’s reasonable to expect similar results for configurations with larger RAM configurations.

Performance and Scalability

ESG lab next ran the full workload simulation on the SAP ERP 6.0 server, starting with four vCPUs and scaling up to 32 vCPUs, capturing multiple key metrics such as the number of users, IOPS, vCPU utilization, and application response time as the workload was increased.

ESG Lab Testing

The number of users simulated, IOPS driven, and response time as well as the average CPU utilization while the workload was running were monitored as a scalable number of users were emulated on a single physical server virtualized with VMware vSphere 5. The results are summarized in Figure 8 and Table 1.

Also shown in Table 1 are results captured using a 32-core physical machine ESG Lab tested in 2009[4] using the same workload.

What the Numbers Mean

• A real-world OLTP ERP workload was emulated using SAP application and database servers running in one virtual machine.
• A single virtual machine scaled from 4 to 32 vCPUs as simultaneous SAP users were increased from 600 to 3,000, proving that SAP systems running on VMware can now easily scale as an environment grows to meet the needs of the largest SAP systems.
• Average CPU utilization was remarkably consistent through the entire range of tests, illustrating vSphere 5’s ability to balance the load evenly across multiple cores and sockets as the size of the VM and the workload were increased.
• Dialog response time, representing the average application response time to users for all transaction types, ranged from .1 seconds to 1.1 seconds during the test for the virtual machines.
• ESG Lab compared these results with results obtained with a 32-core physical server tested using the same SAP configuration and transaction types and found that the vSphere virtualized HP ProLiant DL980, while driving approximately two-thirds of the users and IOPS of the tuned physical server, did so with half the dialog response time, which represents the delay users experience waiting for a transaction to complete.
• Considering that the SAP workload tested utilized only half of the CPU and one quarter of the available RAM installed in the DL980 tested, it is not unreasonable to expect that a single DL980 could easily support a second virtualized SAP workload at a similarly high utilization level and/or multiple less intensive workloads driven by other applications.

Virtual CPU utilization and dialog response time were monitored to confirm that vSphere 5 was distributing the load smoothly across all 32 vCPUs during testing and that transactions were being satisfied within accepted response time guidelines. Virtual CPU utilization was evenly distributed while transaction response time was low during the 4 vCPU test and remained low during the 8, 16, and 32 vCPU tests. The efficiency of VMware vSphere 5 running on a powerful HP ProLiant DL980 attached to a 1 GB iSCSI disk array with fast 15K RPM drives provided more than enough horsepower to support a very large and busy SAP server.

ESG Lab Validation Highlights

• An HP ProLiant DL980 physical server, running a tier-1 SAP ERP 6.0 implementation deployed within one VMware vSphere 5 virtual machine, scaled nearly linearly in performance as vCPUs and memory were increased—up to a total of 32 vCPUs in a single virtual machine.
• CPU utilization and response times remained low throughout testing, indicating significant headroom to support larger workloads with increased storage bandwidth and system tuning.
• A sustained CPU-intensive workload was evenly distributed across all 32 vCPUs in the virtual machine.

Issues to Consider

• While the scalability of both vSphere 5 and the ProLiant DL980 server are extensive (up to 1 TB per VM, 2 TB per host server), the tests ESG is validating were done without taking full advantage of the capabilities of this combination solution, using only 512 GB of memory in the server. ESG believes the performance of VMs with maximum allocations of memory running on the platform will be consistent with the results validated in this report.
• Capacity planning and performance analysis of SAP deployments is recommended to not only determine if your organization’s workload is suitable for virtualization, but also to plan the processor, memory, storage, and network resources that need to be configured within each virtual machine.
• SAP application and database server roles are both good candidates for virtualization. For database implementations at the very high end (ie:32 vCPUs in a single virtual machine),users would be well-advised to consider deploying application servers on virtual machines separate from resource-intensive database instances.
• The test results presented in this report are based on a simulated workload deployed against an SAP ERP 6.0 installation in a controlled environment. Due to the many variables in each production data center environment, capacity planning and testing in your own environment is highly recommended. Sizing SAP deployments is always under the purview of the hardware partner; work with your HP team for sizing, performance, and capacity planning.
• Default server BIOS, operating system, SAP, and SQL Server settings were used during ESG Lab testing. As expected after any performance test of this magnitude, analysis of the results indicates that tuning would probably yield higher absolute results. Given that the goal of this test was not to generate a big number, ESG Lab is confident that the results presented in this report demonstrate the performance and scalability of tier-1 application workloads running in consolidated VMware vSphere 5 environments.

The Bigger Truth

Server virtualization is being deployed by a large and growing number of organizations with the ambitious, parallel goals of lowered costs, improved resource utilization, non-disruptive upgrades, and increased availability. Each of these benefits is fundamentally enabled by decoupling servers, applications, and data from specific physical assets—bringing IT organizations one step closer to the ideal of a completely virtualized IT infrastructure. A recent ESG survey indicated that increasing the use of server virtualization was users’ number one IT priority over the last two years and will continue to be the top priority for the next 12-18 months. But while server virtualization continues to gain momentum, some IT organizations are still reluctant to fully embrace virtualization.

It’s a common strategy for organizations to focus server virtualization efforts on consolidation until they build the confidence and expertise to consider the next tier of applications; ESG found that 59% have yet to employ virtualization where it will provide the most benefit: their data-intensive tier-1 applications. For IT organizations supporting large numbers of users, this hesitance stems from the perception that it adds performance overhead and unpredictable scalability and availability challenges to the tier-1 applications relied upon by the majority of their users.

Multiple vendors currently offer server virtualization solutions and the field continues to expand. Choosing a solution set to best fit a specific environment, business need, or budget can be a significant challenge. HP, Intel, and VMware answer this challenge with massively scalable server platforms built on powerful, virtualization-optimized processors and robust, scalable, easy-to-use virtualization software which integrates seamlessly with HP server management software such as HP Insight Control for ProLiant.

The HP ProLiant DL980 platform is a dense, scale-up industry-standard server powered by the latest Intel Xeon processor technology and  designed to support the largest and most challenging virtualized applications and workloads with the added goal of fast, easy deployments that users can customize to grow computing power in the data center on demand.

VMware vSphere 5 virtualizes servers with the goal of transforming traditional x86-based IT infrastructure into an automated, always-on computing environment. Customers have reduced IT infrastructure costs while streamlining management of IT environments and delivering improved service levels to the business. vSphere 5 builds on the robust tools and capabilities of VMware, including the capability of creating virtual machines with up to 32 virtual CPU cores and true NUMA functionality, while VMware High Availability and Fault Tolerance capabilities integrate tightly with HP to offer business continuity.

ESG Lab hands-on testing has confirmed that HP, Intel, and VMware technology can be used to meet the performance and scalability requirements of virtualized tier-1 workloads. Predictably low response times and near linear performance scalability were achieved during testing as a single server hosting a virtualized SAP ERP deployment within one vSphere 5 virtual machine supported up to 3,000 simulated SAP users. Extremely low application response times were observed in an un-tuned environment, implying significant headroom to support more users when the environment and application are tuned for performance. Considering the large amount of unused resource capacity in the DL980 under test, ESG Lab believes that a single DL980 could easily support a second virtualized SAP workload at a similarly high utilization level and/or multiple less intensive workloads driven by other applications.

Virtualizing SAP and other tier-1 applications’ workloads with the HP ProLiant DL980, powered by Intel Xeon processors in combination with vSphere 5, enables businesses to overcome scalability and performance concerns as they lower costs and increase the agility and availability of a consolidated IT infrastructure. With the ability to host multiple, large-scale virtual machines with very large memory footprints inside a single physical server, administrators can support a very large tier-1 application environment in a very small data center footprint.

It’s clear to ESG Lab that the complementary technologies offered by HP ProLiant DL980 servers, Intel Xeon processors, and VMware vSphere 5 virtualization software can be used to effectively support vital enterprise workloads like SAP, providing the reliability and performance customers require while simplifying operations, lowering costs, and providing tier-1 class availability—all in a virtualized x86 environment.

Appendix

[2] Source: ESG Research Report, 2011 IT Spending Intentions Survey, January 2011.

[3] Source: ESG Research Report, The Evolution of Server Virtualization, November 2010.

[4] Configuration details can be found in the Appendix.

[5] Source: ESG Research Report, The Evolution of Server Virtualization, November 2010.

Background

Organizations need to be able to respond to rapidly changing market dynamics, customer requirements, and competitive threats. Many are building out private cloud infrastructures in order to achieve the requisite level of agility. One of the foundational technologies of a private cloud is server virtualization, so it shouldn’t be a surprise to learn that increasing the use of server virtualization technology has been the top IT priority for the last two years in a row according to ESG research.[1] This, in turn, is driving increased growth in networked storage environments, which are critical to enabling virtualized deployments to be highly mobile and available. In fact, ESG research indicates that the increased use of SANs was the number one impact server virtualization has on storage environments.

This rapid growth in server virtualization and SAN deployment is affecting the management of these increasingly complex and abstracted environments. In most cases, organizations end up changing management tools and processes. And this goes beyond just the server environment to include storage and network environments (see Figure 1).[2]

As organizations continue to evolve their environments and deploy highly virtualized cloud infrastructures in production, easy to deploy, simple to use management tools that provide end-to-end configuration and performance visibility will become increasingly critical.

EMC ProSphere

EMC ProSphere is an end-to-end storage resource management application designed for quick, easy, and cost-effective deployment to enable a rapid return on investment. With ProSphere, storage professionals can create a detailed view of their storage infrastructure from the host, through the SAN, to the backend storage environment.

ProSphere is designed to be deployed in a VMware environment as a virtual appliance or vApp. For ease of deployment, it’s packaged as an OVF template the can be downloaded from the EMC support website. To further simplify and reduce the management burden, it leverages an agent-less discovery and data ingest paradigm.  ProSphere gets its performance data, alerts, and end-to-end visibility using standard protocols like SNMP and SMI- S for switches and arrays and WMI, SSH, or vCenter for physical and virtual hosts.

• ProSphere is designed to be easily deployed as a virtual appliance in a VMware environment.
• The discovery and data ingest processes are agent-less, leveraging SMI-S, SNMP and manufacturer APIs.
• ProSphere can be integrated with Microsoft AD and the LDAP protocol.
• Customers with active EMC ControlCenter support contracts can use ProSphere at no cost.
• EMC ControlCenter performance data can be imported into ProSphere.
• Administrators can manage multiple ProSphere instances without logging into each site separately.

ESG Lab Validation

ESG Lab performed hands-on evaluation and testing of ProSphere at an EMC facility in Hopkinton, MA. Testing was designed to demonstrate how EMC ProSphere intuitively provides end-to-end resource management for storage environments. Also of interest were ease of use and implementation as well as agent-less host, SAN, and array data collection.

The configuration used for ESG Lab testing is shown in Figure 3. On the left side of the figure are the three VMware virtual machines that made up the EMC ProSphere operating environment. These VMs consist of the Historical Database, Discovery Appliance, and the Storage Resource Manager. On the right side of the figure are the test bed objects that were discovered and monitored during the ESG Lab testing. EMC VMAX and CLARiiON arrays were discovered, as were Cisco and Brocade SAN switches. The host environment consisted of physical and virtual Windows servers as well as Solaris 9 and Solaris 10 servers. A LAN with multiple VLANs provided connectivity between the ProSphere environment and the objects being monitored.

Getting Started

Getting started with EMC ProSphere begins with obtaining and installing the OVF template packaged code to bring the environment to the initial login screen. This was accomplished by downloading the ProSphere application software from the EMC web support portal[3] and installing the application into an existing VMware environment.

ESG Lab Testing

ESG Lab began testing ProSphere by copying the downloaded OVF formatted application code to the test VMware vSphere environment. ESG Lab used the “Deploy OVF Template” option to select and install the ProSphere application. Figure 4 shows the summary of entries including hostnames, networks, and other common system configuration settings that were selected as part of the template-driven install process. The settings were confirmed and the “Finish Install” tab was selected. The ProSphere installation process completed in approximately 30 minutes and resulted in the creation of a virtual application with three virtual machines.

ESG Lab used the vSphere client to verify the creation of the application. Figure 5 shows the ProSphere application (ESG-ProSphere) and the three associated virtual machines (Discovery Appliance, Historical Database, and Storage Resource Manager). “Power on” was selected at the virtual application level to start the three virtual machines in the correct order.

Next, ESG Lab performed an upgrade of the ProSphere application using VMware Studio as shown in Figure 6.

In preparation for the ProSphere vApp upgrade, ESG Lab configured a CD/DVD drive to the VMs. This enabled access to the ISO formatted upgrade image. VMware Studio was then launched from a browser for each VM to conduct the upgrade.

Finally, ESG Lab connected to the ProSphere vApp using the browser-based GUI to validate login access and confirm proper operation. Figure 7 shows the Administration view from the ProSphere user interface. From this view, user account and security certificates can be added and managed. The Administration view also enables integration configuration for LDAP and Microsoft Active Directory.

Configuration and Discovery

Configuration is the process of setting up ProSphere to ingest data for performance analysis and storage environment alerting. Discovery is the process of identifying and ingesting performance and operating metrics as well as configuration data for the storage environment components or objects including hosts, switches, and storage arrays. With ProSphere, this process does not require an agent on each discrete object. It enables the administrator to create custom end-to-end views of the storage environment.

ESG Lab Testing

ESG Lab launched the ProSphere user interface and selected the Discovery icon to start the discovery configuration process. The first step in setting up each discovery job was defining its access credentials. Access credentials are a set of properties that define how ProSphere will connect to an object on the network.  These property sets contain permission settings for security and define the connection type used for data discovery.  Any access credential can be designated as global. Standard global access credentials can be defined and used for similar object types to streamline the discovery process by allowing a discovery job to automatically find and pick an appropriate, previously defined global credential.  This process separates the creation and management of access credentials from that of discovery jobs.  Server administrators can manage their credentials within the ProSphere environment while storage teams can then use those credentials to manage discovery jobs and meet security requirements for access management.

As shown in Figure 8, the Access Credentials tab was selected from within the discovery view. ESG Lab leveraged this configuration tab to set up discovery permission for a number of objects in the environment. Discovery access was defined for EMC storage arrays; Brocade and Cisco SAN switches; and Windows, UNIX, and VMware hosts.

Next, ESG Lab configured a number of different discovery jobs. As shown in Figure 9, ESG Lab configured a job to discover an EMC storage array. First, the scope of the job was set by selecting the object type and where on the network to look. In this case, the hostname of the server running the EMC Array SMI-S was selected, along with the object type of array.  Then the previously-defined access credentials were selected for the new discovery job. The last step was defining the discovery schedule.

It should be noted that a range of IP addresses can also be used for discovery to help simplify the process in large dynamic infrastructures such as rapidly growing virtual environments. This option also allows for excludes to be set for objects that a user does not want to include in discovery.

Figure 10 shows the results of a successful discovery job for host type objects. Here, ESG Lab selected the “Discovered Host” tab from the objects list view. Also available are switches, arrays, and fabric level object views.

Finally, ESG Lab configured groups of objects. As shown on the left of Figure 11, ESG Lab configured three groups (ESG Lab Windows Servers, ESG Lab ESX, and ESG Lab Physical Hosts). The groups were used to create more manageable views. Large environments could have hundreds or thousands of servers, making the objects list view difficult to use when searching for a specific object of interest. Groups can also be used to view objects with inter-dependencies such as a group of servers or the server, switch, and storage that support a specific application.

Three group classifications are available in ProSphere. The three groups are defined as system groups, simple groups, and smart groups. System groups are automatically created by the application to group objects by type. Simple groups can be created and used for a collection of objects that rarely change.  Objects must be manually added to simple groups.  Smart groups can be used to dynamically scan and group objects based on user-defined parameters. With smart groups, objects can be properly grouped for analysis based on business requirements such as mission-critical application or utilization for bill back purposes.

Monitoring, Alerting, Troubleshooting, and Analysis

Performance monitoring is the ability to select, view, and analyze detailed information for objects in the storage infrastructure. ProSphere can create end-to-end topology views and detailed performance charts. Alerting is the ability to monitor SAN and storage objects in the infrastructure by setting thresholds and notification. Combined, performance monitoring and alerting help accelerate the identification and remediation of performance bottlenecks and component failures.

ESG Lab Testing

ESG Lab explored the performance capabilities of ProSphere by selecting a virtual machine in the storage environment and analyzing the supporting components from the host through the storage. Figure 12 shows a topology view from the virtual machine on the left side of the figure all the way through the SAN in the middle to the storage hosting the VM datastore on the far right. The high level overview was created by selecting specific objects of interest from the host object list. ESG Lab used a drill-down process from this view to analyze detailed information of each component. Hosts, switches, and arrays in the topology view can be expanded into a detailed view by clicking on the desired icon.

In addition to simply selecting items from the object lists, ProSphere has sophisticated search capabilities. ESG Lab used the intelligent filtering feature of the search functionality to quickly and easily set views into different objects without having to scroll through the entire list to find a specific object.

Intelligent search functionality becomes more important as the environment grows organically or through the federation of multiple ProSphere sites. Geographically separated ProSphere implementations can be federated through synchronization, a process that creates a synchronized data set between sites. The search feature can retrieve data in a federated environment eliminating the need to log in to each site and enabling users in different locations to perform analysis and troubleshooting of remote environments.

Next, ESG Lab reviewed the performance metrics of the VM selected in the topology view. As shown in Figure 13, ESG was able to analyze and review resource utilization, response times, and performance characteristics for the virtual machine. Similar views are provided for physical hosts as well.

As shown in Figure 14, ESG Lab also monitored performance data on the EMC array containing the datastore for the selected VM. ESG Lab noted a number of storage performance metrics (e.g., response time, IO, and throughput) that can be used to quickly analyze the health of the supporting array.

Lastly, ESG Lab used the alerts tab to configure thresholds and then monitor storage and switch metrics in the test environment. As shown in Figure 15, utilization levels were set to flag warnings and critical conditions when a defined percent was exceeded. Alert levels can be set for the following metrics:

• Switch port link utilization
• Symmetrix port utilization
• Symmetrix host director utilization
• CLARiiON SP utilization
• CLARiiON SP % dirty pages

Setting up and monitoring thresholds allows for easy, at-a-glance identification of performance issues in the storage environment. A quick review of the alerts page will show if a switch or array component in the environments has exceeded defined parameters. ProShpere can then be used to view storage component mapping and identify and analyze physical hosts or virtual machines that might be impacted by changes in SAN performance.

ESG Lab Validation Highlights

• The installation process for ProSphere was quick, easy, and intuitive. ESG Lab downloaded the application images from EMC’s website, moved them to the VMware environment, and easily navigated through the OVF template install process, which took about 30 minutes.
• ESG Lab validated the efficiency of the agent-less discovery process driven by ProSphere. Without installing a single agent, ESG Lab was able to easily set up and discover over 100 physical and virtual hosts, four storage area networks, and six storage arrays. This was accomplished by leveraging SMI-S and SNMP for storage and switch objects and WMI, SSH, and vCenter connectivity for physical and virtual hosts.
• With ProSphere, ESG Lab confirmed that EMC can offer a comprehensive topology view from the host to the LUN on the supporting array, enabling end-to-end visibility of resources for the storage or system administrator.
• ESG Lab confirmed the ability to set up alerting for switch and storage components in the monitored environment. Thresholds can be set that notify the administrator when critical or warning conditions occur in the environment, allowing corrective action to be quickly taken to remediate the condition.

Issues to Consider

• It should be noted that currently, the historical database cannot be pruned. Best practice should dictate monitoring disk capacity for this application component. Also, to avoid corruption, the historical database requires an orderly shutdown. EMC recommends that ProSphere be started and stopped from the vApp level to maintain database consistency.
• Special attention should be paid to best practice recommendations when using VMware snapshots in federated ProSphere environments. If one site in a synchronized environment is rolled back to a snapshot, an error will be returned when users try to access discovered object detail. To avoid this, each site must be rolled back to a corresponding snapshot.
• Since EMC ProSphere was in its first GA release at the time of the validation, ESG expects the application to gain new features over time and hopes to see mixed object group views at the dashboard level.

The Bigger Truth

In an effort to respond more quickly to changing business needs, IT is transitioning to highly virtualized and cloud environments. To ensure a successful transition, storage administrators need to leverage management software that was specifically built for these new data center paradigms. This software should include comprehensive management of the networked storage environment and provide visibility into the virtual server environment, as these previously siloed technology domains are now more tightly integrated and interdependent.

ESG Lab validated that EMC ProSphere’s efficient architecture enables it to be deployed easily, accelerating the time to deliver value for organizations transitioning to cloud environments. It is able to provide end-to-end views and the performance information required to ensure applications will run in an optimized environment while meeting demanding SLAs.

ESG Lab believes EMC has made it easy for clients currently running ControlCenter to test and then migrate to ProSphere thanks to its ability to ingest ControlCenter performance data. EMC has also allowed ControlCenter clients with active maintenance contracts to deploy ProSphere at no cost.

As organizations evolve their highly virtualized infrastructures into scalable cloud solutions, they will need management solutions that can adapt to these dynamic and agile environments. By designing and architecting management software for the Cloud Computing Era, EMC enabled ProSphere to effectively manage and optimize cloud storage environments.

Appendix

[2] Source: ESG Research Report, The Evolution of Server Virtualization, November 2010.

[3] https://powerlink.emc.com

Background

Server virtualization has and will continue to have a major impact on IT organizations. According to ESG research, it is the top IT priority for organizations over the next 12-18 months (see Figure 1). Of the 611 senior IT professionals surveyed, 30% said their organization’s most important priority was the increased usage of server virtualization. What’s more, this is the third consecutive year in which server virtualization has topped the priorities list.^[1] Organizations continue to realize the significant value server virtualization offers as they more effectively and efficiently utilize existing IT resources.

Citrix XenServer

Citrix XenServer is a virtualized server platform built on the powerful, open-source Xen hypervisor. This enterprise-ready and cloud-proven solution allows for the quick creation and efficient management of virtual infrastructures to positively impact an evolving data center. By enabling increased IT infrastructure agility and efficiency, XenServer allows for dynamic adjustment of VM resources as well as flexible VM placement within an infrastructure.

XenServer is a full, virtual solution, helping companies achieve increased administrative efficiency through data center automation, business continuity through higher reliability, and the extension of an IT infrastructure by leveraging the cloud.

• Automation. XenServer consolidates server workloads to help optimize existing hardware utilization by reducing power, cooling, and management costs. The automation of key IT processes can also help save time and money by improving application service delivery for IT administrators.
• Adaptability. XenServer dynamically optimizes server workloads to increase performance and utilization. Adding new applications becomes easier and faster without sacrificing existing performance necessities and application requirements.
• Reliability. Application downtime or disruption is unacceptable for many businesses. Keeping an infrastructure up and running and protected from disaster is imperative for efficiency and productivity. With XenServer, virtual servers and physical hardware are separated and therefore easier to protect than in a physical environment, greatly reducing organization downtime.
• Cloud Readiness. IT infrastructure flexibility is becoming a requirement. This is especially true as more organizations move to the cloud to lower costs and increase IT service responsiveness. XenServer’s flexibility allows for quick customizations to fit any business.

ESG Lab Validation

ESG Lab performed hands-on evaluation and testing of XenServer at a Citrix’s Bedford, MA facility. Testing was designed to demonstrate ease of physical to virtual server migration, the ability to support enterprise applications in a virtual environment, and the manageability and optimization of virtual machine performance.

ESG Lab Test Bed

ESG Lab focused its testing around physical and virtual application performance and manageability. The goal was to attain similar performance in a virtual environment when compared to a physical environment. Both environments ran individual and simultaneous application simulations to not only show real world performance possibilities and results, but also the resource and performance management capabilities of XenServer 6.0. The test beds used by ESG Lab are shown in Figure 3 and Figure 4. Figure 3 shows the Intel server—running Window Server 2008 R2—with 64 cores used for physical server testing. The server was connected to a dual-controller LSI storage array via a one gigabit iSCSI connection.

Figure 4 shows the same physical machine, but now it is running XenServer, allocating eight cores and 24 GB of memory to four separate virtual machines running Windows 2008 R2 Enterprise Edition. Each virtual machine was connected to the LSI storage array via a 1 GbE iSCSI connection. Each VM was allocated storage space on the LSI array and assigned drive letters accordingly.

Conversion

Citrix has a physical-to-virtual conversion tool, named XenConvert, which assists in the shift to a virtual environment. XenConvert takes Windows operating systems, applications, or data from a physical server and converts them to virtual machines within XenServer as virtual appliances or virtual disks.

ESG Lab Testing

After completing the testing required with the physical test bed, ESG Lab moved to the virtual test bed. ESG Lab used XenConvert to convert the physical test bad to a virtual test bed. ESG Lab noted that XenConvert’s easy-to-follow wizard-driven interface provided guidance during the conversion. Figure 5 shows a XenConvert interface. This interface was used to provide storage space allocation details for a particular virtual environment.

Four VMs, all running Windows Server 2008 R2, were created on the XenServer. ESG Lab maintained the integrity of all applications and external storage during the XenConvert process. The change to the virtual solution was smooth and quite impressive.

Performance

Application performance can change when moving from a physical to a virtualized environment due to a potential for different resource utilization. It is very important for an IT organization to verify similar application performance in both environments to insure the move to a virtual environment is acceptable.

ESG Lab used three industry standard tools to simulate four different workloads. The application simulations were run both individually and simultaneously in each test bed. The goal was to find the total throughput impact when switching from physical to virtual servers, as well as response time changes due to new performance overhead introduced by the Xen hypervisor.

• Microsoft Exchange Server Jetstress 2010 is a Microsoft tool used to simulate disk IO on an exchange server. Jetstress 2010 works with the Microsoft Exchange Server 2010 database engine to simulate Exchange database and log disk input/output load. Jetstress simulates the Exchange database and log file loads produced by a specified number of users. [3]
• SQLIO is a Microsoft tool used to determine the IO capacity of a given configuration. ESG Lab used an 8KB random IO workload with this tool.[4]
• Iometer is an open source workload generation tool. ESG Lab used Iometer to generate two simulated applications: a backup reader and a web server. The backup reader workload mimics the behavior of a backup application during a restore operation with a default block size of 64 KB. The web server workload is meant to mimic the IO activity of a web server, such as Apache. [5]

ESG Lab Testing

In the physical testing scenario, all four workloads ran on a single physical server to different parts of an LSI 2600 storage array. The virtual testing scenario differed in that each of the four virtual machines was designated one workload each. The storage remained the same between the physical and virtual test cases. Figure 6 shows the results of running four applications simultaneously in both environments.

Figure 7 shows the throughput of running the backup reader simulation as an individual application in both environments.

What the Numbers Mean

• As seen in Figure 7, the total throughput for all workloads running simultaneously is 8% lower on virtual machines than on a physical machine.
• Individual application throughput while running with other applications varies between the two test cases due to different resource utilization. This can be seen when comparing throughput of the backup reader and web server.
• Figure 7 shows the results of running backup reader as a single application. The physical test bed yielded 15% more MB/s than the virtual test bed.

ESG Lab ran an 8K random, read workload using SQLIO. Figure 8 shows the latency differences between the virtual and physical environments.

Jetstress was configured to emulate 1,200 users with 200 MB mailboxes performing typical Exchange operations at an IO rate of 0.12 IOPS per mailbox. A response time goal of 20 milliseconds or less for database reads is required to pass the test. These values are defined by Microsoft as a limit beyond which end-users will feel that their e-mail system is acting slowly.[6] Figure 9 shows the latency differences from running Jetstress as an individual application in both environments.

What the Numbers Mean

• Faster response times are seen on the physical test bed, as opposed to 10% degradation in the virtual environment. This can be seen in Figure 8.
• Figure 9 shows the average read latency of the Jetstress testing. Response times increased slightly, but not to the extent that users would notice. Physical and virtual response times were within the 20ms Microsoft guideline.

ESG Lab installed Microsoft SQL server on a physical server and virtual server and timed two different SQL scripts. Before running the scripts, a table was created within a duplicated production database from ESG’s internal IT operation and a large number of rows were added to the new table. The first script was a basic SQL query selecting everything from the newly-created table. The second script contained an SQL query consisting of a three table join. Each script was timed and the average duration of each script was recorded. Table 1 shows the results from the basic SQL query and the more complex SQL query.

Manageability and Optimization

Citrix XenServer is managed by the XenCenter graphical user interface. Dynamic memory control and performance monitoring tools give XenCenter the flexibility to effectively manage a XenServer deployment. These features allow IT professionals to recognize where and when resource limitations are occurring, and arm them with the ability to adjust resource usage for continued, acceptable performance in a virtualized environment.

Dynamic Memory Control

Dynamic Memory Control (DMC) is a tool available in XenServer Advanced Edition and can be managed through XenCenter. Whether powering on more virtual machines within a physical server or consolidating virtual machines during business downtime, DMC gives IT professionals much-needed flexibility to manage their XenServer resource pool environments.

As seen in Figure 10, ESG Lab configured three virtual machines with static memory settings. The first two VMs were powered on with their static memory settings fully utilizing all of the memory allocated to the resource pool. The third VM was powered off.

Figure 11 shows the results of attempting to power on the third virtual machine. Due to the fully utilized resource pool memory and the first two VMs having static memory settings, ESG Lab was unable to power on the third VM.

ESG Lab then configured the first two VMs with dynamic memory settings by setting a minimum and maximum memory requirement as shown in Figure 12.

Finally, Figure 13 shows the results when ESG Lab powered on the third VM after adjusting the memory settings of the first two VMs to utilize DMC. The used memory of the first two VMs was automatically adjusted to allow the third VM to power on.

Performance Monitoring

XenCenter gives administrators the ability to monitor, in both real-time and historically, the way system resources are being utilized in the virtual infrastructure. Performance data about each XenServer and each virtual machine within a XenServer is automatically collected. These results can be seen at various levels of granularity, from seconds and minutes up to one year, within XenCenter.

Figure 14 shows an example of what to expect when viewing the performance tab for a particular VM. Three graphs were displayed, showing statistics about CPU utilization, network traffic, and memory usage.

Adding new performance graphs was quite intuitive. ESG Lab selected the “configure graphs” button to present a list of possible statistical counters. After selecting the desired counters, a graph was then added to the overview screen and real-time monitoring began.

ESG Lab Validation Highlights

• The migration from a physical server to a XenServer was quick and easy. ESG Lab was able to quickly create multiple VMs on the XenServer through XenCenter with minimal effort.
• The virtual environment running the simultaneous application simulations yielded 8% lower IO/s than the physical environment. ESG Lab believes this to be more than acceptable with all of the other advantages XenServer offers.
• Dynamic Memory Control was validated by ESG Lab by attempting to power on a static-memory-allocated virtual machine with two other virtual machines utilizing all of the allocated memory. The memory usage was able to dynamically adjust to allow the third virtual machine to power on.
• ESG Lab was able to use XenCenter’s performance monitoring tool to view current resource utilization including CPU, network traffic, and memory usage during application simulations. This included the ability to add new counters and graphs while monitoring the existing counters in real time.

Issues to Consider

• Testing of Citrix XenServer exposed ESG Lab to the XenCenter management console. The console provided a single pane of glass, multi-server management interface for the XenServer environment. It is, however, a Windows application only and must be deployed on the Windows platform; this must be taken into consideration when planning deployment of a XenServer environment.
• ESG Lab used the XenConvert tool to migrate from a physical environment to a virtual environment. This tool, like XenCenter, is limited by its operating system dependency: Windows. That being said, XenConvert can create virtual Linux environments from a physical Windows environment.
• With features like XenConvert and Dynamic Memory Control, ESG Lab found that Xen server provided real flexibility and agility for migrating and consolidating physical servers and applications into a virtual environment. However, it does not eliminate the need for proper planning and best practice implementation. A properly sized and architected virtual infrastructure, along with a clear understanding of minimum resource requirements, is critical to avoiding stability and performance issues while meeting customer SLAs.

The Bigger Truth

Application performance in a virtualized infrastructure continues to be a challenge faced by IT professionals.  Performance bottlenecks remain a concern, specifically with IO-intensive, enterprise applications. With the increased use of virtualization continuing to be a top IT priority,^[7] the movement of high-throughput-demanding applications to a virtual solution will become a necessity rather than an option. Coupled with a continued need for better resource utilization while continuing to cut costs, a major dilemma quickly presents itself. Where should the sacrifice come from? Performance? Cost? Reliability?

The reduction of operational costs while continuing to meet demanding business requirements is a mandate for many businesses. Consolidating server workloads, dynamically optimizing workloads, and disruption-free reliability are key criteria when selecting a more flexible virtual platform to help leverage your existing infrastructure. Maintaining acceptable application performance while satisfying the aforementioned criteria creates a recipe for an ideal IT infrastructure. Through hands-on evaluation and testing, ESG Lab has confirmed that Citrix’s XenServer has all of the capabilities necessary to address each of these challenges in a simple, affordable way.

Tools, such as XenConvert, make the convergence to a virtual environment a breeze, with immediate accessibility to your virtual machines upon completion. The manageability provided by XenCenter creates an easy, navigational view between a virtual infrastructure and administrator. The ability to see resource utilization over time and in real time gives IT professionals the ability to be proactive when bottlenecks begin to present themselves. With dynamic memory allocation, XenCenter can provide a better understanding of how applications consume resources without sacrificing resource integrity.

ESG Lab has verified that acceptable performance scalability was attained using XenServer, both at a throughput and a response time level. Minimal degradations are expected when moving from a physical environment to a virtual environment, and those seen during ESG Lab’s evaluation are well within expectations. ESG Lab has confirmed that Citrix XenServer is a powerful, cost-effective, integrated server virtualization solution.

Appendix

[2] See: ESG Lab Report, Citrix XenServer: Complete, Cost-effective Data Center and Server Virtualization, September 2010.

[3] http://www.microsoft.com/download/en/details.aspx?id=4167

[4] http://www.microsoft.com/download/en/details.aspx?id=20163

[5] http://www.enterprisestrategygroup.com/using-esg-lab-workloads/

[6] http://technet.microsoft.com/en-us/library/bb738152(EXCHG.80).aspx

[7] Source: ESG Research Report, 2011 IT Spending Intentions Survey, January 2011.

Background

With continued economic uncertainty, it’s no surprise IT organizations are looking for ways to reduce data center costs without impacting business continuity and performance. In a recent survey, ESG asked IT professionals to name the greatest challenges being faced by midmarket organizations with respect to storage environments. As shown in Figure 1, the top-two responses, both at 38%, were cost-reduction initiatives and maintaining an acceptable pace with overall data growth.[1]

ESG research also shows that the increased use of server virtualization is a top priority in the IT industry over the next 12-18 months.[2] How will businesses reduce cost and improve business processes? To consolidate IT infrastructures, organizations are using server virtualization, which helps reduce the complexity that comes with managing data growth in an efficient, cost-effective way. This ESG Lab Report documents and validates the NetApp FAS2240 entry-level solution as a means to easily keep up with the ever-growing data in a data center at an affordable price, as well as to counter the complexity of managing that data.

Introducing the NetApp FAS2240

The FAS2240 is a new entry-level storage system offered by NetApp. With increased performance, the inclusion of ONTAP Essentials, and configuration flexibility, the FAS2240 can help storage professionals increase storage system utilization and efficiency. The product’s affordability, best-in-class technology, adaptability, and manageability help customers start right with simple-to-manage storage that can grow to meet business demands.

ESG Lab evaluated the FAS2240, focusing on quick deployment, simple manageability, and scalable performance. Figure 2 demonstrates how the FAS2240 fits into any small to medium-sized business’s data center.

The FAS2240 comes in two options: 2U and 4U. The 2U option uses SAS drives internally, and the 4U option uses SATA drives internally. Both allow up to 24 internal disks, and both leverage the same new controller, which substantially boosts performance. The internal drive storage configurations for a single shelf range from 5.2 TB up to 72 TB. These options provide flexibility depending on the needs of the environment, which may be more performance or more capacity oriented.

The standard FAS2240 configuration comes integrated with four 1 GbE ports for front-end connectivity and two 6 Gb SAS ports per controller for back-end expandability. An optional flexible IO port, which can contain dual 10 GbE Ethernet or dual 8 Gb Fibre Channel ports, is also available. Also, the FAS2240 now offers highly reliable and scalable cluster-mode configurations to the SMB market space. The FAS2240 is based on NetApp’s existing disk shelf design, which enables customers to convert the FAS2240 to a shelf in a mid-range or high-end storage system if more performance or space is required.

NetApp now offers its existing mid-range and high-end software features in low-end systems. Included in the entry-level bundle are ONTAP Essentials and all protocols. Coupled with a new pricing structure, the FAS2240 is channel-friendly and affordable.

OnCommand System Manager 2.0

OnCommand System Manager is a simple, powerful tool used to manage NetApp storage. By merging the pre-existing System Manager, FilerView, and Cluster Element Manager into a single management solution, the new tool makes it easy to manage everything from a consolidated web interface. System Manager automates the discovery of all NetApp storage, simplifying the setup, protocol management, and configuration of a NAS and SAN. Figure 3 shows an example dashboard view of OnCommand System Manager, including storage capacity and availability, as well as configured aggregates and volumes.

OnCommand System Manager aids in implementing storage best practices from a single interface. Improved storage utilization and space savings are possible through the use of the product’s advanced storage capabilities including deduplication and thin provisioning. Real-time views of system health are visible through a performance dashboard.

NetApp’s System Manager monitors storage system health and performance, and it manages all storage components, which allows IT administrators to increase system efficiency and strengthen business continuity.

ESG Lab Validation

ESG Lab performed hands-on evaluation and testing of the FAS2240 at the Sunnyvale, CA, NetApp facility. Testing was designed to demonstrate how quick and easy it is to deploy and manage a new FAS2240 and to assess its ability to support real-world workload performance requirements for small and mid-sized organizations.

Start Right

“Starting right” is the ability to deploy a storage solution that can be easily managed, changed, and scaled to meet the ever-changing business requirements of modern IT environments. Figure 4 shows the test bed used by ESG Lab for the validation of the NetApp FAS2240. The left side of Figure 4 shows the Windows and Linux hosts used for testing.  The right side of Figure 4 shows the multi-protocol (file- and block-capable) FAS2240 storage array that supported the lab validation host environment. The center of Figure 4 shows the OnCommand System Manager that was used to set up and configure the test environment from a “single pane of glass” management interface.

ESG Lab Testing

ESG Lab testing began right after the FAS2240 was rack mounted and provided with power and physical network connections.  The initial boot and configuration was initiated from a console connection on the FAS2240. As part of this process, ESG Lab assigned simple configuration information, such as hostname, IP address, subnet, and gateway. In less than 10 minutes, the FAS2240 was on the network and ready for more advanced configuration tasks.

As shown in Figure 5, ESG Lab next launched the OnCommand System Manager interface and used the discovery feature to identify the FAS2240 filer configuration. After the test filer was discovered, ESG Lab leveraged the System Manager interface for all other validation, configuration, and management procedures.

ESG Lab configured the FAS2240 with its controllers in active/active mode as an HA pair. Figure 5 shows the test bed filer detail, including HA pair controller partners, IP address of each controller, and their serial numbers. ESG Lab first configured the filer to support multi-protocol NAS testing with both CIFS and NFS. Later, the same filer was converted to iSCSI block storage to support mixed workload performance testing.

With a 24-drive FAS2240 base configuration, ESG Lab divided the available disks into two sets of equal size with a hot spare in each set. One disk set was configured to Controller A and the other to Controller B. The creation of an aggregate is shown in Figure 6. Using RAID DP configured disks, two aggregates were created. Each aggregate was configured with 11 physical drives. For ultimate flexibility and scalability, two dynamically sizeable volumes (FlexVols) capable of supporting multiple protocol block and file devices were deployed on each aggregate.

Keep It Simple

“Keeping it simple” is the design philosophy built into the OnCommand System Manager interface. With this new interface, storage administrators can leverage built-in configuration wizards to simplify common provisioning tasks. The intuitive navigation of OnCommand System Manager, combined with its wizard-driven configuration, helps turn otherwise complex storage procedures that often require specialized skills into a straightforward, guided process.

ESG Lab Testing

ESG Lab used configuration wizards for the majority of validation tasks. As shown in Figure 7, the Create LUN Wizard was used to build the iSCSI LUNs that were presented to the Windows 2008 host for the workload testing detailed later in the “Grow Smart” section of this document.  Figure 7 shows each step ESG Lab completed to create the 500 GB LUN used to host the Microsoft Exchange workload testing. These steps were repeated for each independent-workload-associated LUN.

Next ESG Lab created a volume and CIFS share on one of the aggregates that was configured in the “Start Right” section of this document.  Again, a configuration wizard was used for both the volume and network share setup. On the wizard-created new test volume, ESG Lab was able to specify a host name, specify security type of NTFS, and join an existing Windows domain via active directory using the CIFS creation wizard.

The new CIFS share was then connected to the Windows 2008 host in the test bed environment. To validate the successful presentation of storage to the Windows host, multiple test files were created, updated, and copied between a local drive and the new share.

Next, as shown in Figure 8, the test volume was edited to enable mixed-mode security. ESG Lab then created an NFS export on the same volume with the original CIFS share. The new NFS export was mounted on a Linux host in the test environment.

As shown in Figure 9, the VI (visual interface) editor was used from the Linux host to create and edit the “editor.txt” file on the mixed-mode test volume. The same file was updated on the Windows 2008 host using the Notepad text editor. With mixed-mode security enabled, the same file was accessible from both the Windows host via CIFS and the Linux host using NFS. As shown in Figure 9, the update made with Notepad was confirmed on the Linux host using the VI editor.

Finally, ESG Lab used OnCommand System Manager to create and test a snapshot copy.  As shown in Figure 10, a test volume was selected from the Volumes tab for a snapshot copy. The Snapshot Copies dropdown menu (highlighted in red) was selected, launching the volume snapshot configuration wizard. ESG Lab selected the option to make the snapshot visible as part of the configuration process. This option allows the snapshot directories to be browsed from the host.

A snapshot was taken for a volume with a CIFS share. From the Windows host, ESG Lab confirmed access to the snapshot directory.  After the snapshot was created, a file was deleted from the primary share directory on the Windows host.  ESG Lab used the snapshot directory to recover the deleted file from the snapshot to the primary directory using a simple drag-and-drop process.

Grow Smart

“Growing smart” is the ability to provide a flexible architecture that supports the growing and changing business and application demands of an IT environment. The FAS2240 accomplishes this by delivering storage capacity and performance in a multi-protocol, unified solution. Starting with a complete 2U enclosure, the FAS2240 can scale from 24 to 144 drives. Should capacity requirements exceed this total, then the existing enclosures can be repurposed as disk-shelves in a larger storage system.

ESG Lab Testing

ESG Lab tested the performance of a FAS2240 by scaling simultaneous applications focused in two main areas: (1) acceptable email response time, and (2) user scalability, as applications were added to the mix. ESG Lab selected four applications common to small and medium-sized businesses: email, web server, file server, and database. Performance tests were run using two industry-standard tools: Jetstress for the email application and Iometer for the other three applications.

Jetstress was configured to emulate 500 users with 400 MB mailboxes performing typical Exchange operations at an IO rate of 0.12 IOPS per mailbox. A response-time goal of 20 milliseconds or less for database reads is required to pass the test. These values are defined by Microsoft as a limit beyond which end-users will feel that their email system is acting slowly.[3] Figure 11 shows the email response time as applications are added.

What the Numbers Mean

• The red dotted line at the top of the chart represents the acceptable 20 ms response-time cutoff guideline recommended by Microsoft for the Microsoft Exchange application.
• The response time ranged between a low of 9.656 ms for Email and Database together and a high of 11.134 ms with all applications running together, which ends up being a difference of approximately 15%.
• All scenarios are well under the 20 ms latency guideline, which implies there is room for even more applications to run simultaneously without severely impacting response time.

Next ESG Lab calculated the user counts each application could support. By taking the overall throughput of each application and dividing it by the average number of IOPS per user,[4] [5] [6] ESG Lab determined how many users could be supported for each application simultaneously. As shown in Figure 12, the active user count grows in an upward trend as additional applications are layered into the solution.

What the Numbers Mean

• Both email and web services easily support 500 users for each application.
• With file services, 380 active users can access a network-attached shared home or corporate directory.
• Database applications such as Oracle and Microsoft SQL Server can support 144 users.
• 1,524 total active users of four different applications were supported by a single FAS2240.

ESG Lab Validation Highlights

• ESG Lab brought a new FAS2240 system online in less than 10 minutes and evaluated the ease of creating controller-centric aggregates and volumes to present to a physical server.
• ESG Lab explored the NetApp OnCommand System Manager and validated the easy-to-use, wizard-driven setups as well as the intuitive, browser-based management capabilities. This included the creation and presentation of CIFS and NFS file shares over a 1 GbE network, allowing both a Windows and a Linux host to create and edit files.
• ESG Lab successfully scaled the environment to four common business applications. While running the applications simultaneously, the Exchange workload simulation easily remained within response-time requirements. (The maximum response time of 11.134 ms was witnessed as all four applications ran simultaneously.)
• ESG Lab scaled the active user count from 500 users using an email application only, up to 1,524 users using email, file services, web services, and a database.

Issues to Consider

• It should be noted that the standard controller configuration mode for the FAS2240 is active/active. This means that the drive assignment will be split between controller one and controller two. This requires due diligence in implementing best practices, especially in smaller configurations, as this can limit the number of spindles available when creating an aggregate. If larger aggregates are required, the FAS2240 can be configured in active/passive mode, but only through the command line interface.
• Using the snapshot functionality included at no additional cost in the Data ONTAP Essentials software bundle, ESG Lab explored the ability to easily create snapshot copies for individual file recovery. This was accomplished using the Snapshot Creation Wizard option, which made the snapshot directory structure visible and available at the host level. Features such as SnapRestore , SnapMirror , SnapVault and FlexClone are also available for the FAS2240 as software options that extend functionality beyond the Essentials bundle. These options are competitively priced and can be added individually or added as a group via the SnapManager Suite.
• For those environments that need to support a high performance application, ESG Lab was pleased to see that the FAS2240 currently offers SSD drive support via the optional expansion disk enclosures. We believe this feature would be extremely valuable if it were extended to base FAS2240 enclosure and hope to see this option in a future release.

The Bigger Truth

Small and medium-sized businesses are faced with many of the same storage challenges as their counterparts in the enterprise. Explosive data growth, backup and recovery challenges, and flat to declining IT budgets are pervasive—especially in mid-sized businesses and distributed enterprises with undertrained IT staff. The NetApp FAS2000 Series was designed from the ground up to address those challenges. ESG Lab confirmed that the NetApp FAS2240 is easy to deploy, simple to manage, and able to cost-effectively grow to the meet the needs of the business.

A FAS2240 with prices starting under $16,000 was tested by ESG Lab. Keeping it simple with the slick graphical user interface provided by the new OnCommand System Manager, we were able to efficiently configure storage and implement NetApp storage-management best practices through a single interface.  The ONTAP Essentials package included features that enabled seamless and flexible scaling. Not only did the bundled software features provide flexibility, but the hardware did too: It can be repurposed into larger NetApp storage solutions as capacity and performance requirements exceed the SMB space.

The ability to grow smart with enterprise-class functionality and performance was tested with a mix of applications that are commonly deployed in mid-sized businesses and distributed enterprise environments. These workloads included email, database, web, and file services. Starting with a 500-user Exchange environment that maintained performance levels that are well within Microsoft’s recommended guidelines, response times stayed manageably low as additional application workloads were tested simultaneously. We were able to support practical user counts as business applications were added to the test environment.

ESG Lab feels the FAS2240 is packaged well with a robust family of high-end features and functions that are simple to use and priced right for mid-sized organizations and their channel partners. IT managers within small to medium-sized businesses and distributed enterprises would be wise to consider the benefits of starting right, keeping it simple, and growing smart with the NetApp FAS2240.

Appendix

[2] Source: ESG Research Report, 2011 IT Spending Intentions Survey, January 2011.

[3] http://technet.microsoft.com/en-us/library/bb738152(EXCHG.80).aspx

[4] Source: http://aws.amazon.com/ebs/

[5] Source: http://www.cisco.com/en/US/solutions/collateral/ns340/ns517/ns224/ns955/ns964/netApp_refDG.pdf

[6] Source: http://www.vohkus.co.uk/ipexpo/downloads/FlexPod_Data_Ctr_Sizing_Guide.pdf

Background

The underlying search engine must scale in performance and manageability as archives get bigger, and they will get bigger. ESG estimates that organizations will archive over 44,000 petabytes of e-mail by 2015 (see Figure 1).[1] When including file and database data, the total capacity increases to nearly 303,000 petabytes. Examining just e-mail archives, the average deployment saw a 200% increase in storage capacity between 2007 and 2010.[2]

Search engines have to keep up in terms of performance for index and retrieval while remaining easy to manage. An archive should not increase back-end operational cost or complexity—the search engine is often a prime driver of those pitfalls.

Symantec Enterprise Vault 10

Symantec’s version 10 of Enterprise Vault (EV10) provides a new 64-bit index for archived items across multiple sources. Content can be archived into a single repository where the data is indexed, deduplicated, and compressed, helping to reduce storage costs and backup window time requirements.

Enterprise Vault 10 (EV10) for Exchange offers the following key technology features:

• Optimized indexing. Mail items archived in EV10 are indexed with a 64-bit index vs. a 32-bit index with earlier versions.
• Integrated end-user experience. Enterprise Vault works seamlessly with Outlook clients for Windows and Mac, OWA, and mobile devices to make searching archived items quick and painless.
• Simplified installation and administration. Administrators can leverage wizard-driven installation along with a management program that consolidates all archiving, indexing, and policy functions into a simple to use tool.
• File system archiving filterpoint. Customers can deploy filters to enable archiving and retention decisions to be made on the content of the file, and not solely on file metadata.
• Data classification. Organizations can meet e-mail management policies by applying retention and expiry rules across multiple classes of e-mail. Unique tags can also be applied to e-mail to help speed up discovery requests, providing a more intelligent, proactive approach to searches for relevant content.
• Cloud storage support. Leveraging Symantec’s OpenStorage API archived e-mail can be stored in the cloud with Nirvanix’s Storage Delivery Network.

ESG Lab Validation

ESG has witnessed several different search engine upgrades and replacements, and few vendor commitments work “as advertised.” As such, ESG Lab put Symantec’s two Enterprise Vault search deployment options to the test, performing hands-on evaluation and testing of EV10 at Symantec’s facilities in Mountain View, California. Testing was designed to examine the consistency and integrity of searches in both federated and fully upgraded indexes.

ESG Lab used ten Microsoft Exchange 2007 user mailboxes containing e-mail imported from PST files from the publicly available Enron data set to test the indexing integrity between versions 9 and 10 of Enterprise Vault. As shown in Figure 3, one server was used to host the Exchange mail system. A second server hosted a Microsoft SQL server used by Enterprise Vault to maintain system configuration information. The third server contained the Enterprise Vault archive indexes of the Exchange mailboxes. ESG Lab used a Microsoft Terminal Services connection to a Windows 7 workstation to control the Enterprise Vault administrative tool and to manage the upgrades and perform searches on the indexes.

Federation Across EV Indexes of Different Versions

With large indexes involving thousands of users, it is often impractical to complete a full upgrade in a short maintenance and outage window. For this reason, Symantec offers a federated model that allows two or more indexes of different versions of Enterprise Vault to work together to provide a unified search. During the upgrade, the current index of data remains intact and an index with the new version is created. After the upgrade, all new data is sent to the new 64-bit index, but data is searchable across both indexes. Since there are separate threads for index updates and searches, administrators can upgrade data incrementally to the new index during normal business hours or maintenance windows, whichever they prefer.

ESG Lab Testing

ESG Lab completed numerous tests with consistent queries to compare the search results of 32-bit and 64-bit indexes. The sequence of tests is as follows:

1. A test run in an Enterprise Vault 9 (EV9) system to set a baseline for search results for the queries.
2. A test run in a federated system consisting of both 32-bit and 64-bit indexes to compare the query results with the baseline test from EV9.
3. A test run in a fully upgraded EV10 index to compare the query results with the baseline test from EV9.

The expected outcome from the multiple tests is a consistent result using the same queries across all tests. If the data is consistent in all environments, the tests are considered successful.

EV9 Baseline Test

ESG Lab tested the integrity of data in the federated model to ensure that the search results produced in EV9 are consistent with results in the federated indexes. In order to set a baseline for search results, ESG Lab performed four queries in EV9 using the Enterprise Vault Advanced Web Search program through a standard web browser. Figure 4 shows the search parameters available with Enterprise Vault.

ESG Lab used the following parameters for the four baseline searches.

Query 1: Subject contains any of natural gas explosion

Results will return archived e-mail items and attachments that contain either “natural,” “gas,” or “explosion” in the subject line.

Query 2: Subject contains any of natural gas explosion, File Extension .xls

Results will return archived e-mail items that contain either “natural,” “gas,” or “explosion” in the subject line and have a spreadsheet attachment.

Query 3: Content contains phrase price fix*

Results will return archived e-mail items and attachments that contain any phrase starting with “price fix” in the content of the e-mail.

Query 4: Content contains all of “natural gas” settlement

Results will return archived e-mail items and attachments that contain the phrase “natural gas” and the word “settlement” in the content of the e-mail.

Table 1 represents the results with the number of e-mail items found with each search test.

Federated Test

After the initial baseline searches, ESG Lab upgraded the EV9 software to EV10 in a federated configuration. With the new EV10 system, all existing archived e-mail remained in the 32-bit EV9 index. Any new archived e-mail was indexed in the new 64-bit EV10 index.

In order to compare results, ESG Lab imported the same ten user mailboxes into Exchange with an extension “-ev10” added to the mailbox name. Figure 5 shows the indexes in a mixed EV9 and EV10 federated environment.

When ESG Lab updated the software to EV10, new 64-bit indexes were created for each mailbox. However, the data still resided in the 32-bit indexes. As ESG Lab imported the same ten mailboxes with the “-ev10” extension into the EV10 system, indexes were created with the same number of items as the 32-bit indexes, essentially creating identical mailboxes in both EV9 and EV10 indexes for direct comparison.

With identical mailboxes in both 32-bit and 64-bit indexes, ESG Lab expected to see identical search results when querying across a federated environment.

ESG Lab then repeated the four search queries on the federated system with the results listed in Table 2. As the results indicate, the searches in the 64-bit indexes returned the same number of items as in the 32-bit indexes in the federated environment.

Full Upgrade to EV10 Index

Once Enterprise Vault software is upgraded to version 10, a new 64-bit index is created for each mailbox archive. The old EV9 index still exists for the existing archive until an upgrade on that index is performed. Administrators have the option to upgrade the entire index at once, usually in a single maintenance window, or incrementally. For large organizations, the incremental approach is required due to the sheer volume of mail items that cannot be entirely upgraded during a maintenance window. When upgrading incrementally administrators can choose any number of user mailboxes for each upgrade task scheduled. The index process can be stopped and restarted at any time and the process will resume where it left off.

ESG Lab Testing

ESG Lab tested the consistency of data searches after a full upgrade of EV9 indexes to EV10. To begin, ESG Lab selected the ten mailboxes in the 32-bit index and created an indexing task to upgrade the mailbox archives, and started the task immediately. Figure 6 shows the progress of the upgrade of all ten archives to EV10.

While the upgrade was in progress, ESG Lab tested the system’s search capability to ensure that performance would not degrade and to verify that the data returned was still consistent with earlier results in the federated system.

Since the 32-bit index was still searchable during the upgrade, ESG Lab selected the EV9 mailboxes and performed the second search test. Figure 7 shows the mailboxes selected for the search.

When compared to the second test in Table 1, the results shown in Figure 8 match the results from earlier tests.

ESG Lab also performed additional random searches on the index while the upgrade was in progress and found that search performance was not negatively impacted from a user perspective.

After the upgrade completed, ESG Lab repeated the four search tests on the new 64-bit indexes. As the results in Table 3 indicate, the items returned in the four tests were identical to the initial baseline tests run in the EV9 index.

Index Manageability

Enterprise Vault boasts many improvements in its latest version, including data classification services, archive to the cloud, support for Mac Outlook clients, and enhancements in management functions, particularly indexing tasks. ESG Lab looked specifically at the management functionality and its improvements over EV9. Indexing is now fully managed by the Administration Console, which includes a new Manage Indexes wizard and Monitor Indexing Tasks page.

ESG Lab Testing

ESG Lab examined the new management functions for the index. The most significant difference ESG Lab observed was the ability to manage indexes through one utility. In the Enterprise Vault administration tool, ESG Lab navigated to the indexing section, which showed a summary for the available index. ESG Lab was able to access the utility by either right-clicking the indexing item or selecting the “Manage Index” tab on the indexing summary page, as shown in Figure 9.

This utility enables users to perform index management tasks, allowing them to rebuild, synchronize, or verify the health of an index. ESG Lab chose the Upgrade option shown in Figure 10.

With EV10, indexing tasks can be scheduled, much like mailbox archiving tasks, allowing administrators more flexibility in setting up indexing functions to be performed during maintenance windows. ESG Lab looked at this capability when performing an upgrade on the ten user indexes from EV9 to EV10. As Figure 11 shows, ESG Lab was able to create a schedule in the properties page of the “Index Administration Task,” choosing dates and times that the task would be run.

ESG Lab Validation Highlights

• Running in a federated environment, Enterprise Vault 10 was able to return consistent results with searches performed in an EV9 system.
• After upgrade of EV9 indexes to EV10, search results were consistent with those obtained in EV9 and federated environments.
• Upgrading to EV10 was simple and intuitive. The deployment scanner provided a proactive approach to analyzing any prerequisites that would need addressing before an upgrade.
• Indexing functions were consolidated into one utility, making management of the indexes straightforward.

Issues to Consider

• Enterprise Vault introduced the “ItemGranularity” feature in its index schemas in EV 6.0 based on numerous requests from customer feedback. The ItemGranularity schema combines the information from attachments into the top level parent document in the index, which is the actual mail item. This results in greater search efficiency. However, the net result is that attachments are not shown separately in the search results. Searches with the Enterprise Vault browser or integrated search features will return top level document results only and not individual attachments as separate search results. Therefore, a search which hits on multiple attachments for the same message would return different results when searching against a 32-bit EV9 index and a 64-bit EV10 index, since EV9 does not have ItemGranularity enabled by default for mailbox archiving. For example, if searching the 32-bit index in one query returns 12 hits that include eight attachments from four distinct mail items, the same query in EV10 64-bit index would return four hits because Enterprise Vault only returns the four distinct parent messages. In order to test for consistent results across the two versions of Enterprise Vault, the ItemGranularity option must first be turned on in EV9 and the indexes must be rebuilt so attachments are not separate items in the search results.
• EV10 has the ability to search on items using wildcards combined with just one matching letter. EV9 required at least three letters with a wildcard to return matching items. As a result, using the search parameter “price f*” in EV10 could return any item that started with that phrase. However, EV9 would return no results since it isn’t supported as a valid search option. This is a functional difference between 32-bit and 64-bit indexes. So, when operating in a federated environment where there is a mix of 32-bit and 64-bit indexes, a query using “price f*” would return results starting with that phrase in the 64-bit index but no results in the 32-bit index.
• Customers may notice that the default sort order of query results has been changed to Date Descending rather than relevance which may affect the look of query results when comparing EV9 and EV10 archives.

The Bigger Truth

Current archive market trends indicate that the next wave of innovation will center on archive access. The current size and projected growth of archives coupled with organizations’ desire to search and utilize the retained information, amongst other drivers, are spurring the enhancements and evolution of many products including Symantec’s Enterprise Vault 10. With this version, available to the market as of July 2011, Symantec upgraded its search engine to establish a foundation for faster queries, contextual presentation of query results, and smaller index footprint as archives continue to get larger.

Introducing a new search engine into an archive creates plenty of opportunities, especially as it pertains to making information more accessible and usable to the business. However, there are also potential risks as upgrading data to a new search engine requires an upgrade and re-index process or the need for federation where a new search engine can query previously indexed data. Symantec has committed to both options: a non-disruptive upgrade from a previous EV version to 10.0 or a federated deployment where existing indexes created by an older version of EV are searchable via a 10.0 implementation.

ESG Lab tested indexing in both a federated configuration and a fully EV10 index, and found the results returned from multiple queries consistent with the previous EV9 results.

Symantec has shown that it understands the upgrade concerns of IT organizations and has provided a strong solution that not only preserves the integrity of archived data, but eases the burden for organizations with large archives by offering an incremental upgrade path to EV10.

Appendix

[2] Source: ESG Research Report, E-mail Archiving Market Trends, May 2010.

Background

Recently, ESG asked IT managers to name their top IT priorities over the next 12-18 months and the top two responses were to increase the use of server virtualization and to manage data growth.[1] When asked about their spending plans for storage infrastructure specifically, purchasing new SAN storage systems was high on the list, implying that many organizations are “managing data growth” by purchasing new storage. With IT under constant pressure to find ways to reduce costs and address data growth without interrupting business operations, rapid deployment of storage and IT resources to meet increasing demand becomes a necessity.

As businesses challenge IT to provide dynamic, flexible services, the need to move resources fluidly from one location to another and provide universal access becomes essential. As server virtualization becomes ubiquitous, moving data and applications needs to be fast and non-disruptive. Decentralizing services across multiple data centers in various locations demands the ability to extend the storage network infrastructure across the WAN.

The QLogic iSR Series

The QLogic iSR series multi-protocol storage routers are based on QLogic’s dual-blade TrueFlex architecture. The QLogic iSR addresses the challenges presented by highly virtualized, dynamic data centers with a highly available fabric routing and application hosting platform that enables data migration services, multi-protocol routing, and SAN-over-WAN fabric extension. The iSR series utilizes initiator virtualization technology to provide a “virtualization layer” of storage connectivity for large numbers of physical and virtual servers. The dual-blade architecture is designed to offer the highly available, high performance flexibility required to support business and mission-critical applications for organizations of all sizes.

Benefits provided by the QLogic iSR series include:

• TrueFlex architecture – provides any-to-any protocol flexibility and connectivity while the modular design enables easy upgrades to new protocols or technologies.
• Open fabric interoperability – supports heterogeneous existing networks, SANs, and WANs.
• Enterprise-class high availability – dual hot-swap power supplies and router blades provide no single point of failure.
• High performance – 8Gb Fibre Channel ports deliver up to 6.4GB/sec throughput and over 400,000 IOPS.
• Comprehensive Heterogeneous Data Migration – is supported across all major FC, iSCSI, and FCoE storage arrays, networks, and operating systems in an online, offline, local, or remote migration.
• Initiator virtualization technology – provides support for up to 1,024 virtual machines using 2,048 iSCSI initiators and 8,192 LUNs in a dual-blade iSR6260 configuration.
• Simultaneous iSCSI and SAN-over-WAN connections – using the same Ethernet ports reduces the cost of storage consolidation and distance connectivity for dynamic reconfiguration and simplifies ongoing management.
• Ease of implementation and management – installation and configuration wizards enable fast setup and simple administration.
• Data scrubbing – provides the ability to wipe existing data residing on a source LUN after migration for data confidentiality or re-purposing of the drive. Data scrubbing implements US Department of Defense methodologies including the data pattern written and the number of passes required.
• Remote migration – Online Remote Data Migration capability provides the ability to move datasets between data centers, simplifying management and dynamic reconfiguration.
• Disaster Recovery Capable – Online Remote Data Migration, combined with built in WAN optimization provides the ability to keep production data sets consistent with copies at remote locations.

ESG Lab Validation

ESG Lab performed hands-on evaluation and testing of the QLogic Intelligent Storage Router in a QLogic facility in Aliso Viejo, CA. Testing was designed primarily to validate the QLogic iSR6250’s ability to provide non-disruptive multi-protocol data migration services in a heterogeneous storage environment. Areas of focus included ease of setup and administration, performance, and migration from traditionally provisioned storage to thin provisioned storage.

Getting Started – Integration and Management

The test bed used by ESG Lab is illustrated in Figure 3.[2] One VMware ESX server hosting a Microsoft Windows 2008 Enterprise Server was connected to a heterogeneous Fibre Channel SAN using a dual port QLogic 2500 8Gb Fibre Channel HBA. Two onboard 1GbE interfaces were used for connectivity to an iSCSI SAN. Fibre Channel storage for these tests was provided by an HP 3PAR F200 storage system and an EMC CX3-20 storage array. A Dell EqualLogic PS 4000 iSCSI SAN provided iSCSI storage. One QLogic iSR6250 was used for testing, with two 8Gbps Fibre Channel ports and four 1Gbps Ethernet ports.

ESG Lab Testing

ESG Lab began with the Windows 2008 server mounting volumes from the EMC CX3-20 over the Fibre Channel SAN.   First, the QLogic iSR6250 router was integrated into the environment. After physical installation, the next step was configuration of the iSR’s IP information through the management port. Then the QLogic SANsurfer Router Manager utility was installed on the virtual Win2k8 server. QLogic SANsurfer can be installed on any system with IP network access to the iSR router and supports Windows, Linux, and Macintosh operating systems. ESG Lab was able to connect to the iSR6250 through QLogic SANsurfer, which provided a clear view of the status of the router, including ports (shown in Figure 4), volumes, and replication sessions.

ESG Lab then drilled down to the port information from QLogic SANsurfer (shown in Figure 5), and validated link status, port speed, and iSCSi network settings.

ESG Lab continued the iSR integration process by zoning the iSR ports to the server and storage, presenting LUNs from 3PAR and EMC storage arrays to the iSR. The iSR automatically discovered LUNs from both the EMC and 3PAR arrays, discovered multiple paths for each LUN and represented the 3PAR F200 and EMC CX as two independent storage arrays.  iSR integration continued by setting the source and destination arrays (as shown in Figure 6), populating the Virtual Port Groups (VPGs) with the LUNS to be migrated, presenting the  LUNs to the host from the  iSR,  and finished with insertion of the iSR6250 into the host path.

Prior to insertion of the iSR, ESG Lab generated a continuous stream of IO to the volumes being migrated using Medusa Labs Test Tools Suite to simulate Microsoft Exchange 2010 database traffic as shown in Figure 7.

ESG Lab verified that IO continued to the devices with no errors or lapses in connectivity during insertion of the iSR6250 into all four host paths to storage. No new MPIO software was required to insert the iSR in the Data Path.

ESG Lab configured the iSR6250, identified the volumes to be migrated, completed all SAN zoning, and inserted the router into the host path in less than 25 minutes.

Highly Available Data Migration

The QLogic iSR6200 series is designed to provide online, server-less, minimally disruptive, data migration services. QLogic Intelligent Storage Routers combine dual power supplies and hot swappable dual blade configurations with redundant backed up configuration files for no single point of failure. Blade configurations and migration states are backed up by each blade in a configuration for the other so that a dual blade configuration can survive any issue up to and including a router blade outage. Figure 8 shows the test bed used to move volumes between heterogeneous storage arrays while a workload was running against them. As shown with the green arrows in Figure 8, the iSR6250 router will handle read/write requests during an online data migration.

ESG Lab Testing

With the simulated Exchange IO stream still running against the volumes in the EMC CX3 array (as seen in Figure 8), ESG Lab launched the Data Migration Wizard in QLogic SANsurfer and began configuring an online data migration of four fully provisioned 50 GB LUNs mounted from the EMC CX3-20 to four thin provisioned LUNs mounted from the HP 3PAR F200. The source and destination LUNs as seen by QLogic SANsurfer are shown in Figure 9. Selecting data migration source and destination LUN is a simple drag and drop procedure. QLogic data migration services required classification of arrays explicitly as source and destination which prevents source volumes from being accidentally overwritten.

Next, the data migration wizard prompted the user to select the data migration job options seen in Figure 10.

ESG Lab selected “Online” (Local) which enables data migration while applications remain online and continue to access data. “Schedule for later” provides the ability to schedule jobs at a specific time to optimize bandwidth and Thin Provisioning Validation (“TP Validation”) allows a traditionally provisioned source LUN  to convert to a thin provisioned destination LUN during migration and ensures that old data on the destination is cleared. After completing the data migration wizard, all the data migration jobs automatically started at the scheduled time, as shown in Figure 11.

Then, ESG Lab simulated a single iSR6250 blade failure while the four data migration jobs and simulated Exchange IO were running. The blade failure was simulated by rebooting the blade, which was executing all four data migrations.  While the blade was rebooting, the data migration jobs were suspended. Once the iSR6250 blade came back online, the data migration jobs resumed, as shown in Figure 12 by the QLogic SANsurfer Router Manager Log.

Although the data migration jobs paused while the blade was down, host IO stream continued to flow through the two active paths on the online blade. Because the configuration of each blade is stored on both blades, replacing a router requires no reconfiguration or restarting of migration jobs.

Performance

Organizations with increasingly dynamic, virtualized data centers measure performance of a data migration solution not only by the speed at which data can be moved between the source and target systems, but also by the impact of a migration when executed against systems with online applications.

ESG Lab Testing

ESG Lab tested the iSR 6200 series’ performance by simulating an online application data stream using an attached server, and running multiple data migrations simultaneously. IO was generated using the Medusa Labs Test Suite, shown in Figure 13, and designed to emulate a Microsoft Exchange 2010 database workload running against a 50 GB volume hosted on the EMC CX3-20 array.[5] ESG Lab set the workload to run continuously throughout the test.

Once the workload was running, ESG Lab kicked off a migration of four 50 GB volumes from the EMC CX3-20 array to the 3PAR F200 array. Figure 14 shows the performance of the host, disk, and SSD arrays throughout testing. It’s important to note that the SSD array’s throughput was nearly 40 times that of the host or disk-based arrays. The data is represented in a logarithmic scale to allow all four streams to be represented on the same chart. In Figure 14, the server’s IO is represented by the blue line, while the source array’s IO is represented by the red line. When the data migration to the target array is kicked off, the source array’s IO increases while the host’s IO dips. This is to be expected as the disks on the host array are at this point serving host IO and transferring data to the target array. Next, a data migration was executed between SSD volumes. The SSD arrays were able to drive migration IO of more than 1,800 MB/sec. The iSR6250 was able to process this data stream with zero impact to the server’s IO or the disk to disk migration IO, which continued to run with zero degradation.

Table 1 presents the average IO rates throughout the duration of the test. Prior to the migration, the source disk array was servicing approximately 8 MB/sec of IO, which increased to 87 MB/sec when the disk to disk migration was kicked off. Note that there is no meaningful change of IO from the host, source, or destination array while the SSD migration is running.

What the Numbers Mean

• The iSR 6250 was able to sustain more than 2,000 MB/sec of mixed host and migration IO.
• Host application and disk-to-disk migration performance were stable and consistent, even when the router was under heavy load.

When running online migrations, maintaining application IO can be critical. When migrating from a legacy array with limited performance capabilities or when migrating in a very busy environment, providing effective quality of service control is essential to maintaining service levels while performing migrations.

Finally, ESG Lab tested quality of service control in the iSR6250 by executing a migration and applying an array bandwidth throttle while the migration was executing. The iSR series allows a bandwidth throttle to be applied to any attached storage array. Throttling ensures host IO performance is not negatively impacted by the addition of migration IO traffic on the source array.  Users can choose from a set of predefined bandwidth throttles or set their own custom throttle. As seen in Figure 15, ESG Lab selected the pre-defined “Slow” throttle, which limited migration traffic to 50 MB/sec for all migration jobs in the CX3-20 array.

Figure 16 displays output from the iSR CLI, showing the bandwidth in use for multiple migration jobs running across two 8Gbps Fibre Channel ports between two disk arrays consuming between 120 and 180 MB/sec of bandwidth. Within seconds of setting the throttle, migration IO dropped to 50 MB/sec and remained throttled for the remainder of the migration. The log in Figure 16 is reporting greater than 50 MB/sec of IO on the FC ports because host IO is included as well as migration IO.

Flexibility and Interoperability

ESG Lab validated the flexibility and interoperability of the QLogic iSR6200 series routers by configuring and executing a cross-protocol, online data migration in a heterogeneous SAN environment. As shown in Figure 17, the QLogic iSR 6250 router provides flexibility through unified storage connectivity with support for up to 8Gbps Fibre Channel and one or 10GbE iSCSI in a heterogeneous storage environment. ESG Lab used the same host configuration, running the same simulated Microsoft Exchange 2010 database workload.

The QLogic iSR6200 series routers support integration of enterprise-class features like thin provisioning into a complex, legacy storage environment to help reduce the cost and complexity of storage provisioning while providing significant capacity savings. QLogic SANsurfer’s wizard-driven configuration process and advanced management capabilities provide any to any protocol routing for all classes of SAN-attached storage arrays.

ESG Lab Testing

ESG Lab performed a data migration from a traditionally provisioned Fibre Channel storage array to a thin provisioned iSCSI storage array. ESG Lab started by adding a Dell EqualLogic PS6010 Storage Array as an iSCSI destination shown in Figure 18 below.

The Data Migration wizard (shown in Figure 19) was used to configure two 100 GB fully provisioned LUNs from the FC HP 3PAR F200 as the source LUNs and two thin provisioned 100 GB LUNs from the iSCSI Dell EqualLogic array as the destination LUNs.

Finally, the FC to iSCSI data migration job options were configured. When ESG Lab clicked “Finish,” the Data Migration wizard created the new target volume, started the migration job, and moved the volume transparently to the new storage array with no impact to the reads and writes occurring on the host.

Data migration progress was monitored using the QLogic SANsurfer console, as seen in Figure 20.

When the migration completed, ESG Lab verified that the host was accessing its data exclusively from the target iSCSI LUNs by running the data scrubbing wizard on the source LUNS. Data scrubbing provides a highly secure method of wiping data from a LUN for security, compliance, or repurposing of the drive. As seen in Figure 21, the most stringent DoD algorithm supported by the iSR was selected.

When data scrubbing of the source volumes on the EMC CX3-20 array was complete, ESG Lab mounted the volumes from another virtual machine and confirmed that they now looked like new, empty LUNs.

ESG Lab Validation Highlights

• The iSR6200 series was very easy to implement and manage. In less than 25 minutes, ESG Lab was able to configure an iSR6250 router and insert it into a running host’s storage path without any disruption to IO or running applications.
• Performance was impressive. In ESG Lab’s testing, the iSR 6250 was able to sustain more than 2,000 MB/sec of mixed host and migration IO.
• Host application and disk-to-disk migration performance were stable and consistent, even when the router was under heavy load.
• ESG Lab used the QLogic iSR6200 series router to migrate from fully provisioned volumes to thin provisioned volumes, across heterogeneous Fibre Channel and iSCSI networks, between heterogeneous disk systems. Hosts and applications experienced minimal impact, remaining online and processing IO throughout the data migrations.
• ESG Lab found Data Scrubbing and detailed job reporting easy to use while enabling complete Data Migration services.

Issues to Consider

• While the QLogic iSR6200 series can be inserted into the data path of hosts online and non-disruptively, storage paths must be re-mapped after the migration is complete to remove the iSR from the storage path. It is important to note that this is a host OS requirement rather than a limitation of the iSR. Some users choose to leave the iSR permanently in place to facilitate server mobility across dissimilar platforms (e.g., moving an application from an FC-attached server to an iSCSI-attached server, maintaining access to the FC storage). A single dual-blade iSR 6250 can support a massively consolidated server and storage environment: 1,024 virtual machines, 2,048 iSCSI initiators, and 8,192 LUNs.

The Bigger Truth

Businesses face a number of serious challenges in the data center. The increasing use of server virtualization and accelerating data growth are driving new storage purchases. IT is being challenged to find ways to address data growth and server mobility without interrupting business operations. The deployment of new storage resources to meet increasing demand is a necessity and data migrations are playing a more prominent role.

The rapid rate of change taking place in the data center, combined with increased adoption of iSCSI and FCoE alongside Fibre Channel-based SANs, is driving organizations to look beyond traditional host- or array-based migration methods and tools. The QLogic Intelligent Storage Router is designed to provide advanced technology to enable IT to deliver flexible and dynamic data migration services to the enterprise.

The QLogic iSR6200 Series of Intelligent Storage Routers offer a powerful combination of cost-effective, high performance, multi-protocol storage connectivity with powerful online and offline data migration capabilities. Initiator virtualization technology enables IT to deliver these services with minimal impact to users’ networks.

ESG Lab found the QLogic iSR 6250 series very easy to install and manage. In less than 25 minutes, ESG Lab was able to configure a new iSR6250 router and insert it into a running host’s storage paths without any disruption to IO. The 200 GB data migration from traditional to thin provisioned storage completed without disrupting applications running on the test server and data migrations survived a simulated failure, automatically restarting when the node came back online.

Performance was excellent for both the data migration and the applications running on the server. ESG Lab observed minimal impact to server performance while the router was handling nearly 2 GB/sec of migration traffic. The QLogic iSR’s multi-protocol support was equally impressive, migrating from legacy Fibre Channel storage to a thin provisioned iSCSI array with the same minimally disruptive process as was used for migrating from FC to FC systems.

As organizations continue to expand their use of virtualization technologies and optimize their infrastructure based on criticality of applications and types of workloads, deploying a universal protocol connectivity tool makes sense.  The QLogic iSR6200 series leverages QLogic’s deep experience designing and bringing to market storage networking solutions for business-critical, open system environments that require ultra-high density, high performance, and enterprise class availability. Organizations in need of data migration and storage connectivity solutions for their data centers would be well advised to examine QLogic’s Intelligent Storage Router solutions for a flexible and easy to manage connectivity platform.

Appendix

[2] Configuration details can be found in the Appendix.

[3] Source: ESG Research Report, 2011 IT Spending Intentions Survey, January 2011.

[4] Source: ESG Research Report, 2011 IT Spending Intentions Survey, January 2011.

[5] Configuration details can be found in the Appendix.

[6] Source: ESG Research Report, The Evolution of Server Virtualization, January 2011.