Click here for the PowerPoint slide.

Click here for the full ESG Research Report: 2010 IT Spending Intentions Survey. ]]> http://www.enterprisestrategygroup.com/2010/01/key-areas-of-investment-for-data-storage-over-the-next-12-18-months/feed/ 0 http://www.enterprisestrategygroup.com/2009/11/san-blueprints-by-sanpulse-automated-intelligent-san-insight-and-optimization/ http://www.enterprisestrategygroup.com/2009/11/san-blueprints-by-sanpulse-automated-intelligent-san-insight-and-optimization/#comments Mon, 23 Nov 2009 15:41:48 +0000 Garrett Doherty http://www.esgmedia.net/?p=7286 A SAN Blueprint from SANpulse Technologies reduces the cost and complexity of SAN discovery and analysis as it provides the insight needed to understand, optimize, and update SAN infrastructure. This ESG Lab Validation report presents highlights of recently completed hands-on testing of a SAN Blueprint, the technology used to create that SAN Blueprint, and feedback from production users.

Introduction

A SAN Blueprint from SANpulse Technologies reduces the cost and complexity of SAN discovery and analysis as it provides the insight needed to understand, optimize, and update SAN infrastructure. This ESG Lab Validation report presents highlights of recently completed hands-on testing of a SAN Blueprint, the technology used to create that SAN Blueprint, and feedback from production users.

Challenges

In a recent ESG survey, more than five hundred IT managers within enterprise-class organizations outlined their greatest challenges with respect to their storage environments.[1] At the top of the list are issues that should sound familiar to any IT manager: keeping pace with data growth and rising storage system costs.

While data growth is an obvious cause of rising storage costs, it’s not the only culprit. In many cases, more than half of the cost of the storage infrastructure can be attributed to inefficient use of assets that have already been deployed. Often, there is a lack of visibility into the end-to-end relationship between applications and the storage assets they are using.  This leads to high perceived utilization rates; the actual utilization rates are often quite low.  Poor storage utilization not only increases capital costs, it also increases operational costs (e.g., space, power, cooling, and management).

It is unlikely that the growth of data will abate any time soon, so organizations looking to reduce costs need to focus on managing infrastructure more efficiently. The first step is gaining visibility into a complicated and rapidly growing storage environment. This is not an easy task; manual methods leveraging armies of people and Excel spreadsheets are time consuming and error prone. Storage resource management tools can be effective, but some require agents and others are limited to a single vendor’s products.  Ideally, insight into a large heterogeneous enterprise-class SAN infrastructure would be provided with a single tool that supports agent-less collection methods.

SAN Blueprint from SANpulse Technologies

As shown in Figure 2, SANpulse Technologies uses a combination of people, processes, and technology to deliver a SAN Blueprint that provides visibility into the inner workings of a complex heterogeneous storage infrastructure. These SAN Blueprints are used to plan, optimize, consolidate, and upgrade complex enterprise-class storage infrastructures.

A SAN Blueprint is a vital first step in any SAN migration and optimization project. It delivers significant savings as it reduces capital costs (buy less or make better use of what you already own) and operational costs (use less manpower, space, power, and cooling). Ongoing and consistent use of a SAN Blueprint ensures that every project—whether it’s storage consolidation, re-tiering, or migration—will be properly assessed, planned, and executed.

The balance of this report explores how SANpulse has combined decades of hands-on expertise with powerful software and automation to create an agentless, modular, role-based software platform which provides an intuitive, end-to-end view of a heterogeneous SAN infrastructure.  A SAN Blueprint not only provides a real-time view of the SAN, it can also be used to automate best practices as it delivers an actionable plan for optimization, consolidation, and savings.

ESG Lab Validation

ESG Lab first examined SANpulse’s technology and services in 2008 with a focus on its ability to automate and streamline data migration projects. A hands-on analysis of SANpulse software (SANlogics) and a visit with a SANpulse customer in the middle of a multi-month data migration project proved that the automation and intelligence built into SANlogics can be used to significantly reduce the time and costs associated with a data migration project.  Compared to traditional data migration methods, SANpulse customers reduced the time required for the planning phase by 65%, the implementation phase by 86%, and the execution phase by 52%.  The average cost savings over the entire duration of a project amounted to 75%.

ESG Lab returned to SANpulse in 2009 and met two more customers with a goal of taking a closer look at the content and value of a SAN Blueprint.  This second ESG Lab report begins with an examination of a SAN Blueprint recently created for a large, Global 100 organization. Next, the technology behind a SAN Blueprint is presented based on the results of hands-on testing of the latest version of SANlogics software.   And finally, feedback on the value of a SAN Blueprint is presented based on the results of interviews with two satisfied SANpulse customers: one that had contracted SANpulse to do a SAN Blueprint of their massive SAN infrastructure for strategic planning purposes and a second that had witnessed the power and flexibility of a SAN Blueprint during a data migration project.

A SAN Blueprint

SANpulse creates SAN Blueprints for three types of projects:

1. As part of services engagement to help understand and optimize current SAN infrastructure
2. As a deliverable during the planning phase of a SANpulse managed data migration project
3. On an ongoing basis to optimize return on investment (ROI) during routine storage operations

A bound copy of the SAN Blueprint shown in Figure 3 was created as part of a data migration project with a large organization in the banking industry.[2]

The SAN Blueprint was used to investigate the connectivity, utilization, and efficiency of five EMC Symmetrix and five EMC CLARiiON disk arrays connected to servers running mission-critical banking and business productivity applications. The project began with a data collection effort aimed at creating a comprehensive inventory of the SAN infrastructure. Data collection and automated expert correlation were used to create the SAN Blueprint as shown in Figure 4.

When collecting the information needed to create a SAN Blueprint, SANpulse engineers work hand-in-hand with an internal IT organization.  A variety of collection models are supported, depending on an organization’s security policies and the tools and processes already in place. Sometimes, gathered information is sent via FTP, but in most cases, a SANpulse engineer visits the data center and collects data under the direction of an authorized storage administrator.  The administrator provides the security credentials needed to access the management interfaces of storage-related IT infrastructure including disk arrays, switches, and servers. Often, the administrator provides the SANpulse engineer with copies of offline reports and spreadsheets as well.  In some cases, an organization’s management security policies may not allow administrator-level access for third party service providers. In these instances, SANpulse prepares scripts which are then run by an authorized administrator.  In this customer’s case, a SANpulse engineer visited the bank’s primary data center and spent approximately four hours collecting host, switch, and array configuration data.

To ensure that data collection is non-disruptive, SANpulse uses agentless data collection methods supporting a number of industry standards including SNMP, telnet, ssh, and Web services. Vendor-specific data collection methods including grabs, command line interfaces, application programming interfaces, and storage resource management reports are supported as well as customer-specific data sources such as Excel spreadsheets and ODBC enabled database exports.  In this case, the bulk of the bank’s SAN infrastructure data came from array-specific command line interfaces, the SNMP interfaces of servers and switches, and storage resource management software.

As configuration data is gathered from a number of sources, confidence in the accuracy of the inventory increases. This is accomplished using an intelligent correlation engine within the SANlogics software platform.  Online data sources are trusted more than offline sources, since they are often out of date.  As multiple data sources converge on the same data points, confidence in the configuration data is increased. The SANlogics software platform and the SANpulse team go to extreme measures to check, validate, and re-check the inventory at every stage. The end result of all of this data collection and expert correlation is a SAN Blueprint loaded with charts and tables giving a summary of the customer’s SAN infrastructure. Consider, for example, the graph shown in Figure 5.  This chart provides a useful high level summary of the total configured capacity for the five Symmetrix and five CLARiiON disk arrays. Note that the chart also shows how much of the storage capacity is actually used.

Excerpts from one of the tables in the SAN Blueprint are listed in Table 1.

The data presented in Table 1 illustrates one of the key advantages of a SAN Blueprint: knowing which servers are connected to each disk array. This seemingly simple information can be maddeningly difficult to obtain and maintain using traditional storage management methods. Disk array management information typically stops at the border between the disk array and the storage area network. Painstaking and error prone correlation is often needed to turn the FC address and zoning data presented by servers, switches, and disk arrays into an end-to-connectivity map of the SAN.

The SAN Blueprint not only provides an up to date and accurate count of SAN attached servers, it also provides a list of those servers along with their storage-specific configuration details including the operating system, HBA firmware, HBA driver, multi-path, and volume manager.

Note that Table 1 also includes a list of SAN configuration problems that often occur within a dynamically changing SAN such as the number of servers with assigned storage that’s never been accessed.  Potential SAN wiring, zoning, and access control issues are reported. Potentially unsupportable software version levels (HBA, multi-path, disk array) are also noted.  What’s more, drill-down reports provide the information needed to find and fix each of these potential issues.

The SAN Blueprint also includes a number of charts which highlight potential resource balancing issues. For example, Figure 6 shows how LUNs are allocated on an uneven basis across a server-facing port within one of the disk arrays.

SANlogics

SANpulse experts use SANlogics software to deliver a SAN Blueprint as either a standalone offering or as part of a SAN optimization or data migration initiative. At the core of the SANlogics software platform is a powerful database and query engine.  The database is used to collect and correlate a wide range of data sources and the codified expertise of expert SAN administrators.  ESG Lab first evaluated SANlogics software in 2008 during a data migration project. During this second ESG Lab Validation, we examined the enhanced SANlogics software platform with a focus on its role in creating a SAN Blueprint.

ESG Lab worked side by side with SANpulse engineers browsing a SANlogics database being used to plan for a large-scale data migration project at a Fortune 100 financial services company.

Working from the top down at a business unit level as shown in Figure 7 or from the bottom up with a detailed view of the configuration details within a single disk array, ESG Lab found that the latest version of SANlogics software provides a rich set of graphs and tables which provide intelligent and actionable insight into an organization’s storage infrastructure.[4]

Moving from a macro-level review of the SAN infrastructure, ESG Lab examined a series of new micro-level reporting capabilities that have been built into the SANlogics software platform.  The SAN Blueprint presented earlier shows how SANlogics helps identify potential FC wiring, zoning, and configuration errors. While the SAN Blueprint presents a high level summary of potential SAN issues, the SANlogics software platform provides detailed, on-demand reporting that can be used to find and fix common issues including:

1. Reclaiming wired, but unused, SAN switch ports
2. Reclaiming allocated, but incorrectly configured, storage capacity
3. Cleaning stale zoning and access control definitions

The report excerpt in Figure 8 shows an end-to-end view of FC connectivity status that ESG Lab believes to be powerful and unique in the industry.  The table shows the current array, switch, and zone login status. This, and other information in the SANlogics database, is correlated with a goal of identifying configured array, switch, and host ports that have capacity allocated, but are not currently logged in and don’t appear to have ever logged in.

The richness of the table data is enhanced by a color coded map that makes it easy to identify potential candidates for reclamation.  A green box indicates a potential resource that can be reclaimed. A row of green boxes indicates the best place to look for reclaimable ports and capacity. Note that the table not only presents the end-to-end details needed to isolate and reclaim potentially wasted SAN assets, it also calculates the capacity savings that can be achieved.

SANlogics incorporates GUI-based, executive dashboard-level views as well as many drilled-down SAN Blueprint graphs and charts.  Those graphs and charts can be either exported to Adobe PDF documents and Microsoft Excel spreadsheets or dynamically updated through the use of a SANlogics Excel Reporting Module.  Microsoft Excel spreadsheets generated by SANlogics include built-in filters—as shown in Figure 9—that can be used to quickly and dynamically browse the correlated contents of the SANlogics database.  While industry experts often malign the widespread use of manually created spreadsheets to keep track of SAN assets, ESG Lab believes that SAN administrators will appreciate the familiar look and feel of automatically created, easily sorted and filtered SANlogics spreadsheets.

SANpulse in Action

ESG Lab met with IT managers from two enterprise-class SANpulse customers that have recently received a SAN Blueprint as part of a professional services engagement.

First Customer Evaluation

The first customer interviewed by ESG Lab is a storage architect for a large enterprise services division of a Fortune 100 organization that delivers technology products and services.  Its extremely large SAN infrastructure has swelled in recent years, with hundreds of enterprise-class disk arrays and tens of thousands of SAN ports deployed in multiple data centers that are running out of space, power, and cooling capabilities. A number of strategic global initiatives are underway with a goal of better utilizing the existing infrastructure (e.g., server virtualization, data archiving, and storage consolidation).

The Challenges

While the size of the SAN infrastructure has grown dramatically in recent years, the headcount within the IT organization has not kept pace.  Existing storage acquisition and deployment processes are unable to keep up with the needs of new and existing applications.

The storage architect knew intuitively that existing SAN assets were not being fully utilized. His new boss wanted a better view of the storage assets he was responsible for, but he didn’t have the tools or manpower for a detailed assessment of his SAN infrastructure. SANpulse was hired to provide a SAN Blueprint.

The Solution

SANpulse presented the SAN Blueprint and all the supported findings during a meeting with the storage architect, his boss, members of the storage team, and representatives from their preferred storage vendor.  As the storage architect expected, the utilization of the existing SAN infrastructure was low (less than 50%).  While his boss had been told that the infrastructure was underutilized and would benefit from tuning and consolidation, he was alarmed by the scale and the cost of the wasted resources that had accumulated before he took charge.

The Benefits

The SAN Blueprint led to a number of significant benefits for this organization:

1. Reducing Costs: Analysis of the SAN Blueprint identified 55 TB of storage that was pre-mapped, but not allocated to any host. Further investigation led to a disaster recovery reconfiguration with storage being released for other applications. Additional storage was reclaimed based on a SAN Blueprint inspired recommendation of using RAID-5 instead of RAID-1 for disaster recovery. Using RAID-5 for disaster recovery can release significant amounts of disk capacity as it uses less disk capacity than RAID-1 (50+%).  The customer was able to delay upcoming storage purchases and repurpose the funds by moving from RAID-1 to RAID-5 replication for nearly 60% of its applications.
2. Mitigating Risk: Load balancing and failover issues were identified and corrected based on the information provided by the SAN Blueprint. In addition, the SAN Blueprint identified approximately 900 devices that were not mapped, zoned, or allocated.  This did not comply with best practice and could have easily led to operational and business application outages in the event of a disaster.  Further investigation corrected the problem and ensured business continuity for all applications.
3. The Bottom Line, as estimated by this happy SANpulse customer, began with a minimum of $250,000 in storage capacity savings.  This is on top of the immeasurable, but sizable, indirect cost savings associated with lessening the risk of data loss after a disaster.  Overall, the storage architect was thrilled with the results. First, it required no effort on his part and required minimal interaction with his production team.  Second, it provided his boss with a realistic view of the challenges he was facing. Third, he had a current view of his SAN infrastructure with more detail than he could have obtained using existing tools and processes. And, last but not least, he and his boss felt that the value greatly exceeded the reasonable price of the SANpulse engagement.

Second Customer Evaluation

This Fortune 100 SANpulse customer received a SAN Blueprint as part of a proof of concept data migration project. For this organization, large-scale data migration projects are required on an ongoing basis to accommodate the replacement of disk arrays that have reached the end of a lease or depreciation schedule. A proof of concept team worked with SANpulse to determine the feasibility and value of a SANpulse professional services engagement.

The Challenges

This organization routinely outsources data migration projects, which take months to complete and cost hundreds of thousands of dollars. The sheer number of ongoing projects was straining the limits of the existing staff and budget.

The Solution

SANpulse captured data from existing EMC DMX 2000 and DMX 3000 storage arrays that were scheduled to migrate to DMX-3 and DMX-4 arrays located at a different data center. A SAN Blueprint was used to determine the readiness and remediation required for the migration.  The automated discovery of array, switch, host bus adapter, driver, operating system, and multi-path driver configuration data was compared to vendor and internal support matrices to create a plan for remediation.

The Benefits

The SAN Blueprint was used to optimize the data migration planning process.

1. Automating Remediation: The customer provided a list of 104 hosts that were targeted for the initial migration.  Only 41 of the 104 hosts were up to date. The rest required a multi-level remediation initiative which included service pack upgrades, HBA firmware upgrades, HBA driver upgrades, multi-path driver upgrades, volume manager upgrades, and cluster server upgrades. In addition to the list of 104 hosts that were scheduled to migrate, the SAN Blueprint identified an additional 127 host connections that appeared to be connected to the disk arrays that were scheduled for migration.  This information was used to clean up the configuration and add missing servers to the remediation and migration plan.
2. Reducing Risk: The SAN Blueprint was used to identify and fix a number of issues that put the schedule for the project—and the likelihood of success—at risk. A number of potential issues were investigated and resolved (e.g., switch zone entries that were configured incorrectly, array masking that was wrong,  fiber connections with low light, and  ports that were disabled). In addition, eleven hosts were identified as being connected to multiple disk arrays. This is a common issue that has an impact on data migrations—especially when a file system or application unintentionally spans multiple arrays.
3. The Bottom Line for this customer was a data migration project that completed flawlessly and on schedule. The SAN Blueprint automated the planning process. SANlogics software generated executable work files which automated the remediation and migration. The customer was very impressed by the speed and ease in which the data was collected and correlated, the depth of the findings, and the minimal impact to their team.  SANpulse has been added to the approved vendor list and has won subsequent data migration projects.

This customer was most impressed by the flexibility of the SANpulse technology and team. They were amazed that last minute changes in the scope of the project did not impact the scheduled migrations.  Just before the planned migration during a weekend maintenance window, the team learned that a number of hosts were not ready to be migrated based on the lack of approval from the application owner. Another potential delay was identified when an automated pre-migration audit revealed that an array which was planned to be migrated was not available. Based on their experience with past data migration projects, the client team assumed that a last minute change in scope would defer the migration to the next available maintenance window.  Instead, SANlogics software was used to instantly re-discover the environment and create a new set of data migration output on the fly. Shortly afterwards the migration was underway as planned.

ESG Lab Validation Highlights

• ESG Lab examined a SAN Blueprint that had been created for a large financial services organization.
• End-to-end details presented in the SAN Blueprint included a number of graphs and tables which not only presented the overall utilization of infrastructure, but also provided guidelines for remediation, consolidation, and savings.
• ESG Lab performed hands-on testing of SANlogics, the software used to create a SAN Blueprint. High level utilization at the business unit level and low level reports which provided a view into SAN fabric utilization and optimization were examined. The ability to export tables in Microsoft Excel format with built-in filtering and sorting was noted.
• ESG Lab met with two customers who have received a SAN Blueprint as part of a professional services engagement with SANpulse Technologies.  Both extolled the value of a SAN Blueprint. Both indicated that they plan on doing business with SANpulse again in the future.

Issues to Consider

• SANpulse currently delivers SAN Blueprints as a standalone offering or as part of a larger SAN migration and optimization initiative. As a result, it represents the state of the SAN at a single point in time.  To keep up with the ever-changing nature of an enterprise-class SAN, ESG Lab recommends that IT organizations consider the purchase of SAN Blueprint on an ongoing, scheduled basis (e.g., quarterly).
• SANlogics software has undergone a recent usability enhancement with a goal of turning it into a product that IT administrators can run on their own. At this time, however, ESG Lab believes that SANlogics should be managed by an experienced SANpulse professional.  The value that SANpulse delivers includes the team and its rich heritage of SAN management expertise.

The Bigger Truth

Modern data centers are incredibly complex. In order to meet the needs of the business in a dynamic global market, IT is always changing and adapting. New applications are added, IT infrastructure is continually growing, and new technology is constantly being deployed in order to keep pace.  IT organizations struggle to keep up and bad habits tend to develop due to a lack of time and resources.  Consider, for example, how fully documented change processes are circumvented with hand-written notes, spreadsheets, and Visio diagrams. IT administrators with the best of intentions plan on updating documentation to reflect changes that were made during the heat of a moment. Yet, more often than not, the next emergency comes before the last round of changes can be documented.

Clearly, these ad hoc methods are not scalable. They often result in underutilized infrastructure and changes can have unintended negative consequences. In many cases, as new technology is brought in, those bad habits are migrated to the new infrastructure, reducing the potential return on investment. IT needs to be able to get an accurate view of the infrastructure in order to make intelligent decisions.  Getting more out of your existing investment is clearly more cost effective than buying new capital equipment. If and when it makes sense to invest in new capital equipment, a good understanding of your current infrastructure is needed to maximize the value of your investment. Put simply, it’s hard to improve what you can’t measure.

Measuring an enterprise-class SAN storage infrastructure is particularly challenging. Even in the current economic climate and depressed global economy, the growth of data has not slowed. Unless the organization is prepared to dramatically reduce other areas of the budget, IT needs to find ways to reduce costs and become more efficient. Given that storage consumes such a large portion of the IT budget, it makes sense to start there. Since most storage utilization rates are typically less than 50%, this would seem like an easy task. Unfortunately it isn’t. Trying to get a holistic view of an enterprise-class SAN infrastructure can be very difficult.

The bigger problem is that storage administrators—and their managers—often aren’t aware of the extent of the problem. To a storage administrator, storage that has been allocated is utilized. It is only when we take a top down view from the server’s perspective that many of the real inefficiencies can be spotted. Orphaned and underutilized storage need to be identified and reclaimed, if possible. Only by getting complete visibility and a deep understanding of the current environment can IT hope to reduce capital and operational expenses.

ESG Lab testing and conversations with customers have confirmed that the actionable output of a SAN Blueprint delivers measurable results. Customers that we talked to realized considerable savings from the reclamation of unutilized storage resources. As a matter of fact, the savings were far more than the cost of the original investment in a SAN Blueprint.

ESG Lab has confirmed that SANpulse offers solutions which help organizations measure—and improve—their current SAN infrastructure. Leveraging skilled people, proven processes, and a flexible, automated SANlogics technology platform, a SAN Blueprint provides the visibility that is needed to make strategic decisions about the storage environment, defer capital purchases, optimize existing assets, and, last but not least, ensure that bad habits are not migrated to a new infrastructure.

If your organization would like to know more about its SAN infrastructure as part of a server virtualization, data center consolidation, disk array upgrade, or long-term data archiving project, ESG Lab recommends that you consider a SAN Blueprint from SANpulse Technologies.

[2] To protect the confidentiality of the SANpulse customer, all company-specific details including data center names, server names, zone names, and storage system serial numbers were redacted from the blueprint given to ESG Lab.

[3] Source: ESG Research Report, 2008 Enterprise Storage Systems Survey, November 2008.

[4] For confidentiality, the actual business unit names were redacted from the graph shown in this report.

[5] Source: ESG Research Report, 2009 Data Center Spending Intentions Survey, March 2009.

Overview

ESG polled end-users about their backup processes and found that the majority (72%) back up to on-premises storage and transfer copies to a secondary site via removable tape media or replication.  Twenty percent of organizations rely exclusively on onsite storage and do not move data off site for long-term retention. This riskier approach is more common among small and medium-sized businesses (SMBs) (24%) than enterprise-class organizations (15%). The remainder of respondents (8%) had an all off-premises strategy.  ESG also found that 86% of respondents rely on physical tape media for backup processes, although respondents indicated a desire to shift to disk-based solutions.[1]

Data growth is highlighting the problems of tape-centric data protection strategies.  Tape’s performance and reliability issues are often the culprits in not meeting backup and recovery service level agreements (SLAs), leaving gaps in data protection.  Tape backup operations can also reign over IT staff, requiring operator intervention for tape handling and troubleshooting.  These issues are increasingly driving organizations to adopt disk in the backup process, but disk only addresses onsite operational recovery.  Transporting backup copies off site to enable data recovery in the event of a site outage or regional disaster minimizes risk.

That’s where the “cloud” comes in. The cloud is on-demand network access to a shared pool of “elastic” storage resources.  Used as a backup target-as an alternative to portable media, such as tape-cloud services are typically charged back to the subscriber on a consumption basis.  The lower cost of disk has changed the economics of applying it in the backup process and, similarly, more affordable bandwidth and capacity optimization technologies have changed the economics of leveraging cloud-based storage for disaster recovery purposes.

The cost efficiency achieved through automation and reduction of manual labor by applying both local disk-based backup and remote cloud-based storage for disaster recovery (DR) is a compelling benefit.  However, the indisputable improvements in reliability, scalability, security, and SLAs may be more important considerations for deploying a hybrid on- and off-premises solution that leverages the cloud.

SunGard, a leader in availability services, has a long history of ensuring uninterrupted access to applications and data.  The introduction of the company’s Secure2Disk hybrid backup service is a natural extension to its business continuity portfolio.

Analysis

SunGard Secure2Disk consists of an onsite appliance that facilitates backup of production systems’ data with an agentless approach.  Backup data is stored locally for rapid recovery.  A copy of backup data on the local appliance is deduplicated, compressed, encrypted, and transferred off site to the cloud-in this case, to a backup vault in one of SunGard’s six SAS-70 Type II-certified data centers-for DR purposes.  On-premises monitoring and management of backup and recovery operations, user access, and policies occur via Secure2Disk’s Web-based interface.  SunGard takes care of all of the behind-the-scenes management of the offsite backup vault.

Secure2Disk has several advantages over on-premises, licensed backup software protecting data on tape storage, including:

• Simplified installation – “Agentless” approaches minimize the overhead and invasiveness of a solution at the primary site. Instead of necessitating the installation of a client or application backup agent on every system to protect, a Secure2Disk appliance is installed on the LAN to facilitate protection of systems and data.
• Performance – Secure2Disk features backup to local disk-based storage with capacity-optimized copies of data transferred to a remote backup vault outside the backup window. Writing data to a sequential access medium, such as tape, involves more overhead and is slower. Eliminating tape media creation accelerates time to protection (locally) and time to DR. Recovering from Secure2Disk’s local vault speeds recovery versus a tape-based strategy. SunGard offers services to package and transport the initial backup data set to the central backup vault, as well as recover a full system or site.
• Reliability – The capacity of backups for some organizations can exceed the capacity of the tape media, requiring more than one piece of media in the process. Unless some type of tape automation is in place, an operator has to get involved with tape handling (tape handling will be required regardless as tape media has to be loaded, removed, and transported off site for DR). Tape backup can be unreliable, as operator intervention can introduce errors, and tape media or tape device faults can cause backup/recovery failures. By removing tape from the backup equation, reliability is improved for both backup and recovery.
• Scalability – On-premises infrastructure will have some bounds, whether restricted by physical space or budgets. Secure2Disk’s cloud-based storage delivers savings in acquisition, installation, provisioning, and management of onsite storage capacity and provides on-demand scale with a consumption cost model.
• Operations/management - Eliminating tape significantly reduces the need for operator intervention. Furthermore, on-premises backup requires ongoing monitoring and management to maintain optimal service levels and efficiency. Secure2Disk service personnel take on the responsibility of maintaining backup infrastructure and ensuring that backup/recovery service levels are met.
• Security – One of the biggest risks with an on-premises tape-based backup strategy is the transport of media offsite for DR. There have been countless publicized data breaches resulting from unencrypted data being compromised outside of an organization’s custody. Secure2Disk leverages encryption to secure data in flight and at rest, with the responsibility for key management resting with the Secure2Disk subscriber. Decryption of data requires the encryption key, restricting access to data to only designated personnel with security credentials.
• Cost – The capital and operational expenses associated with tape-based backup may strain IT budgets. Secure2Disk pricing is more inclusive. Its per-gigabyte pricing is based on the volume of protected data (deduplicated and compressed data) at the offsite backup vault. Secure2Disk’s pricing includes the on-premises appliance, storage and software; Metro bandwidth; cloud-based vault storage; and 24/7 monitoring, management, and support.

It’s this last point that SunGard is emphasizing with the release of Secure2Disk.  The company believes its solution presents a total cost of ownership (TCO) approaching that of a tape-based equivalent, but with superiority in security, scalability, reliability, and service level delivery.

The Bottom Line

The allure of cloud computing is attracting vendors and end-users alike.  ESG has seen an influx of top-tier vendors and new entrants introduce full or partial cloud-enabled data protection solutions.  End-users-especially those small to mid-size organizations with a single data center, a high reliance on tape infrastructure, and regulatory or corporate mandates for data protection or retention-like the low-cost and low-risk entry to the model, as well as the high rewards.

SunGard has the brand, infrastructure, and expertise to win customers.  However, the company is not just resting on its reputation.  SunGard has based its solution on solid data protection technology that supports major physical and virtual server platforms such as Windows, HP-UX, AIX, Solaris, Linux (Red Hat and SUSE), iSeries, VMware and XenSource, and applications, including Oracle, Microsoft SQL, Exchange/Outlook, SharePoint, DB2, MySQL, Lotus Notes/Domino, and SAP.

The company is also doing some interesting things to optimize storage and pass on the savings to its subscribers.  Secure2Disk leverages deduplication and compression to maximize the use of bandwidth in the transfer of data.  These techniques also promote storage efficiency.  Rather than base its consumption model on the volume of data to back up (at the primary site), SunGard’s pricing is based on the volume of data protected (at the SunGard backup vault).  This, combined with SunGard’s 20% discount in its introductory period (calendar 2009), makes Secure2Disk an attractive alternative to on-premises, tape-centric strategies-especially in tough economic times.

Nexenta

Nexenta has been around since 2005 and has been shipping its ZFS-based NexentaStor software since late 2007.  NexentaStor is based upon an open source core platform (ZFS), but is itself closed source code that Nexenta integrates, develops, licenses, and supports.  It provides NAS and iSCSI capabilities (Fibre Channel support forthcoming) and offers unique capabilities for the cost-effective management of storage for virtualized environments.  With file-based data growing at a 55% CAGR and expected to make up the vast majority of the data stored in the data center,[1] finding a cost-effective way of storing and managing that data has become of increasing importance across enterprises large and small.

NexentaStor is layered onto the open-source ZFS file system.  Leveraging ZFS provides Nexenta a number of advantages:

1. ZFS is open source. Leveraging the power of community to develop ZFS provides development resources while keeping development costs in check.
2. ZFS, and therefore NexentaStor, is hardware independent. Leveraging ZFS puts choice of hardware platform back into the hands of the buyer.  NexentaStor software provides an impressive range of qualified hardware platforms, connectivity, and drive type options.
3. ZFS was designed from the ground up as a software-based storage management platform. It was not built within the confines of hardware platform limitations and developers could take a functionality-based approach to development rather than an array management-based approach.
4. ZFS lets customers avoid vendor lock-in, even for their software purchases. This may very well be the most important point: these buyers could decide to use another ZFS supporting solution, such as OpenSolaris, to access their data should NexentaStor no longer meet their needs.  Perhaps ironically, this lack of vendor lock-in for both hardware and software reduces risk when purchasing solutions from Nexenta.

It is primarily the robust ZFS functionality that has drawn users in-NexentaStor takes that functionality and builds on it with advanced features.  ZFS covers all the storage management functions typically handled in hardware on competitive RAID arrays: RAID protection, volume management, error checking, etc.  ZFS manages storage using a pooled storage concept and has inherent tiering and load balancing support, so there is no need to move data on or off of solid state drives.  Multiple drive types (e.x., Fibre Channel, solid state (SSD), SATA) can coexist in what is termed as ‘Hybrid Storage Pools.’  These hybrid pools are designed to deliver the optimum blend of maximized performance at minimized cost by moving data to an appropriate pool based on its usage or access patterns.  Leveraging a combination of SSD and dense SATA drives, users can significantly reduce the overall number of drives deployed while maintaining required performance levels and saving on floor space, power, and cooling.  Additional features like thin provisioning, standards-based compression, and unlimited space-saving snapshots and file sizes make ZFS a well-rounded storage management platform upon which to build.

One of the challenges with ZFS has been manageability-ZFS is feature rich, but that richness can lead to complexity and most users are not willing or able to attempt to script and support their own storage solution on OpenSolaris.  NexentaStor is software layered onto the ZFS file system that provides a simple-to-use management interface and some advanced functionality that ZFS lacks, like synchronous remote mirroring, integrated search to actually find the right snapshot amongst the hundreds or thousands that have been created, and a module that aides in the management of storage for virtualized environments.  But there have been a couple of barriers for enterprise adoption of NexentaStor, both of which have been addressed with NexentaStor 2.0: lack of 24 x 7 phone support and fully automated high availability.

With NexentaStor 2.0, Nexenta introduces fully automated high availability, in which failure of a node in a two node cluster automatically routes all traffic to the surviving node.  According to Nexenta, users may see a hiccup, but not much more.  With 2.0, there is no user intervention required for the surviving node to take over operations.  This functionality is more-or-less a jacks-or-better entrée into the enterprise, as is 24 x 7 support.  Nexenta is contracting with an enterprise-class storage services company to assist in providing this support.

Over 300 customers purchased and deployed NexentaStor in its first twelve months of availability-a pretty impressive number for a startup-and it has been downloaded and deployed by several thousand trial or free users.  NexentaStor is also available from a growing number of systems integrators around the world; offering NexentaStor software plus hardware plus services for a total solution.

Why Open Storage is Important

Open storage changes the vendor/consumer relationship and puts much more control into the hands of the buyer. It’s out there, warts and all, for the world to see.  The willingness of Nexenta and other open-source based vendors to solicit community user feedback in the open, not behind closed doors and in double super secret NDA customer councils like so many storage vendors have today, creates realistic customer expectations.  In a world where many tech companies are just moving into social media and largely leveraging it as a marketing tool, open source companies like NexentaStor have been using it for a number of years as a development and user communication methodology.  Don’t storage buyers ultimately want a partnership with their vendors in which they are listened to, paid attention to, and have their requirements are acted upon?  That is exactly what Open storage promises-but, in this case, the process is explicit, open, and collaborative.

Nexenta is able to harness the power of the hundreds of thousands of users that have downloaded the open source foundation of Nexenta-providing a cost-effective development and marketing effort that allows it to pass savings on to the buyer.  Nexenta claims it can provide equivalent or better functionality to proprietary systems at less than 50% of the upfront price.  Of course, savings depend on the hardware choice, but the key here is choice.  Over time, the savings grow because users are free to source disk and server platforms from anyone on the fairly extensive support matrix.

Challenges

The open storage movement is not without challenges.  Sun has been the driver behind ZFS and is one of the most vocal proponents of the movement and with Oracle buying Sun, there is some question about Oracle’s commitment to ZFS. Oracle has the BTRFS file system in development and has restated its commitment to BTRFS since the announced Sun acquisition.  ZFS has a strong support base and momentum.  It is a file system invented for scale, speed, and efficiency.  Oracle may shift ZFS to a GPL-based license and keep the momentum rolling-or it might not.  Users do need to be aware and keep an eye on developments, but because NexentaStor is layered onto the ZFS fie system, it is portable-Nexenta can change the underlying file system if need be, to BTRFS for example, to significantly mitigate risk.  Also remember that you cannot really ‘un-open source’ a technology. Nexenta is in a strong position with a rapidly maturing ZFS file system, regardless of whether Oracle decides to continue to invest at the rate Sun has.

Nexenta’s To-Do List

While NexentaStor 2.0 makes significant strides, there are a couple of items enterprises will want to see addressed.  Using CIFS in a complex Active Directory environment can be a little difficult and is a known ZFS issue-it needs to be simplified.  And NexentaStor does not have integrated deduplication, though it does work with third party deduplication appliances.  Integrated deduplication is becoming a file server checkbox item.  Also, given NexentaStor’s capabilities in the management of storage for virtualized environments, extending this support to Microsoft’s Hyper-V virtualization could be important as ESG believes that Hyper-V is likely to become a powerful force in the market.  ESG would not be surprised to see these three issues addressed by Nexenta in the not-too-distant future.

The Bottom Line

ESG recently performed an unscientific Twitter survey looking for feedback on NexentaStor.  We were surprised on two fronts: the first being the number of people that replied. The solicitation went as a “Tweet” to under 400 users and, after all, who’s ever heard of Nexenta?  It turns out a lot of people have.  The second surprising point was that all the respondents were happy users.  In a world where the average person passes on a positive experience to only three friends but shouts about negative experiences from the rooftops, we couldn’t dig up any dirt.  That’s pretty impressive for a company that openly admits it did not have a complete enterprise-class solution until NexentaStor 2.0.

Open storage is here. It is important, it is viable, and now, with automated high availability functionality and 24 x 7 support, NexentaStor is enterprise ready.  The proof point is in the 11,100 (and growing) registered users-a remarkable number for a product that has been available for less than two years.  Perhaps the most important point in the open storage effort is that is gives users the ability to choose the hardware platform they are most comfortable with.  The storage industry is a proprietary holdout-there are no widely adopted and deployed interoperability standards, so when a user picks a storage hardware platform, they have no choice about upgrades.  The advent of open storage means choice and interoperability.  Efforts like Nexenta’s are propelling storage into the twenty first century-much like RedHat did for the server industry with Linux.

Introduction

Deduplication dramatically improves the value proposition of disk-based data protection as it eliminates the redundancy typically seen in secondary storage processes.  The use of deduplication will drive further backup-to-disk adoption and deliver associated performance and reliability benefits.

Selecting a strategy for data deduplication requires consideration of several factors in order to avoid surprises later.  Clearly understanding how deduplication works—especially in conjunction with other requirements such as performance, ease of use, and offsite copy creation—should go a long way toward selecting and designing a solution that delivers maximum business, operational, and financial benefits.

Deduplication has been more popularly deployed in target storage hardware, including virtual tape libraries (VTLs) and storage appliances; however, recent introductions by backup vendors have shifted the spotlight.  How do hardware and software approaches differ? What should be considered when evaluating solutions?

External Forces Contribute to IT Challenges

There are a number of external forces working against IT organizations today, including data growth, compliance, and economic difficulties. These realities are impacting not only data protection processes and secondary storage environments, but data center environmentals such as power, cooling, and floor space, as well as bandwidth between primary and secondary recovery sites.

Relentless information growth is necessitating greater investments in IT infrastructure.  ESG estimates that database data is growing at 25% per annum, with unstructured data increasing at two to three times that rate.[1] Data protection processes, such as backup and replication, compound capacity growth since multiple copies of primary data are made for operational and disaster recovery. ESG research respondents cited data protection as the application that will be most responsible for storage growth over the next 24 months (see Figure 1).[2]

The need to retain information for longer periods of time on accessible media, such as disk, for compliance, eDiscovery, and business intelligence purposes also contributes to capacity overabundance and stress on the data protection infrastructure.  Moreover, in an effort to improve the performance and reliability of backup and recovery operations, organizations have been increasingly using disk as both the initial and final resting place of backup copies.  While capital investments in disk can increase costs, disk-based data protection can contribute to lower operational expenses and improvements in backup and recovery service level agreements (SLAs).

In the midst of global financial turmoil, IT organizations are highly motivated to reduce costs and optimize efficiency—but not at the expense of introducing risk or impacting value.  Technologies that create efficiency and deliver rapid ROI without sacrificing organizations’ other goals—meeting backup and recovery SLAs, for example—are those being considered and deployed.  One such technology is data deduplication.

Data Deduplication

Data deduplication identifies and eliminates redundancy, minimizing bandwidth and storage capacity requirements.  Deduplication, while not new, has gained even greater popularity today:  The increased use of disk in backup and recovery and data protection’s aforementioned contribution to storage capacity growth make deduplication attractive.  In fact, ESG research found that data reduction tops the list of respondents’ top five storage priorities over the next 24 months (see Figure 2).[3]

Deduplication in backup processes ensures that only unique data is stored and replicate data is not.  Initially, data is backed up to the storage device and all subsequently written data is examined for redundancy, with only unique data being written to storage.  When duplicate data is found, only a pointer linked to the original unique piece of data is stored.  This pointer consumes significantly less space than storing the whole item multiple times.

The effectiveness of deduplication is often expressed as a reduction ratio denoting the ratio of protected capacity to the actual physical capacity stored.  A 10:1 ratio means that 10 times more data is protected than the physical space required to store it and a 20:1 ratio means that 20 times more data can be protected.  Factoring in data growth, retention, and assuming deduplication ratios in the 20:1 range, 2 TB of storage capacity could protect up to 40 TB of retained backup data.

The benefit of storing less data is obvious when it comes to reducing storage requirements and saving money.  Another way of looking at it is that storing data more efficiently on disk allows for either longer retention periods or the “recapture” of disk and the ability to extend disk-based data protection to more workloads, contributing to improvements in recovery objectives.  Efficient storage of data on disk can be a catalyst to reduce or eliminate tape media.

When it comes to selecting deduplication technology, many factors should be considered.  After surveying organizations using or considering using deduplication, ESG research found that, not surprisingly, the cost of the solution was the most frequently-cited factor (although savings garnered from capacity reduction often overcomes financial objections to deploying deduplication).  Otherwise, the data suggests that ease of deployment and ease of use, as well as the impact on backup/recovery performance were important considerations—more so than technical implementations, such as inline or post-process approaches or the deduplication ratio (see Figure 3).[4]

Data deduplication is a ground-breaking technology that changes the economics of disk-based backup and recovery, so the decision to adopt it should be easy.  However, organizations must familiarize themselves with the many facets of deduplication solutions and consider them prior to purchase—this can make the evaluation and selection of a solution more complex.

Deduplication in Secondary Storage Processes

Deduplication is a feature of both software- and hardware-based data protection solutions.  Vendors offering this feature have taken different approaches to how, where, and when deduplication occurs and possess varying limitations in the scope of deduplication.

How Deduplication Occurs

Deduplication solutions either have knowledge about the data in the backup stream or they don’t.  Those that do are content-aware—they can look at patterns in the data stream (the bytes that make up a file) and determine the optimal segment boundaries, which maximizes the likelihood of identifying duplicates.  Backup software understands the content, whereas target-side deduplication solutions typically do not. Targets simply receive a “stream” of data from the backup application.  Those target-side deduplication devices that are content-aware typically have to extract the metadata associated with the backup and “reverse engineer” the backup stream to understand its contents.

Hash-Based Algorithms

Deduplication solutions may depend on a hash algorithm to determine redundancy.  Traditionally, hash algorithms were used to compare data read vs. data written by performing a calculation on a “chunk” of the data.  If the result is identical, then the data read is the same as the data written.  The concept has been applied to identifying unique data as a method of “fingerprinting” chunks of data.  The concept, as applied to backup, is that multiple segments of the backup data stream are fingerprinted.  The ID of each chunk of incoming data is compared against the central index. Unique IDs are stored in the index and unique data is written to disk. Any duplicates are discarded and a pointer to the existing data is stored instead.

Considerations with hash-based methods are the size and location of the index.  Hash comparisons made with a memory-resident index will be considerably faster than with a disk-based one.  The index may be kept in RAM, but its size may be constrained by the memory limitations of the solution.  An index stored on disk could grow large; however, disk seeks may impact performance.  These factors, therefore, may impact the capacity of storage contained in a single system.

Delta Differencing

Another deduplication approach is delta differencing.  With this method, there is a level of content awareness.  This means that backup streams can be compared from one to the next, i.e., the backup performed today is compared to yesterday’s backup—an approach often taken by content-aware solutions.  Only the new or changed blocks or bytes (differences) are stored.  Old or recurring blocks or bytes are discarded.  This approach may be faster than hash-based approaches, but cannot deduplicate across backup streams from different backup applications.

Pattern Matching

Other vendors use different approaches for finding replicate data.  Pattern matching doesn’t rely on using hashing; instead, this technique uses an advanced pattern recognition and differencing algorithm to find and keep track of duplicates.  Like delta differencing, this approach examines the incoming data stream to see if similar data was received in the past.  However, that similar data is further scrutinized to find any differences and only the unique bytes are saved.  This method may be faster than traditional hashing methods because it is less CPU- and memory-intensive.  The size of the index is smaller than with traditional hashing methods, often resulting in greater levels of performance and scalability.

Where Deduplication Occurs

In data protection, deduplication can occur at one or more places in the data path: at the system being backed up (source-side deduplication), the backup media server (proxy deduplication), or the destination storage device (target-side deduplication).

Some backup applications deduplicate at the data source via client agent technology.  In this case, client software running on an application server identifies and transfers only unique data to the backup media server and target storage device, providing greater network efficiency.  Other backup software solutions deduplicate the backup stream at the backup server—removing any performance burden from production application servers.  Further, some distribute the deduplication process throughout the data path, performing hashing at the client and deduplication at the media server.  It will be important to understand if and/or how deduplication solutions optimize performance and distribute the deduplication workload in software-based approaches.

Deduplicating data after it has passed through the media server is referred to as target-side deduplication.  This approach typically leverages powerful purpose-built storage appliances to accommodate processing of the entire (non-deduplicated) backup load either pre- or post-ingestion.

There are pros and cons to every approach, and selecting one over another depends on multiple factors, such as performance requirements, flexibility, scalability, and cost.  One of the drawbacks of a software approach is that adopting that feature could require a switch or upgrade in backup application or client agents. However, software-based deduplication may offer more flexibility, especially for disk vendor selection. As a built-in feature of backup, a big benefit may be no added cost.  Performance may or may not be a drawback.  This will depend on the characteristics of the hardware where deduplication takes place, whether or not deduplication processing is distributed, and the aforementioned method for identifying duplicates.

Performance is often considered less of an issue with hardware-based deduplication as it typically leverages powerful purpose-built storage appliances.  The trade-offs could be flexibility (in disk vendor) and scalability, depending on the solution.  The key is finding a solution that allows for capacity and performance growth without necessitating a “forklift” upgrade.  This could mean a single highly-scalable system or multiple systems that are managed and monitored from a single management interface.

When Deduplication Occurs

Deduplication can occur before data is written to disk (inline processing) or after it is written to disk (post-processing). Inline approaches inspect and deduplicate data at the source, at the media server, or upon ingest at the disk.  The tradeoffs with this approach are related to performance, which depends on a few factors such as how duplicates are identified, the granularity of deduplication, how the deduplication processing workload is distributed, network performance, and more.  An inline approach may be preferred for workloads if replication to an offsite location is needed immediately.

Post-process deduplication will write the backup image to disk before initiating deduplication, which allows the backup to complete at full disk performance. Oftentimes, the trade-off with this approach is the amount of disk capacity required for the solution as disk capacity will be required to temporarily store the backup stream plus the deduplicated backup.  Some post-process solutions perform deduplication on a job-by-job basis, acting on data as it arrives and releasing the space once the deduplication process is completed for that job, while others deduplicate as data is ingested, minimizing the need for a landing area.  A post-process approach may be preferred if the workload includes a lot of new data, if the backup window is small, or if replicating data to an off-site location can afford some lag time.

A few vendors offer both inline and post-process options on a per job basis, which offers additional flexibility.  This makes it possible to customize deduplication strategies for specific workloads.

Deduplication Domains

The deduplication domain refers to the realm of data used for subsequent comparisons when identifying duplicates.  Local deduplication only compares data against other data passing through the same system.  Most target-side deduplication solutions fall into this category.  The good news is that this approach is more field-proven.  The bad news is that local deduplication is often the consequence of scalability limitations.

Conversely, global deduplication makes comparisons within and across systems. This capability is more often seen in software-based and grid-architecture approaches, but may also be supported for target deduplication systems that replicate in a hub-and-spoke fashion (with global deduplication occurring at the hub).  Global deduplication can result in higher deduplication ratios as data is deduplicated within and across backup sources, and greater economies of scale with respect to operational overhead and capital costs.

Another aspect of deduplication domain is the storage tiers where deduplication can be applied.  Target-side deduplication solutions are limited to disk-based storage, while backup software with media management capabilities may extend to the tape tier, too.  The ability to move data in the compressed state from disk to tape introduces capacity savings for long-term archiving.

Considerations for Evaluating Deduplication

Ease of Deployment and Use

As previously noted, ESG research found that the ability to integrate with existing backup processes and overall ease of use are of greater importance to users than more specific technical considerations. If a deduplication solution is not easy to manage and does not benignly integrate with existing data protection processes, even the best-performing product with the latest whiz-bang features will be a non-starter.

Software incorporating deduplication is either going to be the easiest to deploy and use—or be the most disruptive.  This will depend on if a switch from incumbent software is required and whether or not disk is already incorporated in the backup process.

Hardware-based deduplication solutions have garnered popularity as they are easy to deploy and less disruptive to existing backup environments and processes.  Most vendors’ solutions are delivered as target storage systems, appearing as a file server over Ethernet or as a VTL over Fibre Channel.  They offer a plug-and-play experience and don’t require client software.  Compatibility with existing backup software depends on the deduplication approach since content-aware target-side solutions require some development efforts for each backup application supported.  One of the drawbacks to this approach is that the quantity of backup applications compatible with a target deduplication solution may be limited—and the vendor may be slow to add support.

Performance

To understand the optimal deduplication strategy, organizations need to examine backup data sets—size, frequency, criticality, and whether or not deduplication makes sense—to determine a deduplication strategy and how it impacts the overall performance of backup.  Providing policy-based deduplication (the ability to turn it on or off depending on the workload and its requirements) gives the flexibility to enable deduplication for data sets with lower backup performance requirements or high data redundancy, and disable deduplication for data sets with high backup performance requirements or little data redundancy.  Policy-based deduplication also extends to determining whether inline or post-processing implementations occur.  Post-process deduplication approaches have less impact on backup windows, while an inline method could introduce some impact on performance.

Deduplication solutions require regular housekeeping operations called “cleaning” or “garbage collection.”  This process reorganizes stored data—reclaiming space freed by expired data and consolidating free capacity.  Housekeeping operations could impact performance, so it is important to understand if and how the process can be scheduled to avoid peak backup windows.  Enabling housekeeping during an evaluation will provide a real-world simulation of system performance.  Importantly, when evaluating deduplication solutions, it’s important to test over an extended period of time—a minimum of one to two weeks or several backup cycles.

Recovery performance is equally important. With deduplication making it much more economical to store backup data sets for longer periods of time, it is more likely that data will be restored from deduplicated data. It will, therefore, be important to test how a deduplication engine performs in several recovery scenarios, especially for data stored over a longer period of time, to judge the potential impact of deduplication in the environment.

Scalability

Data deduplication should mitigate the need to expand storage capacity.  However, it is still important to understand what the upper threshold of capacity is for the solution and, when additional capacity is required, how easy or difficult it is to augment it.  For example, can the solution’s repository expand on a per system basis or will a device upgrade require a new system to be deployed and data to be migrated?  The worst case scenario is for the IT organization to manage an ever-growing number of independent silos.

Manageability

Manageability is a key concern that is often overlooked.  Backup is typically managed from the backup application.  Configuring policy settings, monitoring operations, and reporting results and statistics are centralized.  Adding target-side deduplication creates another point of management where deduplication-specific policies must be set.

While target-side deduplication is simple to implement, what is the management impact as backup capacity grows and the environment scales?  Fewer silos mean fewer points of management; therefore, what are the long-term prospects for managing the backup environment with multiple target deduplication systems?  Does each device have to be managed individually?  Can data be deduplicated across target devices or is each a silo?

Centralized management of policies simplifies administration, decreases complexity, and reduces operational costs.  Backup software with deduplication consolidates policy management and may provide better visibility of operations.

Offsite Copies

Typically, disk-based backup with deduplication is a replacement for tape-based backup.  If that’s the case, then how can backup sets be moved offsite for DR purposes?  Hardware-based deduplication solutions often offer device-to-device remote replication.  While there may be an added cost for acquiring and deploying a second system at a remote location, doing so will provide a safeguard in the event that the primary site (or the backup set managed at that site) is unavailable.  Some backup applications can also replicate data from site to site. It’s important to understand if the data replicated between sites—by either hardware or software solutions—maintains data’s deduplicated state to optimize bandwidth.

For many environments, physical tape creation is still necessary to fulfill DR and retention requirements.  Most backup software solutions and some VTLs with deduplication offer the added capability of creating physical tape media.  Most solutions supporting tape must “reinflate” data prior to it being copied to tape, eliminating the benefits of deduplication.  One backup solution does offer the ability to move deduplicated data from disk to tape, minimizing the number of tapes required to store data for long-term archiving.  When a recovery is required, tape-based deduplicated data must be copied back to disk, and will then be available to the end-user or application.

Summary

Before seeking out specific vendors, it is most important to understand the organization’s deduplication needs to ensure the right fit for the environment and ease of integration.  This process includes some capacity planning to make sure that the pursued solution will have some longevity and that capacity scaling needs are well understood before purchase.

Once a vendor short list is determined, vet the company and its deduplication product.  Seek out references, understand how many active deployments of the technology are in place, leverage the vendor’s ROI model, compare the results versus competing solutions, and get a glimpse into the product roadmap.  It is important to understand the vendor’s business success and long-term viability, its support capability, how well it communicates with you, and what other services or products it could offer to you today and over time.

Next, test, test, test.  Beginning with installation and configuration, test viable solutions using real data based on policies in place in the current data protection environment.  Record deduplication ratio results, backup and recovery performance (single stream and aggregate performance), and replication performance over an extended period of time—a week or two weeks at a minimum.  These tests should include deleting and expiring data, as well as simulating expected change rates.  If applicable, test the physical tape creation process.  Finally, simulate system failure to test resiliency.

Choosing a deduplication strategy is not a simple task.  Technology maturity varies considerably and the vendor landscape is in flux.  As solutions are considered, cut through the hyperbole by requesting real-world references and proof points to vendor claims.  Test backup and, importantly, restore performance.  Thorough due diligence up front may stave off surprises later.

[1] Source: ESG Research Report, Database Archiving Survey, December 2007.

[2] Source: ESG Research Report, Enterprise Storage Survey, December 2008.

[3] Source: ESG Research Report, Enterprise Storage Survey, December 2008.

[4] Source: ESG Research Report, Data Protection Market Trends, January 2008.

Introduction

By now, it should be clear that buying an e-mail archive solution is a logical investment.  However, at the end of 2007, ESG estimated that nearly two-thirds of organizations had yet to purchase a purpose-built e-mail archive solution.[1] While that number likely decreased over the past 14 months, plenty of companies still manage e-mail using outdated methods, including enforcing mailbox quotas, retaining messages on backup tapes, and constantly buying more storage and servers in support of e-mail applications.  Recent ESG research confirms the latter as almost half of the 550 organizations surveyed stated that e-mail would significantly impact server and storage infrastructure spending over the next two years.[2]

ESG believes that one reason why companies have yet to make an e-mail archive investment is poor information, resulting in skewed Return on Investment (ROI) calculations-a process that most companies go through before making any IT purchase.  Some organizations fail to take into account all of the achievable benefits made possible by e-mail archiving, which limits the “return” metric in an ROI modeling exercise.

Inaccurate ROI modeling can lead to uninformed decisions about e-mail archive solutions, often leading customers to maintain the status quo when it comes to message management.  This outcome can actually be much more expensive in the long run as e-mail storage capacity will spiral out of control, e-mail application performance will eventually degrade, and litigation and compliance costs will increase exponentially.  This makes it extremely important for customers to understand all of the factors that should be included in an e-mail archive ROI exercise.

Knowing which e-mail archive solutions can deliver all the benefits that ultimately deliver the “return” and minimize the “investment” is also critical for customers.  This paper is organized to assist customers in building more informed e-mail archive ROI models-regardless of whether they are looking at a new purchase or replacing an existing implementation-and highlights how one solution, EMC SourceOne Email Management, makes it possible to achieve the results that help customers justify a purchase in the first place.

Reminder:  E-mail Management Challenges

Two intertwined factors make e-mail management an ongoing challenge for most IT departments: data growth and data retention.

• E-mail data growth is unlikely to subside because messaging applications are the primary means for collaboration and information sharing in almost every corporation. ESG estimates that overall corporate data growth will be 25% in 2009[3]-and e-mail comprises just over 20% of corporate data.[4] With attachment sizes getting bigger (rich media, etc.) in addition to Unified Communications Systems being put into place, there is a high likelihood that going forward, e-mail will grow at two to three times the rate of overall corporate information, forcing IT to figure out a way to constantly manage and store more and more messages.
• Most organizations are now retaining e-mail for longer periods of time for regulatory compliance, electronic discovery, corporate governance, and knowledge management. With so much business being conducted via e-mail, messages and attachments are now business records and therefore subject to the laws governing the retention of that information. Healthcare, energy, local governments, education, telecommunications, and other industry governing bodies are joining financial services regulators in issuing interpretations to existing record retention rules that include e-mail. Additionally, ESG estimates that 80% of electronic discovery inquiries include e-mail[5]-by far the most common data type requested. As a result, companies must identify and preserve a subset of all relevant messages and attachments based on the scope of the discovery inquiry.[6] Failure to save the right business records or preserve evidence can lead to fines or unfavorable legal outcomes.

Some organizations are voluntarily keeping messages in support of corporate governance and knowledge management initiatives.  For example, companies are retaining merger and acquisition correspondence between a finance team and an external investment banker.  If a deal is postponed and then rekindled a few months later, finance employees can see what work has already been done and who the lead banker contacts were.  Other organizations are retaining customer service related e-mails and making them available to sales teams, enabling the latter to remain aware of the issues may be occurring with their respective clients.  These companies are taking a proactive approach to e-mail retention because they find value in new and historical content and believe that saving it is the right thing to do as it provides benefits to key stakeholders while enhancing employee productivity.

When data growth is combined with the average retention periods for e-mail (see Figure 1), IT suddenly finds itself dealing with out of control storage costs, longer backup and recovery operations, and poor e-mail application performance.  Meanwhile, compliance officers, corporate counsel, and records managers need to implement e-mail management policies in addition to accessing messaging systems on a regular basis just to do their own jobs.  Employees may have to manage their own inboxes just to save data that is worth keeping and valuable productivity hours can be lost if knowledge workers cannot easily locate new and old messages and attachments.

Old Methods Do Not Work

The easy way to solve e-mail storage challenges is to delete data, which is not always an option because of the aforementioned regulatory and legal requirements.  A derivative process of deleting e-mails is the mailbox quota. IT creates a policy establishing a maximum size of a mailbox.  If there is an infraction, an employee may not be able to send or receive messages until they reduce the size of their inbox, which essentially forces employees to manage their own inboxes.  Employees may choose to delete messages or create personal archives-the latter complicates electronic discovery processes as attorneys have to find each and every personal archive when looking for relevant messages after receiving an inquiry.  Personal archives also do not solve the e-mail storage problem; they merely shift it as employees save these messages on PCs or shared file servers.   Additionally, if a laptop is stolen or breached, there is a significant risk that personal archives (PST and NSF files) will be compromised, potentially leading to privacy concerns and intellectually property theft.

Mailbox quotas may have little impact on the overall e-mail storage environment, but they can impede employee productivity as workers spend more time managing inboxes rather than creating and responding to critical business communications.  Another ineffective e-mail management method is using backup tapes to retain and preserve messages for long periods of time because:

• Companies save more information than required. All messages on a specific tape are saved even if some of them do not need to be-companies cannot select what data on the tape can be deleted versus what must be retained. Additionally, all messages on the tape are retained for the same period of time as the tape will be saved for the longest retention period (i.e., if some messages need to be saved for 90 days while others need to be kept for 2 years, the tape will be kept for two years.) Saving more data than needed-or saving data for longer than needed-increases storage costs as more media is needed to retain all the data. Electronic discovery expenses also increase as corporate counsels have more data to search through.
• Tapes are not easily accessible or searchable, making it an expensive endeavor if corporate counsels have to look for old messages. If the tapes are offsite, companies must pay a service provider to recall them. Hopefully, the tapes still exist, which is not a guarantee if the data is older-the media may have been misplaced over the years. Then, IT departments have to restore the data from tape, index it, and turn it over to attorneys who can then look for relevant e-mails. Very often, IT staffs do not have the resources to complete this exercise in a short period of time, forcing them to outsource-an expensive proposition given that service providers can charge between $500-$2000 (depending on the size and data format) per backup tape.

Measuring an E-mail Archive ROI

E-mail archive solutions enable customers to move or copy e-mails to a tertiary environment (an archive), allowing them to control how this data is retained and managed.  Users can determine what storage the archived messages are saved on, set access permissions for the content, and consistently enforce retention policies against the now centralized information.  With data in an e-mail archive environment, organizations can:

• Have the option of enforcing mailbox quotas within the primary e-mail environment, but move older messages to the archive (as opposed to deleting them) or eliminate mailbox quotas altogether by continuously moving messages into the archive. When in the archive, messages can be stored on lower cost systems and still remain accessible to employees.
• Eliminate the need for employees to manage their own inboxes by creating personal archives to avoid quotas or simply to save messages for future reference.
• Minimize long term storage costs by single instancing any duplicate messages and deploying lower cost systems to retain archived content (a concept often referred to as tiered storage).
• Implement consistent electronic records management programs by accurately retaining a subset of messages based on sender or recipient, date, and other criteria.
• Save messages within a “tamper proof” archive as retention policies can be enforced to meet regulatory compliance, legal hold, and corporate governance mandates. Customers can control who has access to the messages, preventing unauthorized deletion or alteration of content.
• Facilitate more efficient electronic discovery as corporate counsels have one central location to search for relevant messages after receiving a discovery request.
• Improve e-mail application performance by moving older data to the archive.
• Reduce the time it takes to complete e-mail application backups and restore operations because there is less data in the primary environment to copy.

To build an accurate e-mail archive ROI model, customers must translate these benefits into metrics that impact their business.  Some of the metrics that customers should consider including in their ROI are:

• Time savings
  • Employees no longer have to manage their own inboxes due to mailbox quotas
  • IT can quickly execute searches against an archive to identify all messages and attachments that fall within the scope of an electronic discovery request as opposed to looking across the primary message environment, multiple employee PCs and file shares (for personal mail archives), and backup tapes (for older messages)
  • Corporate counsel spend more time reviewing evidence rather than waiting for IT to collect it
  • IT does not have to constantly tune or run additional servers to improve e-mail application performance
• Capital expense reduction
  • Save a subset of information for the appropriate period of time, rather than saving everything forever
  • Only save one copy of a message and attachment
  • Avoid purchasing more servers simply to boost e-mail application performance
  • Store older messages on lower cost, yet still accessible, storage systems
• Risk reduction
  • Avoid fines by saving all business records for the required retention period
  • Mitigate chances of evidence spoliation and unfavorable outcomes/settlements by properly preserving e-mails on legal hold

To complete an ROI model, customers must also understand the “investment” portion of the equation.  This measurement goes beyond the initial acquisition cost of the e-mail archive software.   They must also consider any additional servers and storage needed to run the archive software and store the content, including the index used to support archive searches.

There is also the operational investment customers must make to configure, monitor, and maintain the archive environment.  Customers must find ways to lower these operational costs over several years because an archive environment is not a one or two year implementation.  Longer retention periods mean the archive environment will need to be upgraded as the supporting infrastructure ages or moves off of warranty.  As such, companies must consider e-mail archive features that allow data to be stored on several storage systems and enable the repository to be moved to different devices if one system needs to be upgraded or replaced.  These capabilities actually reduce the ‘Investment’ metric a company should measure when it comes to creating the ROI.

Another parameter of the “investment” metric that should be accounted for is how easy it is for authorized employees to use the e-mail archive software and access content within the archive environment.  Employees should be able to seamlessly work with archived messages from their inbox, regardless of connection or device.  Further, an archive solution needs to support various search roles-employees should only see their messages, but corporate counsel or compliance officers may need to query the entire archive.  It is very difficult for companies to place a value on the end-user experience, but coming up with a scorecard for this part of the investment measurement is worthwhile-it is pointless to retain messages if no one can access them.

EMC’s Impact on the E-mail Archive ROI Quotient

After an organization outlines all of the potential measurements an e-mail archive ROI model should contain, the next step is to evaluate solutions that can maximize returns while lowering the overall investment.  This is a simple concept, yet many companies rarely connect an e-mail archive solution’s features with the potential benefits or cost savings and update their ROI calculations.  Missing this step means that an e-mail archive solution may not deliver the returns originally anticipated.  Organizations can correct this misalignment simply by listing all of the benefits and required investments, along with their associated measurements, and then building an ROI for each product.  Some solutions will have features that drive up the benefits, but cost a lot to acquire and manage. Others may not offer as many features, but are inexpensive to operate.

If a customer puts EMC SourceOne Email Management, the company’s recently announced e-mail archive offering, through this exercise, they may realize that several of the solution’s benefits can be achieved with minimal cost.  Some of the benefits that customers can expect to experience by deploying EMC SourceOne Email Management include:

• Flexible message archiving options. SourceOne supports Exchange and Lotus environments and archives e-mails three ways: via a journaling feature within these messaging applications, by allowing users to move messages into the archive directly from their inboxes, or automatically executing archive policies (i.e., archive any messages older than 90 days) against the primary messaging application. These options make it very easy for customers to archive content for regulatory or discovery purposes-the journaling option is best suited for these scenarios. For mailbox and records management implementations, user-based or automated archive policy enforcement approaches are ideal to help control inbox sizes or categorize a subset of messages as non-regulated business records for corporate governance or knowledge management purposes.

  In addition, EMC SourceOne Email Management also supports automated collection of personal archives such as PST and NSF files, allowing companies to quickly find and archive these messages based on corporate policies.  This helps customers centralize all of their historical messages so that attorneys do not have to go to every PC looking for PST and NSF files and IT does not have to worry about backing them up as part of laptop/desktop or file share data protection operations.

• Rich retention policy management. Customers can archive messages based on several message metadata criteria, including the sender/recipient, the department in which the sender/recipient works (based on Active Directory or other LDAP grouping), and when the message was sent/received. In addition, customers can establish different retention policies for a category of messages. For example, all messages sent from the executive team can be saved for one year, while messages sent by the legal department can be kept for three years. For legal holds, customers can archive information from a set of custodians (employees) indefinitely. To save storage costs, customers may simply archive all messages that are older than 90 days and save them for one year. With the ability to archive messages based on several criteria and establish different retention periods, EMC SourceOne Email Management customers can archive information to meet a variety of IT and business requirements.
• Lower storage costs. The SourceOne Email Management repository can be stored on a multitude of storage devices, which gives customers several options to deploy lower cost systems in support of the e-mail archive environment. One of the storage systems supported is EMC Centera, which can be configured to save the data in WORM format. WORM storage can be used to meet specific financial services regulations (SEC Rule 17a-4) as well as other implementations, including legal hold situations, where customers want to prevent any modification or deletion of content during the retention period assigned by the SourceOne Email Management archive software.
• More efficient electronic discovery processes. SourceOne Discovery Manager, a companion offering from EMC that is integrated with the SourceOne Email Management product, provides a separate search and legal workflow interface that enables corporate counsel to query the entire archive repository (or several repositories) for relevant content. In addition, Discovery Manager enables corporate counsel to tag messages as privileged, responsive, or add a case identifier. These tags allow the messages and attachments to be filtered or grouped together and then queries can be run against a specific subset of messages with the goal always being to enable corporate counsel to quickly understand how much data exists and reduce the amount of data that has to be reviewed. SourceOne Discovery Manager provides corporate counsel and their teams with their own workspace within an e-mail archive so they can be more productive.
• Opportunities to build an integrated content archive. Because of its growth, regulatory requirements, and its popularity as a source of electronic evidence, companies should begin archiving projects with e-mail. However, there are many content types other than e-mail-including enterprise application files and web page forms-that may be considered business records or requested as part of an electronic discovery. This information may also be extremely valuable to business processes and knowledge management programs if it is made accessible over longer periods of time. At some point, customers may want to archive this data along with e-mail to unify their information retention strategies. Those organizations can deploy SourceOnce Email Management to start and then other EMC SourceOne solutions, which enable customers to archive other content types. With EMC’s investment in XML, federated search, and other unifying technologies, it is foreseeable that customers will eventually be able to manage and retain information from a single EMC repository. This repository would make it feasible to consistently set and enforce retention policies across multiple content types, and employees-including corporate counsel, records managers, and knowledge workers-will be able to easily find all relevant data swiftly.

Customers also have to weigh the investment they have to make in acquiring and running SourceOne Email Management.  The acquisition costs are fairly straightforward as SourceOne Email Management is priced on a per mailbox basis.  SourceOne Discovery Manager requires an additional license and is also priced on a per searched mailbox (also referred to as a “per custodian”) basis.    These aggregate acquisition costs can be offset by many of the underlying SourceOne Email Management architectural elements, which can reduce overall archive infrastructure and operating costs and minimize the storage footprint.  Some of the operational savings can be attributed to:

• A scalable application architecture, which lowers e-mail archive infrastructure costs. A single SourceOne Email Management archive server can support several thousand mailboxes. The less hardware needed to run an e-mail archive environment, the simpler the overall implementation is to manage. Fewer servers also reduce the upfront capital expenditures needed for the initial archive deployment.
• A distributed architecture keeping the archive infrastructure highly available. Customers can implement SourceOne Email Management on multiple servers or multiple virtual servers (“guests”) on a single server. With multiple servers, the archive tasks (message capture, indexing, search, etc.) can be distributed, improving overall performance of the archive environment. More importantly, if a server fails, another application server will automatically complete the tasks from the failed server. The SourceOne Email Management application also has the ability to isolate any failed tasks so that it does not continuously retry the operation and negatively impact the performance of the overall archive environment.
• The ability to non-disruptively integrate with existing e-mail environments, reducing the implementation burden on IT. SourceOne Email Management does not require any changes to the primary messaging environment, aside from an administrator enabling the journaling function within Exchange or Domino. Other solutions may require configuration modifications, such as the Lotus Notes mail templates.

E-mail archive storage capacities expand exponentially, forcing customers to constantly purchase and run more and more storage devices.  SourceOne Email Management can generate incremental savings by helping customers reduce the overall archive storage footprint by:

• Single Instancing redundant content across all e-mail archive servers and compressing the messages and attachments before they are stored. Any duplicate messages or attachments sent to the SourceOne Email Management repository will be removed, ensuring customers only save one copy of an e-mail or file. Additionally, the archive software compresses all content before it saves it to disk.
• Providing several indexing options. Customers can choose whether or not they want to create a metadata or full text index within a SourceOne Email Management implementation. Indexing options enable customers to align information access requirements with storage costs as a metadata index consumes less storage than a full text index. For example, if customers are using SourceOne Email Management for legal hold purposes, they may simply want to filter messages by employee and date range within SourceOne Discovery Manager, requiring a metadata index to complete the task. In comparison, if messages are being retained for knowledge management reasons and employees need to search the archives based on keywords found in the body of a message or attachment, e-mail archive administrators should enable full text indexing. In addition, customers can turn on the “content caching” option, which creates an additional index in the form of a text file. If the primary index is corrupted, customers can rebuild it from the text file quickly, rather than re-processing all of the messages and attachments-an operation that can take several hours and reduce the overall performance of the archive.
• Easily and consistently disposing of content. Once a message or attachment’s retention period expires within SourceOne Email Management, the archive application can automatically delete the content from the repository or notify archive administrators of the candidates that are now eligible for expiration. This prevents customers from saving data longer for than is necessary, which can save storage capacity. More importantly, it enables organizations to consistently dispose of information once a retention period expires during the normal course of business to mitigate legal risk-a process that complies with the amended US Federal Rules of Civil Procedure (governing civil litigation processes, including electronic discovery requirements).

One investment metric that often gets overlooked during the ROI modeling process is a solution’s ability to facilitate archive access.  There is little point in retaining messages and attachments if authorized employees cannot search for and retrieve them easily.  SourceOne Email Management enables employees to access content from within the native e-mail application clients (Outlook and Lotus Notes) as well as from any web browser via Outlook Web Access and Domino Web Access.  Customers can also access messages from Entourage if they are running Exchange with Mac clients.  When stubbing messages in mailbox management implementations, SourceOne leverages a feature termed the “Universal URL.”  This URL enables an employee to access an archived message from their inbox via their PC, a mobile device, or any web browser.  In Exchange environments, customers can also cache archived messages so the data is available even if employees are not connected to the archive.  For Lotus Notes users, the use of the Notes local replica facilitates access to message shortcuts even when they are offline.

Many of these SourceOne Email Management capabilities help lower the overall investment in the e-mail archive solution because IT does not have to dramatically change the primary messaging environment to start retaining messages, employees can access archived content from almost anywhere and do not need significant training to do so, and several options are available to help control archive-related storage costs.

By building a list of benefits and potential “returns” along with acquisition and operational “investment” metrics, companies can make more informed decisions before making an e-mail archive purchase.  They can also measure their own effectiveness by comparing actual results to their ROI models after implementing an e-mail archive solution-an exercise that may lead to optimizing the deployments for additional benefits or lower operating costs.

The Bottom Line

Any organization thinking about e-mail archiving should be aware of all the benefits that current customers are experiencing (see Figure 2).  These data points should serve as a catalyst to move an e-mail archive project to the top of any IT executive’s list-a project that should also garner support, and potentially budget from the legal and compliance departments as they stand to gain from e-mail archiving as well.

The next logical step is to build an ROI model; however, this is an exercise in futility if a company does not take into account all of the benefits and costs involved in an e-mail archive implementation.  Further, companies should develop an ROI model that is specific to each individual offering they are evaluating as some solutions do not have the capabilities to deliver all of the benefits and others are more expensive to operate.

EMC, with the SourceOne Email Management solution inclusive of the scalable, highly available architecture and storage optimization capabilities, is on its way to bucking a trend: the most functional e-mail archive solutions are also more difficult to operate over the long run.  And, with several feature enhancements and additions along with a new architecture that reduces e-mail archive infrastructure and operating costs, it appears EMC is on its way to accomplishing its goal of maximizing e-mail archive returns with the lowest possible investment.  The ultimate proof will be in the numbers.

[2] Source: ESG Research Report, Medium-Size Business Server & Storage Priorities, June 2008.

[3] Source: ESG Research Report, Enterprise Storage Survey, November 2008.

[4] Source: ESG Research Report, Medium-Size Business Server & Storage Priorities, June 2008.

[5] Source: ESG Research Report, 2007 E-mail Archiving Survey, November 2007.

[6] Ibid.

Introduction

The Hitachi Data Systems Adaptable Modular Storage (HDS AMS) 2000 family is a series of next-generation midrange storage systems which use enterprise-class technologies to improve operational efficiencies and significantly reduce a company’s administrative burden. This ESG Lab Report documents the results of hands-on testing of the AMS 2000 with a focus on ease of implementation and management, availability, and agility as well as performance and capacity scalability.

Background

Today’s microeconomic climate is driving users to leverage advanced infrastructure and management technologies, particularly in the storage and server areas, to better support top-priority business requirements. At the same time, these technologies help customers improve resource utilization by enabling IT teams to minimize time spent on routine operations and support activities. Medium-sized organizations—defined as having between 100 and 999 employees—are under increased pressure to more efficiently support their businesses while improving information management and keeping costs down. Storage technology providers, in turn, have traditionally responded with storage systems that include scaled-down and compromised feature sets. But medium-sized businesses are increasingly in search of “enterprise-class” information management and infrastructure technologies—and these organizations are making significant investments in networked storage as a key enabler for improved information management.

As shown in Figure 1, recent ESG research shows that the top IT infrastructure initiative influencing server and storage spending in medium-sized businesses during the next 24 months will be improvement and optimization of IT service levels.[1] Other initiatives with major impact include application and database deployment/upgrades and business continuity and disaster recovery programs. In each case, addressing an initiative’s priorities requires comprehensive IT services to support new and existing applications and processes while optimizing day-to-day workflows and business processes.

HDS Adaptable Modular Storage

HDS has focused on these customer trends and priorities with its AMS 2000 family of next-generation midrange storage systems. These flexible, modular storage systems are designed to address the most crucial challenges faced by storage and IT managers. An AMS 2000 storage system can be configured with as few as four and as many as 480 drives in a single array as shown in Figure 2. Performance of the new AMS 2000 family has increased dramatically and all systems support Fibre Channel or iSCSI host connections.

The HDS AMS 2000 family’s features include:

• Enterprise-class Technology – Symmetric, active/active controllers with load balancing.
• Green IT – Automatic power savings with SATA-II drives and programmable spin down for all drives.
• SAS Architecture – Up to 32 wide point-to-point links to the back-end with fully intermixable SAS & SATA-II.
• Enhanced Mega LUNs – Up to 60 TB LUNs for simplified performance and scalability.
• Cache Partitioning – Cache can be allocated to applications according to their requirements.
• Modular Volume Migration – Dynamic, online, non-disruptive volume migration.
• Online RAID Group Expansion – Drives can be added to an existing RAID group while the system remains online with non-disruptive data re-striping.
• LUN grow/LUN shrink – The capability to increase or reduce the size of any selected LUN

This report documents ESG Lab hands-on testing of the HDS AMS 2000 storage family with a focus on its ability to leverage enterprise functionality to increase performance, availability, and investment protection as it reduces cost, complexity, and energy requirements.

ESG Lab Validation

ESG Lab performed hands-on evaluation and testing of the HDS AMS 2000 family at HDS’ corporate headquarters in Santa Clara, California. Testing began with a look at the Web-based Storage Navigator Modular 2 (SNM2), the management tool for HDS entry level and midrange storage systems. The SNM2 server component runs on either a Windows or Solaris server in the user’s environment; both Internet Explorer and Mozilla-based browsers are supported as clients.[2]

Ease of Implementation and Management

ESG Lab testing was conducted on pre-wired, rack mounted AMS 2300 storage arrays. The ESG Lab test bed, as presented in Figure 3, consisted of two Dell PowerEdge R905 servers SAN attached to an AMS 2300 storage array configured with 105 15K RPM SAS disk drives and sixty 7200 RPM SATA drives.[3] One server was running VMware ESX server with Windows 2008 installed as a Guest OS and the other ran Windows 2008 natively. A thirty-two port 4 Gb/sec Brocade switch was used for Fibre Channel SAN connectivity.

ESG Lab Testing

ESG Lab began by pointing a web browser at the IP address of the SNM2 server. The Add Array Wizard prompted for the IP addresses of the arrays to be managed. IP addresses on AMS arrays are set by HDS at the time of installation.

As seen in Figure 4, the Add Array Wizard can discover systems to be managed using either the specific IP addresses of controllers or a range of addresses. ESG Lab entered a range of four addresses and the wizard discovered two arrays.

Figure 5 depicts the Hitachi SNM2 GUI. The layout was straightforward and easy to navigate. The navigation pane on the left offers a familiar tree-based menu system, allowing the user to access any function in detail. Context sensitive, common task wizards are shown with friendly icons and clear descriptions on the main page. ESG Lab clicked “Create Logical Unit” to launch the Create & Map Volume Wizard.

SNM2 uses a single wizard to create new RAID groups and volumes. In this test, the wizard was used to create new volumes from raw disk and assign them to an attached host. As seen in Figure 6, ESG Lab selected the type of disk, the desired RAID protection, and the size of the RAID group. In this example, a RAID 5 group was created using fifteen SATA drives.

Next, the number and size of LUNs to be created was specified and the LUNs were assigned and connected to a server. Five 25 GB LUNs were created and assigned to the ESX server in the test bed.

The five volumes created can be seen in the confirmation screen, shown in Figure 7.

When ESG Lab clicked “Finish,” the AMS 2300 proceeded to create the RAID group and initialize the LUNs. The entire elapsed time of the process, from the first click of the wizard to the moment the LUNs were visible to the host, was less than five minutes.

ESG Lab also examined the process for creating LUNs on a previous generation AMS 1000 in order to get a feel for the differences between the two systems. Creating a LUN on an AMS 1000 was very similar to the procedure used for the AMS 2300, but with one notable exception: after the new volume was created, ESG Lab had to edit the properties of the volume to assign it to the correct controller as shown in Figure 8. This step is not required with the AMS 2300 due to an Active-Active architecture, which allows hosts to access any volume through either front-end controller.

Next, ESG Lab explored the AMS family’s ability to expand and reduce the size of existing volumes online. Both functions are executed using the same wizard, the Change Logical Unit Capacity Wizard. First, ESG Lab selected a 100 GB LUN for reduction and clicked ‘Change Capacity.’  The Change Logical Unit Capacity Wizard, seen in Figure 9, was launched. The wizard displayed the capacity of the LUN, the free capacity in the array, and allowed input of the volume’s new capacity. ESG lab chose to reduce the capacity of this 100 GB LUN by half, to 50 GB.

Clicking ‘OK’ brought up a warning screen that informs the user that shrinking a LUN is a destructive operation; the administrator must click a checkbox acknowledging this before the procedure can finish. Once the LUN was re-sized, ESG Lab performed a re-scan of the LUN from the Windows server that owned it, followed by a re-format.

Figure 10 shows detail from the Microsoft Disk Administrator before and after the shrink operation.

After re-formatting the volume, new data was added and the LUN was expanded from 50 GB to 200 GB. Using the SNM2 console, the process was d exactly the same as the volume shrink. When the volume was re-scanned using Windows Disk Management in the MMC Computer Management console, the additional capacity appeared as unused space at the end of the existing volume.  ESG Lab was able to take advantage of the additional capacity by expanding the volume using Windows Disk Management as seen in Figure 11.

ESG Lab confirmed that all files copied into the volume before expansions were still accessible and intact.

Simple, Scalable Performance

HDS has made significant changes to the architecture and design of the AMS 2000 series with a goal of improving performance while reducing cost and complexity. The AMS 2000 is the first midrange storage system to offer a high performance SAS back-end and active-active load balancing front-end controllers. SAS (Serial Attached SCSI) is a point-to-point data transfer technology and enterprise SAS drives provide the same reliability and performance as Fibre Channel drives at significantly lower cost. The SAS backplane in the AMS 2000 family can accept both SAS and SATA drives in the same enclosures.

Performance testing began with evaluation of IOPS and throughput characterization tests—often referred to as “the corners” in storage benchmark testing. Corners testing, which provides a good first impression of the theoretical limits of a storage system, was followed by tests designed to measure the performance capabilities of the AMS 2000 family when running a real-world application, Microsoft Exchange. First, ESG Lab audited tests run in HDS performance labs using the industry standard Iometer benchmarking utility.[4] Figure 12 shows the relative performance of the maximum configurations of the previous and current generation of AMS storage systems.

The “corners” tests depicted here measured three industry standard metrics: IOPS, throughput, and random I/O. IOPS (I/Os per second) are commonly used to demonstrate the processing power and front-end efficiency of the disk array, while throughput measures how much data the AMS could move from disk in a given amount of time. The random I/O workload is designed to simulate the I/O pattern that makes up the majority of traffic generated by interactive, response-time sensitive business applications (e.g. databases and e-mail).  The chart shows that HDS has significantly raised the bar, with the AMS 2000 family delivering from 1.6 to 3 times the performance headroom of the previous generation.

ESG lab next analyzed AMS family performance in an Exchange environment via an audit of Microsoft ESRP 2.0 results and hands-on testing using Iometer workloads designed to simulate a Microsoft Exchange 2007 environment. The AMS 1000 results were extracted from published Microsoft Exchange Solution Review Program (ESRP) test results, while ESG Lab performed a series of tests against the AMS 2300 to simulate a Microsoft Exchange 2007 workload using Iometer.

ESRP is a Microsoft program designed to facilitate third party storage testing and solution publishing for Exchange Server. The program combines a storage testing harness (Jetstress) with publishing guidelines for Microsoft Gold Certified and Storage OEM Partners. ESRP employs the Jetstress utility to create real exchange traffic that runs against real exchange databases—with logging and file attachments—exactly as in the real world. The testing is designed to measure both the performance and reliability of a given solution. The performance test runs for two hours while the reliability test runs for 24 hours. Both tests must run without exceeding a prescribed disk latency threshold (20 milliseconds) and a reliability test is performed to check for database and log corruption at the end of the run. Manufacturers use the ESRP framework to test storage solutions and then submit results to Microsoft for review. Approved solution results are posted on the Microsoft Exchange ESRP website.^[5]

ESG Lab Testing was performed against an AMS 2300 using the same quantity and configuration of disks employed for the ESRP test. Iometer was used to generate a workload similar to the IO generated by the JetStress to provide a quick estimate of the number of Exchange 2007 users that the system could support as compared to the previous generation architecture.

As seen in Figure 13, the AMS 2300 was able to sustain enough IO to support nearly twice the number of Exchange users than the previous generation using the same quantity and configuration of disks. It’s important to note that the ESRP published numbers are the result of a rigorous testing program audited by Microsoft to evaluate many facets of a solution in an exchange environment. The ESG Lab tested results represent a methodology that ESG Lab has developed using our experience in the field to quickly estimate performance in an Exchange environment. ESG Lab is confident that when HDS completes ESRP 2.0 testing for the AMS 2000 family their results will be similar.

ESG Lab constructed the Iometer test bed for the AMS 2300 with the same number of disks, raid protection, and layout as were used to obtain the ESRP results.^[6] The number of supported users was derived using a ‘Very Heavy’ user profile, defined by Microsoft as .48 IOPS per user plus 20% for overhead, or .576 total IOPS per user. Table 1 lists the raw results.

What the Numbers Mean

• The AMS 2300 was able to sustain IO sufficient to support 47,333 Exchange users, as projected by ESG Lab.
• Latency was also excellent, remaining below 16 ms throughout the test. This translates to a positive user experience with fewer storage-induced application delays.
• The performance observed by ESG Lab proved that the improved architecture of the AMS2300, including the switch from a FC to SAS back end, improves  performance for real-world applications, In this example, ESG has confirmed that the AMS2300 can support up to 85% more Exchange users than the previous generation AMS1000.
• The AMS 2000’s active-active architecture made volume configuration and layout for the MS Exchange tests easier and faster than with the previous generation arrays.
• The combination of additional headroom plus excellent application performance shows that HDS has substantially improved the operational efficiency of the system.

Symmetric Active–Active controllers

Medium-sized businesses are increasingly being asked to do more with less. Hosting multiple operating systems and applications on a single storage array reduces capital costs for new equipment as well as the operational costs associated with power, cooling, and data center space. A major challenge with midrange storage arrays has been the administrative effort required to manage performance. With traditional midrange storage systems, administrators must carefully map servers to resources to avoid bottlenecks and balance the load evenly across controllers, busses, and disks. This functionality is often automated in enterprise class arrays.

ESG believes that virtualized, networked storage is essential if organizations are going to unlock the full potential of their IT infrastructure. The ability to map physical or virtual machines to storage resources without the need for manual load balancing can significantly reduce the managerial effort required for deployments, implementations, and ongoing administration.

The AMS 2000 family is the only dual controller midrange storage system with symmetric active-active controllers. An AMS 2000 system behaves more like an enterprise class array, providing integrated and automated front-to-back I/O load balancing. Figure 14 compares traditional active-passive controller architecture with the AMS 2000 family. In the traditional architecture, each controller ‘owns’ a set of LUNs and each server accesses individual LUNs one path at a time. Load balancing is a manual, administrative task where LUNs are distributed between front-end controllers. If I/O is unbalanced, the administrator must manually change ownership of LUNs to the opposite controller until the I/O load is balanced between the two, potentially forcing reconfiguration of the server and SAN to maintain access.

In the AMS 2000 architecture, LUNS are not owned by the front-end controllers and servers can access their LUNs through either controller. This has two immediate benefits: administrators no longer have to assign a LUN to a particular controller and the server can be attached to the array through both controllers without having to keep track of the ‘active’ controller for each LUN. Server side load balancing can then use all available paths to each device while the array automatically performs back-end load balancing—internally without disruption.

ESG Lab Testing

ESG Lab evaluated the active-active, load balanced front-end architecture of the AMS 2300 with a single Windows 2008 server SAN attached to the AMS system using two Fibre Channel HBAs. Iometer was used to generate traffic to a single LUN on the AMS 2300 array. A bandwidth intensive 512 KB sequential read workload, similar to a nightly backup job or video streaming application, was run for five minutes. No other I/O was running against the array during this test. As shown in Figure 15, Hitachi Dynamic LUN manager was managing the paths to LUN 0004 on the host. The server was accessing the volume through ports 0D (on controller 0) and 1A (on controller 1).

Iometer reported an average of 676 IOPS (338MB/sec) overall for the test run. Figure 16 shows the traffic as seen from the SNM2 Performance Monitor. The two charts show IOPS on Controller 0 and Controller 1. Each controller is reporting between 300 and 350 IOPS over the five minutes of the test run, perfectly in sync.

ESG Lab examined the SNM2 Performance Monitor periodically throughout testing. Without exception, traffic was evenly balanced between the front-end controllers during every test.

Availability – Remote Replication

The HDS AMS 2000 family provides a broad range of simple and fast volume protection services that enable organizations to provide a highly available storage service to their users across multiple generations of storage systems. Hitachi Copy-on-Write software provides capacity-efficient snapshots, Hitachi ShadowImage Replication software creates point-in-time local clones, and Hitachi TrueCopy software offers synchronous and asynchronous replication.

ESG Lab Testing

ESG Lab tested Hitachi TrueCopy software in synchronous mode between two AMS family storage systems attached to separate SAN domains to simulate an intra-campus disaster recovery strategy. As shown in Figure 17, data in production servers is protected by synchronously mirroring the volumes to a second on-campus data center.

ESG Lab provisioned the ‘Production’ server, a Windows 2003 virtual machine running under VMware ESX, with two 25 GB volumes provisioned on the ‘local’ AMS 2300 array. Two 25 GB target volumes were assigned on a ‘remote’ AMS 1000 array that was connected to the ‘local’ AMS2300 via 4 Gb/sec Fibre Channel. Next, ESG Lab configured synchronous remote mirroring between the two arrays using the SNM2 Create Remote Path Wizard, seen in Figure 18.

ESG Lab generated I/O on a virtual machine in the local data center by running a continuous 4 KB random write workload using Iometer and by playing a video file stored on the mirrored volume. Figure 19 illustrates the steady state, with applications running on the source side and volumes being mirrored between arrays. In a full disaster recovery configuration, servers would use clustering technology to fail applications over to the target data center.

ESG Lab verified I/O was being read from and written to the correct volumes on both arrays. The total time, from first click to volume synchronization, was less than two minutes.

The next step was to simulate a major failure. This was done by physically disconnecting the Fibre Channel ports between the two arrays. ESG lab then assigned the volumes to a server attached to the AMS 1000 and verified that the volumes were accessible by playing back the video and copying additional files into the volume.

A failback was then performed by reconnecting the FC links between the systems, removing the volumes from the remote server and re-establishing the path between the two arrays. Once connected, the new data was synced back to the production side and ESG Lab verified that all data written during the ‘disaster’ was propagated back to the production system.

ESG Lab Validation Highlights

• The Hitachi SNM2 GUI was intuitive and made the AMS200 very easy to configure and manage.
• The AMS2000 family performed very well, showing up to a 3x performance improvement over previous generations in both engineering-style benchmarks AND real world applications.
• ESG Lab found the AMS 2000’s active-active controller technology with automatic load balancing provided consistent, stable performance while greatly reducing the efforts of array configuration and LUN mapping.
• In less than two minutes, ESG Lab was able to configure and start Hitachi TrueCopy remote mirroring between two arrays, then simulate a disaster and failover/failback.

Issues to Consider

• Thin provisioning, a feature offered on Hitachi’s higher-end storage systems, is the ability to provision volumes that only consume capacity based on the amount of data actually written to disk. Thin provisioning can be used to effectively eliminate allocated, but unused, capacity and stranded storage. ESG believes supporting thin provisioning in the AMS family would provide more high-end enterprise capabilities for end-users with midrange storage budgets.
• While Hitachi has scaled up the AMS 2000 family with impressive performance gains over the previous generation, the system supports roughly the same storage capacity as the previous generation ’ AMS 1000 family. As more and more organizations turn to modular storage systems for increased consolidation and savings, ESG recommends that HDS consider increasing the total amount of capacity supported per AMS system.
• The AMS 2000 family supports a range of connectivity options including FC, iSCSI or a mixture of both. FC connectivity is currently supported over 4Gb/sec host connections and iSCSI is supported over 1Gb Ethernet. As organizations continue to increase the number of physical and virtual servers sharing a common pool of modular storage, ESG recommends that HDS consider supporting  8 Gbps FC and 10 Gbps Ethernet host connectivity.

ESG Lab’s View

Medium-sized organizations are increasingly being asked to improve information management to more efficiently support their businesses while keeping costs down. Storage technology providers, in turn, have traditionally responded with ‘midrange’ storage systems, frequently defined as scaled-down and compromised versions of enterprise class arrays. Medium-sized businesses, however, are in need of “enterprise-class’’ information management and infrastructure technologies—without the enterprise price tag.

HDS is once again delivering innovation—the level of capability and scalability in the Hitachi AMS 2000 family is compelling and extremely valuable to end-users.

The AMS 2000 is a highly reliable, flexible, and scalable family of storage systems designed to enable medium-sized businesses and small enterprises to deliver highly available storage services to their users and customers. The AMS 2000 family provides a series of capabilities that individually provide great value; but when these capabilities are combined, the stakes are raised; offering enterprise class functionality and availability in a modular platform—provided by a leading, world-class storage vendor.

ESG Lab found the AMS 2000 to be easy to set up and manage, providing impressive performance for critical applications like Microsoft Exchange. The AMS 2000’s SAS back-end was able to sustain up to 3 times the performance of previous generation models using more expensive Fibre Channel disks. Enterprise class SAS drives, as offered in the AMS 2000 family provide the same reliability and performance as Fibre Channel drives at significantly lower cost. The SAS backplane in the AMS 2300 tested by ESG lab intermixed SAS and SATA drives in the same drive enclosures.

The AMS 2000 also demonstrated enterprise class functionality automatically and transparently; active-active controllers with automatic load balancing allowed for greatly simplified host SAN and LUN provisioning. Availability options are easy to use as well: ESG Lab used Hitachi TrueCopy remote replication technology to construct a synchronously mirrored configuration in minutes.

Hitachi aimed high with the improved scalability and feature-rich redesign of the AMS family of modular storage arrays. The AMS2000 series’ combination of enterprise class features with easy to manage midrange usability and reduced operational costs is powerful and worthy of serious consideration by any IT organization being asked—once again—to do ‘more with less’ In their data center.

Appendix

[1] Source: ESG Research Report, Medium-Size Business Server & Storage Priorities, June 2008.

[2] SNM2 requirements can be found at: http://www.hds.com/products/storage-software/system-requirements/storage-navigator-modular2-system-requirements-and-support-matrix.html

[3] The test bed configuration is listed in detail in the Appendix.

[4] Iometer is an open-source I/O measurement and characterization utility, available for download at: http://www.iometer.org/

[5] http://technet.microsoft.com/en-us/exchange/bb412164.aspx

[6] Storage configuration and layout details can be found in the Appendix.

[7] Source: ESG Research Report, The Impact of Server Virtualization on Storage, December 2007.

[8] Source: ESG Research Report, The Impact of Server Virtualization on Storage, December 2007.

Introduction

The accelerating rate of growth in the volume of unstructured file data organizations need to share, manage, and protect has become a serious challenge for IT managers—particularly those who need to provide online access to shared files for revenue generating workflows and processes. In this Lab Validation, ESG Lab explores how the HP StorageWorks 9100 Extreme Data Storage System leverages standards-based servers and storage, along with clustered file services, to create an extremely scalable, easy to manage networked storage solution designed to reduce data center footprint, administration requirements, and acquisition costs.

Background

The management of file-based or “unstructured” content (i.e., multimedia files, web pages, office productivity documents, etc.) has become one of the most pressing and persistent challenges facing today’s IT organizations. In fact, ESG research indicates that the vast majority of corporate digital assets are stored in unstructured files. Unstructured content—including digital images, audio, and video files—accounted for 77% of global digital archive capacity in 2007 and is expected to constitute the bulk of digital assets for the foreseeable future.[1] This massive amount of file data is creating demand for a new type of file storage solution that can scale bandwidth, performance, and capacity under a single system, reducing management costs as the data under management continues to grow.  The term “scale-out NAS” describes systems designed from the ground up to meet this need. While scale-out NAS has seen most of its early adoption in media and entertainment, Web 2.0, and high performance computing, out-of-control file data growth, larger file formats, and the commercial adoption of file intensive applications has brought the requirement for scale-out systems squarely into traditional data centers. [2]

The growth of file-based data, as well as the shifting nature of the files to richer formats, has left enterprise data centers bursting at the seams.  These factors are leading data center managers to consider taking a new approach to storing and managing file-based data while simultaneously pushing NAS vendors to introduce entirely new architectures. For managing growth and meeting the performance characteristics required by richer file data, scale-out NAS is gaining traction, as seen in Figure 1. Forty-nine percent of IT managers and professionals surveyed by ESG indicated that they had deployed or were planning to deploy scale-out NAS in their environments.

It is no surprise that we often hear data center managers say that they love their first NAS appliance—and curse their tenth, or worse yet, their hundredth. Traditional file storage solutions are designed to be monolithic—all storage sits behind one or two file server heads and capacity is scaled by adding storage behind those heads. When the storage limit is hit or additional performance is required, a new monolithic system is installed.  There is no way to share the workload between the systems and migrating directories or files between systems means remapping and remounting for each and every client requiring access. There is no economical way to scale performance without significant additional cost and complexity.

HP StorageWorks 9100 Extreme Data Storage

The HP StorageWorks 9100 Extreme Data Storage System (ExDS9100) is a scale-out NAS solution aimed at addressing concerns about massive file data growth and management challenges with a capacity-optimized implementation of the PolyServe Clustered file system. This solution is deployed on a massively scalable platform built from HP BladeSystem c-Class blade servers and multiple large SAS arrays.

Figure 2. The HP ExDS9100

ExDS9100 systems are built upon storage elements called capacity blocks and server elements called performance blocks. A single capacity block is composed of a dual redundant SAS controller and two capacity drawers containing a total of eighty-two 1 TB midline SAS drives with RAID-6 data protection. Capacity blocks are divided into eight 10 disk RAID groups with two hot spares for a total usable capacity of 64 TB per capacity block. Each performance block consists of an HP BL460c blade server with a SAS mezzanine card running ExDS9100 file serving and management software on a Linux operating system. A single ExDS9100 can house up to ten capacity blocks for 820 TB of raw storage in just two floor tiles. Up to 16 performance blocks use that storage capacity to present clustered file systems via standard NFS and HTTP protocols. CIFS support is available as an optional protocol enabled by HP.

Since the ExDS9100 runs a Linux-based operating system, many applications—such as those for streaming media—can be run directly on the performance block. HP calls this unique feature ‘application co-hosting.’ Not all applications are able to take advantage of this powerful feature, but if careful attention is paid to specific application requirements, users can unlock huge potential for decreased complexity, increased performance, and reduced cost. By layering the application tier directly on the storage system, administrators can potentially eliminate server tiers from their application architecture. HP recommends that users discuss their application requirements with their account team to understand the technical requirements.

Additional features and benefits of the ExDS9100 solution include:

• Extreme Scalability: The ExDS9100 scales to 820 TB in just 13 square feet of floor space while driving up to 3.2 GB/sec of performance using independent, very dense capacity blocks and high performance blade server-based performance blocks. Performance and capacity scale independently, online, and seamlessly.
• Unified Management: The ExDS9100’s simple management interface provides a single point for configuration, control, and monitoring of the entire system, enabling a single administrator to manage petabytes of storage.
• Extreme Availability: Rolling upgrades to servers and storage mean the system will not have to be taken down for maintenance. RAID-6 provides an extra layer of data protection.
• Extreme Affordability: The ExDS9100 comes fully integrated and configured. All hardware and software required to get up and running quickly is included; no additional software or services are required. Online maintenance and upgrades lower operational costs, eliminating offline windows. The ExDS9100 also provides the ability to host applications directly on the system, simplifying an organization’s overall application architecture.

This ESG Lab Validation report examines the ease of implementation and management of the HP ExDS9100 in addition to its performance and capacity scalability as well as the acquisition and operational cost benefits of the platform.

ESG Lab Validation

ESG Lab performed hands-on evaluation and testing of the HP ExDS9100 at HP’s Executive Briefing Center in Cupertino, CA. Testing began with a physically installed and powered up HP ExDS9100. ESG lab walked through the initial configuration process a typical administrator would experience after an ExDS9100 had been delivered and installed by HP professional services.

Simplicity and Manageability

Figure 3 shows a high level view of the test bed used by ESG Lab. An ExDS9100 configuration containing 246 TB of raw capacity and a client machine running Linux were attached to a Gigabit Ethernet LAN.[3]

While the underlying components of the system each have their own management and monitoring interfaces, the ExDS9100 is configured and managed with a simple command line interface. Many configuration activities are carried out with just a single command. To further simplify configuration and management, the ExDS9100 offers a single capacity-optimized storage layout that enables HP to pre-format the back-end storage at the factory and offload storage management tasks from ExDS administrators.

ESG Lab used the HP CFS (Clustered File System) GUI and the HP Onboard Administrator for the purpose of illustration in this report, but all configuration and management activity was executed from the ExDS Manager Command Line Interface.

ESG Lab Testing

The ExDS was powered on and configured with IP addresses by HP professional services, just as it would be for a new customer. ESG lab used the PuTTY ssh client to establish a secure connection to ExDS1, a blade server in the system. Once connected, the ExDS manager CLI was launched with the ‘exdsmgr’ command. Any blade server may be used to access the ExDS manager console. As shown in Figure 4, ESG lab first created an 8 TB file system using the ‘create filesystem’ command in the ExDS Manager.

The ‘array’ command line parameter was used to define which capacity block should be used to create the file system and the ‘size’ parameter was used to specify the capacity of the new file system. In this example, an 8 TB file system on array three was created. The command completed in just over one minute.

Next, the filesystem was exported via NFS and HTTP protocols using the ‘create vnfs’ and ‘create vhttpd’ commands, shown in Figure 5. These commands are very similar; the only mandatory parameters are the virtual IP addresses of the share and the list of servers that will export this file system.

The commands executed immediately and with no delay, the newly created file system was exported by all servers in the ExDS over NFS and HTTP. Finally, ESG Lab mounted the file system over NFS and pointed a web browser at the virtual IP created in the previous step.

With just three commands and in less than three minutes, ESG Lab was able to configure, export, and mount an 8 TB file system from the ExDS9100.

Scaling Capacity and Performance

The ExDS is a file services platform optimized for capacity, with a pre-defined configuration and layout that offers high system throughput and near linear scaling. Costs are kept low by utilizing 1 TB midline SAS drives, which are SATA disk platters with SAS electronics and connectivity. Midline SAS drives offer the best of both worlds for a capacity optimized system: the high capacity and lower cost of SATA combined with the connectivity, availability, and functionality of SAS.

An HP ExDS9100 file system can scale capacity and performance independently as shown in Figure 7. Performance blocks or capacity blocks can be added hot and online to meet changing business requirements. Blocks are installed in the system with a single command, eliminating the management of disk drives and RAID groups.

ESG Lab Testing

ESG Lab tested the independent online scalability of the ExDS9100 by installing a new performance block and auditing the procedure for installing a new capacity block in a running system. The process could not have been simpler. After inserting the new performance block (a BL460c blade server) into an empty slot in the integrated HP c7000 chassis of the ExDS, ESG Lab confirmed that it was receiving power and issued the ‘install server’ command, as seen in Figure 8.

Figure 8. Installing a Performance Block

This kicked off a completely automated process that installed and configured both the operating system and HP CFS software, then joined the new performance block to the cluster with no further user interaction required. The entire process took about 15 minutes. When exdsmgr indicated that the installation was complete, ESG Lab verified that the server was a live member of the cluster using the HP CFS GUI, seen in Figure 9, and by mounting a file system from the newly installed server, ExDS4.

Next, ESG Lab examined the procedure for upgrading the system’s capacity. Storage upgrades are performed by adding additional capacity blocks to an existing system while it is online. Capacity blocks are pre-provisioned at the factory with a standard LUN layout; initialization on all LUNs is also completed at the factory. All an administrator needs to do is create file systems as required. One or more storage blocks can be added at a time. New capacity blocks are shipped with a five page glossy install guide, shown in Figure 10, which clearly lays out the procedure used to install and configure a new capacity block.

Once the new block is racked, cabled, and powered up, a single command discovers the storage in the capacity block and adds it to the pool of available storage in the cluster. At this point, the storage is available for the administrator to create and export file systems.

ESG Lab next examined the performance scalability of the ExDS9100 in a content delivery environment using the IOzone file system benchmark tool.[5] The term ‘content delivery’ is used to define a broad category of systems which deliver digital files, video, audio, and images over a delivery medium such as a broadcasting network, Intranet, or the Internet. The delivery of large files over a content delivery medium requires a storage solution with lots of capacity and high aggregate read throughput.

IOzone was used to simulate the delivery of content using a number of threads performing 1 MB sequential reads. Multiple threads were directed at multiple file systems on each of four performance blocks spread over two capacity blocks in an ExDS9100.^[6] These results were compared to scale out tests run by HP using the same test parameters in a fully loaded system with sixteen performance blocks and ten capacity blocks. The results, illustrated in Figure 11, show the ExDS9100’s maximum throughput scaling linearly as performance and capacity blocks are added.

The maximum throughput recorded (3,430 MB/sec) was used to calculate the number of streams that could be delivered for a number of well known content types.^[8] For example, a bit stream rate of 128 Kbps for MP3 files was used to determine that the ExDS9100 has the bandwidth required to support up to 219,520 song downloads (3,430 MB/sec divided by 128 Kbps) as shown in Figure 12.

ESG Lab also tested and audited network file system throughput to validate the ExDS9100’s performance in a scale-out NAS environment.

As can be seen in Figure 13, the ExDS9100 was able to sustain impressive throughput that scaled linearly as performance and capacity blocks were added.  Results were very good for transfer block sizes from 32 k down to 4 k, which speaks well to the ExDS9100’s operational efficiency. Many NAS systems generally perform significantly better with larger block sizes, which place lighter demands on a storage system’s resources.

What the Numbers Mean

• A single ExDS9100 has the raw bandwidth required to service 93,662 concurrent Internet video (MPEG) downloads.
• The raw throughput results are based on a content delivery system that accesses content stored on disk using a large block size of 1 MB. With many clustered file system solutions, a smaller block size (e.g., 32 KB) might serve fewer objects. ESG Lab validated that the ExDS9100 delivers nearly identical throughput for a wide range of block sizes.
• The ExDS9100 showed excellent NFS throughput, driving 2.6 GB/Sec—more than 80% of the raw throughput available from the SAS back-end.
• Running small block IO (4 KB) over NFS was equally impressive, sustaining 1.9 GB/Sec or nearly half a million IOs/Sec

Availability

High availability (HA) and fault tolerance are major concerns for today’s IT administrators. They have come to expect the same enterprise-class reliability in a scale-out Tier 2 solution that they have with their primary storage systems. The ExDS9100 is architected as an integrated appliance, meaning that disks and RAID protection are managed automatically. This allows tight control and better response to hardware failures with less administrator interaction.

ESG Lab Testing

ESG Lab simulated disk, SAS switch, and server failures while the system was running and actively servicing clients. First, ESG Lab generated writes with the Linux client by running the linux dd command to copy 500 MB of data to a file on the target file system in a continuous loop. A disk drive was pulled from the capacity block housing the file system while data was being written. An administrator’s e-mail account being monitored by ESG Lab received a message stating that a drive had failed and indicating the specific array, box, bay, and slot the drive was in. In a live customer system, an e-mail would also be sent to HP customer service.

As seen in Figure 14, the ExDS manager ‘show lun’ command displayed a notification that LUN 20 in Array 3 was undergoing a RAID rebuild. While the rebuild was taking place, ESG lab pulled one of the SAS switches from the HP c7000 chassis. These SAS switches provide connectivity between the performance blocks and the capacity blocks.

Again, the administrator’s mailbox almost immediately received a message indicating that a failure had occurred. In this case, interconnect bay 3 had disappeared. ESG Lab verified both errors using the ‘show alert history’ command, which gave detailed data on each failure.

Finally, ESG Lab tested file system availability by shutting down a performance block while a client was mounted to a file system and writing data to it. The net effect of this procedure was exactly the same as a server failure or crash. It is also similar to what takes place when rolling upgrades, which require reboots of the performance blocks, are performed. ESG Lab again generated writes with the Linux client by running the linux dd command to copy 500 MB of data to a file on the target file system in a continuous loop. Next, ESG Lab issued the shutdown server command and observed the transition of performance block ExDS3’s virtual IP (and all file systems exported on it) to server four, as seen in Figure 15.

As the virtual IP transitioned from ExDS3 to ExDS4, write activity paused but the write operation did not fail. The file system remained mounted on the client and the dd operation continued without error after the transition completed in a few seconds. The ExDS9100’s clustered file system’s global file handles ensure that file system access can transition between performance blocks with no disruption and no re-mount required.

ESG Lab Validation Highlights

• Within three minutes of sitting down in front of the console of a factory default ExDS9100, ESG Lab had completed configuration and the ExDS was providing file services over NFS and HTTP.
• With just one command, ESG Lab installed a new performance block. The new server was automatically installed, configured, and joined to the cluster with no interaction required.
• The ExDS was able to sustain 3.2 GB/sec of raw performance and 2.6 GB/sec of network file performance—enough to serve hundreds of thousands of individual streams of internet content.
• The ExDS continued to serve file systems through hard drive, back-end switch, and server failures seamlessly and without interruption.
• The ExDS clustered file system allows file systems to move between servers hot and online without disrupting client access.

Issues to Consider

• The ExDS provides access to CIFS clients through the use of Samba. CIFS access does not come pre-configured like NFS and HTTP, but must be added to the ExDS solution by HP. The Samba solution for CIFS provides high availability for CIFS connections where a single blade manages specific file systems. The exponential growth of unstructured data will soon make systems like the ExDS attractive for large scale CIFS consolidation and a pre-configured CIFS option will make sense.
• While the ExDS CLI is exceptionally easy to use, users have access to the disparate GUIs that are part and parcel of the various components. An ExDS-specific consolidated GUI that brings together all the functionality of the CLI while enabling management of multiple systems from one pane of glass would be a welcome addition.

ESG Lab’s View

Market drivers for early adopters of scale-out NAS include faster provisioning, improved scalability and performance, easier management, and the need to support specific, fast-growing applications. The management efficiencies of scale-out NAS are so compelling that, in a recent ESG survey of 504 North American and European IT professionals, lower cost of infrastructure was literally last on the list of buying criteria for early scale-out adopters.[11] Planned and potential users have vaulted lower cost into the top tier of purchasing criteria, but cost remains second to the improved scalability that scale-out NAS brings to the table.  Even during the current economic crisis, the manageability of scale-out NAS continues to be a major reason users are considering deploying it and is on par with cost as a driver for future adoption.

The ExDS9100 is a highly scalable, affordable, and easy to manage file storage system designed to enable enterprises to store hundreds of terabytes of file-based data in a highly available repository providing storage services to users and customers.  The ExDS9100 delivers excellent performance and availability and its ability to independently scale both capacity and performance provides users and applications with 3.2 GB/sec of raw performance and up to 820 TB of storage in a single system.

ESG Lab found the ExDS9100 to be easy to set up and manage, providing impressive performance in a package that took just three commands to configure.  ESG Lab was able to scale capacity and performance independently with simple commands. A fully configured ExDS9100 was confirmed to sustain up to 3.2 GB/sec of raw disk performance, while providing the ability to host applications on the server blades in the enclosure.

The ExDS9100 also demonstrated enterprise class availability automatically and transparently. File systems transitioned between servers with no disruption to access, eliminating both planned and unplanned outages. Disk and component failures were handled transparently as well, alerting the administrator that an issue had occurred while automatically taking non-disruptive, corrective action.

In the current environment of economic uncertainty, the advanced features that the HP ExDS9100 brings to the table, such as scalability and ease of management, are more important than ever. With the HP ExDS9100, users can scale capacity without scaling headcount. Provisioning is quick and easy.  The system has an “always on” architecture that can seamlessly withstand multiple failure modes. HP has done a good job leveraging IP from multiple business units to create a seamless, appliance-like experience.  Unlike some systems that require multiple interfaces to manage the storage, blades, and file system, HP has an all-in-one management console.  HP seems to have met its design goals with this system; the HP ExDS9100 is a massively scalable, highly affordable file storage appliance that is shockingly easy to deploy, manage, and upgrade.

Appendix

[1] Source: ESG Research Report, Digital Archiving Survey, November 2007.

[2] Source: ESG Research Report, 2008 Enterprise Storage Survey, November 2008.

[3] Configuration details are listed in the Appendix.

[4] Source: ESG Research Report, 2008 Enterprise Storage Survey, November 2008.

[5] http://www.iozone.org

[6] Test configuration details can be found in the Appendix.

[7] Source: ESG Research Report, 2008 Enterprise Storage Survey, November 2008.

[8] The bit rates for each content type are listed in the Appendix.

[9] Source: ESG Research Report, Enterprise Storage Survey, November 2008.

[10] Source: ESG Research Report, 2008 Enterprise Storage Survey, November 2008.

[11] Source: ESG Research Report, 2008 Enterprise Storage Systems Survey, November 2008.

Introduction

In 2007, ESG Lab tested NetApp VTL appliances and found enterprise-class performance and scalability, combined with easy implementation and integration. This ESG Lab Validation report presents the results of a new round of hands-on testing of the NetApp VTL appliances with a focus on recently added data deduplication functionality. Enhanced enterprise-class performance and architectural scalability as well as the ease of deployment, painless integration with physical tape and simplified management are also explored.

Background

ESG has observed that data deduplication technology for disk-based backup systems and virtual tape libraries has been gaining in popularity and is being integrated into increasing numbers of backup solutions. ESG research indicates that no single concern dominates when users are considering data deduplication technology.[1] As shown in Figure 1, cost, compatibility, ease of integration and management, performance and scalability all have significant mindshare with enterprise users selecting deduplication technology. In order to succeed at the enterprise level, data deduplication appliances must provide cost-effective scalability, performance and flexibility.

NetApp VTL is a disk based backup appliance that supports virtual tape library emulation and data deduplication. NetApp uses a modular, controller-based architecture to provide scalability and enterprise-class levels of performance and availability. NetApp VTL scales non-disruptively from an entry-level system with 10 TB of usable disk capacity prior to compression and deduplication up to a multiple controller system supporting 48 disk shelves with over half a petabyte of usable disk capacity with 1 TB disks. This could equate to 10PB of virtual capacity assuming a 20:1 deduplication ratio. The NetApp VTL solution is managed as an integrated appliance, but it can emulate hundreds of virtual tape libraries with thousands of virtual tape drives and tens of thousands of virtual tapes. Disks and RAID protection are all managed transparently by the VTL controller. This approach provides the modularity needed to address a wide range of capacity and performance requirements while keeping management simple and straightforward. Because the VTL owns the disks, no back-end array configuration or management is necessary. Additionally, NetApp VTL is optimized for large block sequential I/O, which characterizes almost all backup and restore activity.

With the release of NetApp VTL 6.0 in 2008, data deduplication was introduced into the NetApp VTL family. The NetApp VTL allows administrators to enable deduplication on an individual virtual library basis, enabling control over which data sets to deduplicate. This is important as users may want to disable deduplication for data that either does not benefit from deduplication or that will not be retained for any significant period of time. Pre-compressed image files or encrypted data sets do not benefit greatly from deduplication—pre-processing data normally makes it difficult to find repeated patterns and severely reduces deduplication effectiveness.

Figure 3 illustrates how this would work in a real environment.  Data sets that can benefit from deduplication such as databases, e-mail and file systems with extended retention periods are backed up to a VTL with deduplication enabled. The NetApp VTL then de-duplicates the data as a post-backup process. Deduplication can be scheduled or performed on-demand. Data sets which would benefit little from deduplication, such as pre-compressed image files, encrypted data sets and database recovery log backups can be sent to a VTL with deduplication disabled, but with hardware compression still enabled. This reduces system overhead and frees up CPU and memory resources that can be used for deduplication of the data sets that can benefit from it.

ESG research indicates that 87% of organizations using VTL continue to use tape as an integral part of their backup policies.^[2] NetApp is certified with all major backup applications and has partnered with several vendors (Symantec, CommVault, BakBone and SyncSort) to more deeply integrate NetApp VTL disk-to-disk-to-tape (D2D2T) technology with their respective backup applications. NetApp was first to market with this level of backup application integration. To date, Symantec (Veritas NetBackup), CommVault (Galaxy) and BakBone have already delivered new releases that take advantage of this capability, and other vendors are in development. NetApp has also written a script-based D2D2T integration for use with EMC Legato NetWorker. It’s important to note that even without this integration NetApp VTL can still be used with all supported backup applications.

NetApp positions the NetApp VTL family as an enterprise-class VTL solution that is easy to integrate into an existing tape-based environment, simple to manage and scalable in both performance and capacity with native data deduplication. Installation of a new system, backups and restores with leading backup applications, data reduction with deduplication, integration with physical tape, performance, scalability and availability are all explored in this report.

ESG Lab Validation

ESG Lab performed hands-on evaluation and testing of the NetApp VTL at the NetApp Sunnyvale, California headquarters. Testing was designed to examine ease of implementation and management, test performance and scalability with and without deduplication and validate integration with existing physical tape environments. As seen in Figure 4, the test bed consisted of Veritas NetBackup and EMC Legato NetWorker Windows-based backup servers, along with two Linux-based load-generation servers running a script that utilized data created by the hpcreatedata utility from Hewlett Packard. A NetApp VTL 700 with two fully populated disk shelves was used for the initial installation tests and as a base platform for the online upgrade tests. A Quantum Scalar i500 tape library was utilized for the physical tape integration testing.

Ease of Implementation and Management

NetApp products have become famous for their streamlined, simple installation and management experience. This phase of ESG Lab testing was performed to evaluate how the NetApp VTL solution‘s ease of installation stacks up.

ESG Lab Testing

ESG Lab began with a NetApp VTL700 appliance that had been racked, prewired, powered up and connected to the administrative LAN prior to our arrival on site. The first step required was to configure the system with an IP address so the administrator could access the HTML-based management GUI. A full-featured CLI is also available to allow scripted activities. The IP address was set directly from the front panel in one step. Once a valid IP address was configured, the administration GUI was launched from a browser. The opening screen is shown in Figure 5.

The Administration GUI is laid out logically, with main tasks listed vertically on the left side of the screen and subtasks arrayed horizontally across the top of the active pane. Upon logging in for the first time, the administrator is introduced to the Actions Required list, outlined in blue in the center of the screenshot. Note that the system provides not only a list of actions needed to resolve known issues; it also provides a resolve link next to each one. This takes the administrator directly to the appropriate screen to address the listed issue. At the top of the screen, the status indicator light in the upper right is glowing yellow, indicating a warning. The status indicator is tied to the first digit of the code attached to each action: 1 = Good (green status), 2 = Warning (yellow status) and 3 = Error (red status).

Next, ESG Lab configured AutoSupport. This feature enables the VTL to alert NetApp technical support of fault conditions. AutoSupport is the mechanism used by all NetApp systems to “phone home” and provides NetApp with a powerful tool that can proactively monitor and track systems remotely. Administrators can choose HTTP or e-mail notification. Using the e-mail option, the local administrator can be copied on all messages sent to NetApp.

Before creating any virtual libraries, virtual tape drives or virtual tapes, the administrator must create RAID groups on the raw disks by using the RAID Groups Configuration interface, as shown in Figure 6. The view presented shows a physical representation of all disk shelves that is brightly color coded to indicate the status of each disk.

ESG Lab selected ‘Automatically Create RAID Group’ to let the NetApp VTL lay out the disks. NetApp VTL RAID management algorithms choose the best layout for performance and availability based on the number of disks and shelves installed. Because the NetApp VTL controller owns the disks directly and acts as the RAID controller, initializing RAID groups is nearly instantaneous. The RAID groups are immediately available for configuration and writing, without the need for any background zeroing or initialization.

Another important distinction is seen when a previously used RAID group is destroyed and reinitialized. This also occurs nearly instantaneously, with the RAID group immediately available for writing. This capability is critical in systems with high-capacity hard drives. Because NetApp VTL RAID creation occurs so quickly, users can immediately back up their data. A non-NetApp VTL system with 1 TB SATA drives based on traditional RAID 5 controllers could easily take 20 hours to complete this basic operation.

With the RAID group creation completed, the next step was to create a virtual library with virtual tape drives and tapes. Those steps are all accomplished on one screen, illustrated in Figure 7. We named our test library test1 and selected a ‘Library Type’ of NetApp. This alerts backup applications that the library is a VTL from NetApp and in the case of Veritas NetBackup (and many other backup applications) allows NetBackup to set timings appropriately so that the backup application does not wait for tapes to mount before starting backups and restores.

ESG Lab elected to create four virtual tape drives and forty slots, fully loaded with virtual tapes. A Fibre Channel port for SAN presentation was selected from a pull-down list. An arbitrary tape labeling scheme was chosen for our testing. An administrator would normally choose a scheme consistent with existing tape labeling standards or allow the NetApp VTL to automatically create virtual tapes that match physical tapes in a library connected to the VTL. New in the NetApp VTL is the ‘Disk Compression Type’ option, used to enable deduplication for a VTL when it is created.

The final step was to discover the new VTL in NetBackup, which was performed in exactly the same manner as discovering a physical tape library and was completed in less than a minute. In less than 10 minutes, the entire system was configured from start to finish and the first backup was kicked off.

Finally, ESG Lab connected two additional shelves to the existing two shelves to validate the ease of capacity expansion. Upon powering up, the shelves were recognized as new, un-configured storage by the NetApp VTL. The only step necessary was to go back to the ‘Create RAID Group’ screen and select ‘Automatically Create RAID Groups,’ as was done in the initial installation. The new RAID groups were immediately available for use, as shown in Figure 8.

Storage Efficiency

NetApp VTL deduplication reduces the capacity required to store backed up data. Data that has been backed up to a VTL with deduplication enabled is scanned and the NetApp VTL examines the contents of the backup images, recognizing and replacing redundant blocks with pointers to the original blocks.

ESG Lab Testing

To evaluate deduplication in the NetApp VTL, ESG Lab ran multiple full backups of an Oracle database and audited the results. ESG Lab also audited NetApp’s in-house results that showed similar deduplication ratios for file system and Exchange data sets. The database was harvested from a corporate data center—full backups performed over ten days were evaluated. The Virtual Tape List screen of the NetApp VTL console, shown in Figure 9, was used to monitor the benefits of data reduction after each NetBackup job had run.

The ‘Data Written’ and ‘Disk Used’ fields were collected and analyzed after the tenth full backup was completed. The data collected is detailed in Table 1.

ESG Lab calculated the data reduction over the course of ten backups and plotted the curve of storage savings, as seen in Figure 10. The NetApp VTL achieved 90% data reduction for a production Oracle database after just four full backups and 95% data reduction after ten backups. Said another way, after just ten full backups, a deduplication ratio of 22:1 was achieved.

What the Numbers Mean

• The total amount of data that the NetApp VTL would have stored on disk over the course of ten days without using deduplication was calculated to be 28.87 TB.
• NetApp VTL deduplication reduced the required disk capacity for ten full Oracle DB backups from 28.87 TB to only 1.29 TB. This would enable organizations to keep backups on disk for MUCH longer, reducing the need to go back to tape for recoveries.
• The amount of capacity reduction that can be achieved with the NetApp VTL will vary according to the backup policy in effect, the number of backups retained on disk and the type of data being stored. In this scenario, capacity was reduced by 95% over just ten days.

Scalability and Performance

NetApp uses a variety of methods to maximize performance and scalability. The latest server and I/O interconnect technology (4Gbps FC) provides the basic building blocks. A VTL controller with fully integrated RAID and I/O functionality is optimized for large-block, sequential I/O. NetApp VTL optimally distributes deduplicated data across all available disks to maximize VTL performance, Selective data deduplication enables administrators to only apply deduplication, a resource-intensive operation, to data sets that will benefit from it.

ESG Lab Testing

ESG Lab simulated multiple backup streams to measure the backup performance capabilities of the NetApp VTL700 with standard hardware compression and data deduplication enabled. Two Linux servers with two Fibre Channel connections were connected to a NetApp VTL700, as shown previously in Figure 4. A PERL script was used in conjunction with the hpcreatedata utility during this phase of testing to avoid file system overhead in the system being backed up, maximize the stress on the VTL700 and minimize the amount of equipment required to obtain valid results. The hpcreatedata utility creates files on disk with variable levels of compressibility. The script was reviewed by ESG Lab to ensure that it issued the same low-level SCSI commands used by common backup applications.[5] The performance results were independently verified against the statistics reported by the Fibre Channel switch providing connectivity between the clients and the VTL700.

Figure 11 shows results obtained for simulated backups using four, ten and forty simultaneous streams of 2:1 compressible data with hardware compression enabled and deduplication disabled. It is important to note that performance scales linearly as streams are added and that the results achieved with forty simultaneous streams exceed the performance numbers that NetApp posts for the VTL700 on its public website. Also of interest is the fact that these numbers were produced by a factory default configuration with only four shelves of disks that had not been tuned for performance.

Figure 12 compares the results of testing with data deduplication in three different configurations. Forty backup streams to forty virtual drives in different Virtual Libraries in a VTL700 were used in each test. Deduplication was enabled for all virtual libraries in the VTL in the first pass, half of the virtual libraries in the second pass (twenty deduplication enabled streams), and zero in the third pass. The NetApp VTL performed very well here also; The Mixed Mode configuration (twenty streams to deduplication enabled VTL’s, twenty streams with hardware compression only) providing more than 80% of the performance of a system with deduplication completely disabled. Again, it is important to note that these numbers were produced by an un-tuned, factory default configuration with a small number of disk shelves.

What the Numbers Mean

• The NetApp VTL provided maximum performance straight out of the box, with no tuning and a relatively small number of disks.
• Deduplication in a mixed mode environment, (which will be used in virtually all real-world backup environments) ran nearly as fast as backups with deduplication disabled. This increases capacity utilization and improves ROI by requiring less physical disk with a low performance impact.
• Backing up data in a mixed mode environment provided sustained throughput of 1,004 MB/sec. This enables users to back up more than 27 TB in a single 8-hour backup window using a single controller.
• ESG Lab ran a restore test and observed aggregate restore throughput sustained at 562 MB/sec.

Integration with Physical Tape

In most backup environments, physical tape is still used for long-term and off-site archiving. VTL users need to export virtual tapes to physical tapes for archiving and import them into a Virtual Tape Library for restores of archived data. To make this work, the virtual library must be configured to emulate drives that are of the same type and capacity as the physical drives in the physical library. The NetApp VTL appliance emulates physical tape libraries and stores large amounts of backup data on disk, automatically copying data directly to and from physical tape.

NetApp VTL uses two features to enhance integration with backup applications: Shadow Tape copies and Direct Tape Creation. Shadow Tape technology automatically caches backups onto disk, enabling fast restores from disk for backups that have already been exported to physical tape. Shadow Tape backups enable storage administrators to meet two often conflicting requirements in a way that is fully compatible with their existing backup software: first, to meet service-level agreements for creating physical copies that must be moved offsite for safe storage within a specified time frame and second, to meet user demands by quickly restoring backup data without having to recall a tape from off-site storage. Direct Tape Creation is used to make copies of virtual tapes without involving the media server in data movement.   The net result is the ability to have both a disk-resident and an off-site backup copy both under control of the backup application.

The traditional method for exporting a virtual tape to a physical tape has been to use the backup application and make the copy by using the media server, illustrated by the gray arrow in Figure 13. This consumes significant server CPU, memory and I/O resources that could otherwise be utilized in performing backups or restores. Direct Tape Creation, represented by the blue arrows, utilizes the VTL to perform the actual tape copy. While some VTLs offer this type of functionality, NetApp’s integration with backup applications sets it apart. NetApp VTL enables backup software applications to keep track of physical tape cartridges created directly by the Virtual Tape Library

The NetApp VTL retains the native format of the backup application when it writes data to disk. This means that when the NetApp VTL copies data to physical tape, the backup application can restore data from these tapes directly. ESG Lab’s goal for this portion of testing was to validate NetApp integration with backup applications and the ability of those applications to keep track of physical and virtual tapes.

ESG Lab Testing

ESG Lab first looked at tape creation procedures. Both virtual tapes and physical tapes created by NetApp VTL are written in the native format of the backup application. This allows the NetApp VTL to import and export tapes without conversion. It also means that the tapes can be restored directly by the backup application without the VTL. In order for this to work, it is crucial that the amount of data on a virtual tape fit on the physical tape without exceeding its capacity, maintaining a one-to-one correlation between a virtual tape and a physical tape. If the data stored on a virtual tape exceeds the capacity of a physical tape, it cannot be exported or cloned.

The NetApp VTL appliance addresses this issue by using a feature called Tape Smart Sizing to control the amount of data written to each virtual tape. Tape Smart Sizing is based on the premise that all data is compressible at differing ratios and that different tape drive types will compress the same data differently. Smart Sizing is designed to allow the VTL to determine the correct size of a virtual tape to ensure that it will fit on a physical tape correctly. Figure 14 shows an example of Smart Sizing in operation. Virtual tape NET-105 was created with a raw capacity of 100 GB and a synthetic data set composed of data with a known compressibility of approximately 2:1 was backed up.

With Tape Smart Sizing, the data stream is monitored to determine the compressibility of the data based on the compression algorithms used by the tape hardware. This allows the NetApp VTL appliance to determine the optimal size for the virtual tape so that it fits correctly on its corresponding physical tape. In the preceding example, 212.84 GB was backed up and stored in one 100GB virtual tape. ESG Lab observed that the virtual tape was exported successfully to a 100 GB physical tape (IBM LTO-1) with no issues.

ESG Lab next tested NetApp VTL Shadow Tape functionality. When enabled, whenever a virtual tape is exported to a physical tape directly from the VTL, the virtual tape is moved to the VTL Shadow Tape Pool. A virtual tape in the Shadow Tape Pool is invisible to the backup application and is not listed as part of a virtual library but is still available for quick access if needed for restore.

Testing began with a backup to a number of virtual tape drives using Veritas NetBackup on a Windows server. Once the virtual tapes were filled, an Eject command was executed from NetBackup for tape volumes EXP400 and EXP401. Figure 15 shows tape volume EXP400 ready to be exported.

Once the Eject command was completed, ESG Lab used the NetApp VTL GUI interface to examine the tape now in the Shadow Tape Pool. It was noted that the operation to move the tape into the Shadow Tape Pool was, in fact, automatic and occurred nearly instantaneously. Figure 16 shows the Shadow Tape Pool with tape volume EXP400, which had been exported using NetBackup. It is also important to note that this 100GB Shadow Tape was deduplicated and consumed only 1.04 MB on disk.

At this point, ESG Lab confirmed that once a virtual tape has been exported to a physical tape, the copy residing in the Shadow Tape Pool can be immediately and automatically available for a Restore operation, even though it is not visible to the backup application. Testing began by performing a Restore operation from NetBackup. Once the Restore command was issued, NetBackup indicated, as seen in Figure 17, that it had a Pending Request for the tape to be mounted.

Within seconds of the command execution, the virtual tape residing in the Shadow Tape Pool was automatically moved back into the Virtual Tape Library being used for this test and NetBackup began transferring the data to the backup host. Once the restore was completed, the tape was moved back into the Shadow Tape Pool per the user-defined shadow pool policy on the NetApp VTL. The entire process was completed without manual intervention.

Reliability and High Availability

High availability (HA) and fault tolerance are major concerns for today’s backup administrators, who have come to expect the same enterprise-class reliability in a VTL solution that they have with their primary storage systems. The NetApp VTL is architected as an integrated appliance, meaning that disks and RAID protection are managed directly by the NetApp VTL software. This allows tighter control and better response to hardware failures. Additionally, the VTL configuration is stored internally within the appliance on disk, facilitating simple controller replacement and upgrades. VTL software can be updated online, with only a short reboot required.

ESG Lab Testing

ESG Lab performed a code upgrade, then simulated SAN and controller failures during this phase of testing. A NetApp VTL System running Software version 5.6.0 was used for this series of tests, as seen in Figure 18. After clicking Support Tools on the right side of the screen, ESG Lab clicked Software Image Upgrade to kick off the upgrade.

As shown in Figure 19, the NetApp VTL displays progress as the code is downloaded, verified and installed. Once installation is completed, the administrator is prompted to click Continue, which confirms a system reboot is required to load the new firmware.

The entire code upgrade, including the reboot, took just four minutes for a system with two disk shelves. NetApp advises that upgrades to systems with more disk shelves would take slightly longer.

ESG Lab next injected a fault by unplugging one redundant FC (Fibre Channel) connection to simulate a problem in the SAN while data was being backed up by NetBackup. For this test, NetBackup was configured in failover mode with two virtual tape drives in a Storage Unit Group. This configuration lets the backup application continue to operate when SAN connectivity to the VTL is compromised. ESG Lab confirmed that the backup application was able to continue to write data to the Virtual Tape Library via the surviving path, with no intervention required.   As shown in Figure 20, when the Fibre Channel cable to TID-0 was pulled, the server could no longer see the tape drive. Visibility to TID-1 was unaffected and NetBackup simply started writing the backup to TID-1.

Finally, ESG Lab simulated a VTL head replacement by resetting the VTL server head to factory defaults, thereby erasing all configuration data. The net effect of this procedure was exactly the same as removing a failed controller (or a controller scheduled for upgrade) and replacing it with a new controller received from the factory. The NetApp VTL provides the ability to save a configuration file, which can be loaded at this point to recreate all tape libraries. If a user does not have this file, the libraries can be recreated manually, because the VTL uses self-describing data on disk. The final steps are to import the RAID groups, assign tapes to libraries, assign libraries to Fibre Channel ports and adjust the SAN zoning. ESG Lab was able to restore all previously configured tape libraries and virtual tape drive and media configurations in less than 10 minutes. ESG Lab estimates that even with SAN reconfiguration (which was not performed here), the entire process could have been accomplished in 15 minutes.

ESG Lab Validation Highlights

• ESG Lab confirmed that a NetApp VTL can be deployed in an existing tape environment without changes to policies or procedures and users can be performing backups in less than 10 minutes.
• ESG Lab has validated that data deduplication in the NetApp VTL can be used to reduce disk capacity significantly while applying deduplication policies appropriate to data type and retention needs.
• ESG Lab observed block level deduplication in the NetApp VTL reduce the required disk capacity for ten full Oracle DB backups from 28.87 TB to only 1.29 TB, a data reduction factor of 95% or 22:1.  ESG lab audited similar backup tests with file system and Exchange data and saw similar reduction ratios.
• NetApp VTL delivered performance that met or exceeded the claims made on the NetApp website using an “out-of-the-box” configuration and only one-sixth of its potential total disk spindles. Backing up a mixed environment including deduplicated and non-deduplicated data sets drove a sustained throughput of 1,004 MB/sec, enabling organizations to protect more than 3 TB per hour, or 27 TB in a single 8-hour backup window, using the configuration tested by ESG.
• Tape Smart Sizing eliminated a huge management headache by automatically ensuring that each virtual tape created will fit on a single, matching physical tape, regardless of the compressibility or deduplication of data.
• ESG Lab was impressed with NetApp Shadow Tape functionality, which enables physical tapes to be sent off-site while maintaining a disk copy for fast restores, as well as the integration achieved with both Veritas NetBackup via API and EMC Networker via PERL scripts.
• New disk shelves were added nearly instantaneously and immediately put into service.

Issues to Consider

• The Management Console is crisp and intuitive, but in ESG’s opinion could use two adjustments. Users would benefit from the ability to view multiple VTL appliances from a single browser window and select the VTL they would like to manage. In addition, the “daily tasks” and “configuration tasks” menus should be collapsed into one master menu for simplicity.
• Although detailed statistics are collected and saved internally by the NetApp VTL, detailed reports are currently not available in the GUI. Customers can create their own reports via scripts, but an enhancement to extract that data either in direct report form or in a format that could be imported into a third-party program for reporting would be extremely useful.
• When customers upgrade from previous versions of VTL code, backups can be converted to the new format, which supports deduplication, on a tape by tape or library by library basis. Conversion is a background task and can happen during the scheduled window or on demand.

ESG Lab’s View

ESG has found that interest in—and adoption of—VTL solutions is both strong and growing, with the vast majority of organizations, both small and large, relying on a mix of disk and tape to meet data protection requirements. A small—and shrinking—percentage of users are still using tape exclusively for data protection. Organizations of all sizes are being asked—and in many cases told—that they’ve got to have secondary data available and quickly accessible in downtime situations. Not doing so can be costly, if not financially devastating.

ESG Lab has validated that the NetApp VTL can be deployed, configured and performing backups in less than ten minutes. The demonstrated capacity savings of NetApp VTL deduplication change the economics of data protection—making backups more cost-effective as more data is stored for longer periods of time on fast and reliable disk. Using a real-world Oracle database, ESG Lab observed 95% data reduction after just ten full backups, a 22:1 deduplication ratio. NetApp deduplication for VTL is a no-cost feature and can be added to all existing VTL 300, VTL 700 and VTL 1400 systems. The NetApp VTL also delivered enterprise class performance, providing 1,004 MB/sec of sustained throughput for a combination of deduplicated and compressed backups through a single controller, enough to protect 27 TB of data in an 8 hour backup window.

ESG Research has confirmed that Virtual Tape Library (VTL) and deduplication technology can no longer be classified as emerging technology and has moved into the early mainstream. Early adopters have confirmed that VTL slips easily into existing environments, while significantly improving the speed and reliability of backup and, more importantly, recovery. Deduplication transparently and dramatically reduces storage requirements, enabling deep archive of backups on disk. NetApp direct-to-tape technology can reduce the number of media servers needed to support a solution. This can reduce hardware, software and management costs by utilizing the VTL to perform tape exports. ESG Lab believes that the enterprise-class performance and scalability of NetApp VTL solutions, combined with deduplication and the long-standing NetApp tradition of easy implementation and integration, provide a compelling approach to a disk-based backup strategy.

Appendix

[2] Source: ESG Research Report, VTL Adoption and Market Trends, March 2007.

[3] Source: ESG Research Report, VTL Adoption and Market Trends, March 2007.

[4] Source: ESG Research Report, Data Protection Market Trends, February 2008.

[5] The script was configured to use a 2 MB buffer sending data that was 2:1 compressible.

[6] Source: ESG Research Report, Data Protection Market Trends, February 2008.

[7] Source: ESG Research Report, Data Protection Market Trends, February 2008.

[8] Source: ESG Research Report, Data Protection Market Trends, February 2008.