Click here to read the new ESG Lab Report on CommVault Simpana 9.0 “OnePass”

Click here to read a new ESG Analyst Brief on CommVault Simpana 9

With data growing at ever increasing rates, more data sets are simply becoming “too big” to back up — at least not in the traditional sense.  To help combat this, Archive is becoming more and more the steady-partner to Backup, whereby once something is adequately backed up, dormant data can be archived off — making future backups better.

That all sounds like steps in the right direction, but let’s take a look using a “Good, Better, Best” perspective for how these come together:

In other words — One Pass on the data, which (not coincidently) is the name of Simpana’s new feature.

CommVault may not be the only vendor to have ever converged its software’s methodologies, but it is now on a very short list of vendors who are addressing multiple data management problems with a truly unified solution through an elegant architecture.  And most impressively, they did it while not even asking for new licensing or deployment methods.  That’s right, existing Simpana 9.0 customers can take advantage of this by simply applying the most recent quarterly software update and then doing their normal agent update process.  After that, two simple checkboxes in the Simpana management console will enable the unified “OnePass” behavior within the Simpana system.  (check out the ESG Lab Report on all of this)

While I would love to say that consolidating the 3 workflows of Backup, Archiving, and SRM into one process gives you 3X return for your backup window, there are too many variables to make that claim, including:  file types and size, amount of redundancy, archiving retention rules, etc.   But by only traversing the disk system once (instead of for each of the three processes) every Simpana customer should see an appreciable improvement in backup window SLA compliance, as well as the less quantifiable but more appreciable reduced I/O impact on production disks and networks and CPU — all of which will free the production environment to do less backup tasks and more production work.

As always, thanks for reading.

Click here to read earlier ESG coverage of CommVault Simpana

The Challenges

Companies of all sizes continue to struggle with the various aspects of data protection. A great deal of attention is paid to solving not only traditional backup/restore, but also adding archiving and storage resource management to their infrastructures. Along with improving backups of virtualization platforms, laptops, and key workloads, ESG research[1] found that IT end-users planning to implement new data protection initiatives had other goals as well:

• 19% plan to implement data archiving
• 19% plan to implement data deduplication
• 18% plan to re-architect their backup processes
• 13% plan to implement reporting of backup/storage

Users attempting to address diverse backup, archive, and reporting needs often employ technologies from multiple vendors—each with their own agent technologies on individual production servers, as well as their own server back-ends and management interfaces. Each point solution performs its own operations on every production server, including traversing the disk, consuming memory/CPU cycles, and contributing to network traffic.

The Solution: CommVault Simpana 9.0 with “OnePass”

CommVault customers running Simpana software have already learned to appreciate something better than a myriad of point solutions. Simpana software’s common platform delivers backup, archive, search and storage resource management administered from a single console. While built on a single software code base, Simpana software modules have previously utilized separate processes and index databases to run archive jobs, followed by backup and, finally, reporting.

Throughout 2011, CommVault regularly added incremental features to its Simpana 9.0 platform—one of which is a new operating methodology referred to as “OnePass,” which enables backup, archiving, and analytical reporting from a single traversal of the file system. By only reading and/or moving data once, redundant backup, archive, and reporting processes are eliminated to speed operations, reduce storage costs, and simplify management.

ESG Lab Testing

ESG Lab tested the new OnePass functionality at a shared CommVault and HP test facility located in Denver, Colorado. The ESG Lab test bed consisted of a typical Simpana software configuration of one CommServe and two MediaAgents, each configured to protect three HP X9720 scale-out NAS nodes sharing a single file system, as seen in Figure 1.

The test bed was provided by HP to assess Simpana 9.0’s ability to protect a high-volume of unstructured data.

ESG Lab investigated how CommVault consolidated data protection methodologies using the OnePass architecture. The left side of Figure 2 shows the typical IO patterns of three related data management workflows, including traditional backup, file-archival for reducing disk consumption, and reporting services. The right side of Figure 2 shows the combined workflow of the OnePass-enabled agent in Simpana 9.0.

Figure 2 shows how “OnePass” traverses the production storage only once, thereby eliminating significant IO redundancies on the primary server, which should dramatically reduce backup windows and the IO penalties associated with data protection and management tasks.

In a traditional environment using three data management tools, ideally with some level of integration or at least reporting, one might:

1. Perform a traditional backup for data recoverability using traditional incremental methods.
2. After the backup is complete and therefore recoverable (just in case), determine if any files are candidates for archive (hierarchical) management. These files should be “stubbed” to save space, meaning that the original file is replaced with a “stub” pointer referring back to the original file held in near-line storage. This ensures that the actual contents are able to be retrieved transparently when the file is accessed.
3. With the backup finished and the appropriate files migrated to near-line storage, update the reporting system for usage and capacity.

In the case of Simpana OnePass functionality, the operating methodology is similar … yet optimized:

1. The agent conducts a backup of changed files.
2. With the backup changes successfully committed on the media server, the same agent then assesses the files as candidates for archival, and, if so, stubs the file.

• No additional file system traversal is necessary because it was done during the backup.
• No additional disk “read” or network “send” operations are performed during stubbing, as would be required by a separate archival product. The archival process knows that the backup process already read the file and sent it during the backup operation—so it already exists within the Simpana unified storage pool.
• Either way, the archival routines within the OnePass agent simply perform the stubbing operation of replacing the actual file with a stub—after which the file-system driver will handle retrieval requests in case the file is accessed.

1. With the backup complete and the appropriate files archived, the reporting mechanism updates its information.  Again, this occurs without any incremental disk traversal or network operations because Simpana OnePass uses a common index and reporting mechanism from a single collection.

ESG Lab tested the unified OnePass operating model by first conducting separate backups, archives, and report generation using Simpana 9.0 without the OnePass methodology at work. The files were spread across six nodes of an HP NAS and were backed up in parallel by one of the two Simpana media server nodes seen in Figure 1. After the initial testing, ESG Lab audited the results of a similar prolonged test provided by CommVault.

ESG Lab found that the overall backup time was reduced anywhere from 30% to 200% based on three key factors: data types and sizes, amount of redundancy among stored files (e.g., versioning), and archival retention settings that will vary by company. At the low end, even a 30% time savings may mean the difference between compliance with backup window SLAs or not. At the high end, the incremental nature of these backup processes, coupled with nearly transparent archival and SRM functionality, may make the entire backup tax nearly vanish for some production environments.

While less quantifiable, ESG Lab noted that by 1) only traversing the file system once, and 2) offloading the analysis processes to the Simpana MediaServer seen in Figure 2, an appreciable amount of disk IO and CPU processing should be relieved from the production server(s). This means that the production platforms should spend far fewer resources on data protection/management, reserving resulting in more IO and CPU for production purposes.

ESG Lab was impressed by how simple the process was to enable OnePass for Simpana customers. As is typical, the actual agent software components are upgradable through either a push from the Simpana administration console or an .MSI through the customer’s typical software deployment tool. The software can be deployed at any point even if the OnePass functionality is not immediately enabled.

Figure 3 shows how enabling the Archive or SRM reporting functions within the unified agent (i.e., enabling “OnePass”) is simply a matter of two checkboxes within the backup configuration in the Simpana administration console.

Simpana Archive Integration with Scale-out NAS

“OnePass” is not the only innovation recently delivered for the Simpana 9.0 customer base. Along with backing up large file systems, CommVault now also offers its archival capabilities as the near-line extension of scale-out NAS platforms, including the HP X9000 (IBRIX) product family.

By integrating the Simpana software’s archival ability with scale-out NAS, CommVault software is able to offer an additional tier of near‑line storage, enabling organizations to leverage a wider range of storage options at a better price point.

ESG Lab Testing

ESG Lab initially treated the HP X9720 platforms as the production server farm being backed up by Simpana. By reconfiguring the test environment, ESG Lab was also able to test Simpana archival storage as a near-line expansion of a scale-out NAS appliance.

Figure 4 shows the reconfigured test bed with the production NAS being archived by the recent enhancements in Simpana 9.0, using the HP X9000 (IBRIX) platforms as a recent example.

In Figure 4, files accessed from the X9000 platforms can be either taken from their own storage pool or transparently retrieved from the Simpana Archive. While the user “sees” all files just as they would expect to within the NAS, those files may be within the primary storage of the scale-out file system or within the Simpana archival storage pool (using any storage that Simpana software supports).

While some NAS vendors provide their own “archival” capabilities through storage tiering and near-line capacity, it doesn’t always align with the “unified” data protection benefits described above unless 1) backup and reporting are also performed within the NAS/SAN and 2) the NAS/SAN platform is common across the entire corporate environment.  Using a software-based approach, customers may be able to leverage the unified data protection/management capabilities of CommVault software across a wide variety of production servers and NAS platforms consistently—and as a complement to any data management functions that may be offered by the NAS itself.

ESG Lab tested this by enabling the Simpana Linux file server agent on each of the HP X9000 NAS nodes. While many data management products purportedly present challenges when integrating with IBRIX platforms, CommVault is able to use its standard agent with the addition of a registry key on each IBRIX node.

After enabling the agent, ESG Lab tested the user experience by defining archival policies within Simpana software for various files and then retrieving them from an NFS client workstation.

Figure 5 shows two files used during testing of the archive integration with scale-out NAS:

• The top file listing shows the files were originally 100 MB.
• The left statistic reveals each file consumes 102,512 KB.
• The right statistic reports each file’s size as 104,857,600 bytes in the directory listing.
• The middle of the screen reveals that the files were stubbed after archive—consuming only 20 KB each within the NAS, while still displaying 100 MB in the directory listing.
• The last file listing shows that after accessing one of the files, it has been retrieved and thus consumes its regular capacity within the NAS while the other file remains archived until first access.

Note, that while Figure 5 shows the attributes from an NFS perspective, Windows (CIFS) users would have a similar experience where the actual consumption size is masked and the user perception is all files being offered and stored on the HP NAS.

After enabling archival, ESG Lab configured recurring jobs to enable migration of data from the shared file system within the six IBRIX nodes to the Simpana ContentStore. Files that have been migrated will be returned to make file requests from a client workstation accessing the NFS shares on the X9000. ESG Lab observed no appreciable lag in performance or changes in the users’ experience as file requests were routed to the Simpana platform and transparently retrieved from the CommVault software-powered archive.

ESG Lab Validation Highlights

• ESG Lab examined and tested the combined methodology of “OnePass” with appreciably reduced overall data protection jobs, as well as reduced impact to the production servers due to the consolidated network and disk operations of “OnePass.”
• ESG Lab observed how easy it was to enable Simpana software as an archive to a scale-out NAS, without perceivable changes to the end-users’ experience.

Issues to Consider

• ESG Lab found that while it would be easy for an experienced Simpana operator to add the OnePass functionality to their environment, the Simpana administration console may appear complex to someone new. This is a reasonable result of a very mature ninth-generation codebase that continually adds new features and options based on feedback from over 15,000 customers.[3] Those considering converting to Simpana for its OnePass functionality, its other workload-specific capabilities, or its ability to provide an archival store for scale-out NAS should be prepared for a learning curve which can be offset by training.
• While the HP X9000 is just one of the scale-out NAS platforms supported by the Simpana software archival function, customers will want to ensure that their specific platform is currently covered. With CommVault routinely producing updates and incremental functionality, those not directly supported today may be supported later in 2012.

The Bigger Truth

Most environments struggle with a myriad of data protection and management technologies, perhaps because of workload-specific requirements, data center solutions that are less ideal in remote offices, or simply different data management goals (e.g., backup, archive, and reporting). For many, the sentiment has often been “If there was a unified solution that did everything well, then we would all own it already.” For others, the potential interoperability of suite-based software or simply complementary products from the same vendor have left customers disappointed as they discovered that each product operates as if it were the only tool that matters.

By simply enabling the “OnePass” capabilities within Simpana 9.0, CommVault customers can enjoy something that many others should find very enviable: a single agent that backs up, archives, and reports on each production server, with only one network stream and significantly optimized disk-IO impact. The result is something that appears so intuitive that it should be the measure by which other unified products aspire—where functions/technologies may have originally been developed or even acquired separately, but eventually become folded into a single agent talking to a unified back end.

Along with observing the before and after effects of “OnePass,” ESG Lab also tested integration of the archival capabilities of Simpana software with scale-out NAS, showing an appreciable benefit to customers with applicable platforms. Without changing the client experience or installing client-side software, even the most advanced NAS platforms can take advantage of an additional tier of storage through the near-line capabilities of Simpana.

If you are currently using a variety of data and management technologies for different purposes and have been disappointed by the lack of integration or coexistence supportability, then Simpana may be exactly what you have been looking for. While individual test results will vary, the fact that common disk reads and network operations are unified should be a valuable optimization method that all environments can take advantage of. Looking at the unified workflow of Simpana software’s OnePass methodology should make you ask, “Why doesn’t everyone do it like that?”

[2] See: ESG Research Report, Scale-Out Storage Market Forecast 2010-2015, December 2010.

[3] CommVault press release, November 2011

You can read Steve’s other blog entries at The Bigger Truth.

Introduction

Background

Back in 2001, when Data Domain was founded and EMC purchased Belgian startup FilePool for its Centera product line, few in the industry had heard of data deduplication. Fewer still were aware that Permabit, founded in 2000, was hard at work developing scalable data deduplication technology. Today, it’s hard to find anyone in the storage industry who hasn’t heard about data deduplication.

ESG research indicates that there has been a large jump in deduplication adoption in the data protection market. As shown in Figure 1, adoption in 2008 was 11%, and grew to 38% of respondents in 2010 with more concentration in enterprises (45%) than in midmarket organizations (29%).[1] Further, the number of organizations that have adopted, or plan on adopting, deduplication has grown to 78%. ESG expects that interest and adoption in deduplication will increase significantly over the next five years as it reaches mainstream market adoption.

Interest outside of the backup and recovery market is growing as well. In primary storage, data deduplication has been added to network attached storage (NAS) systems  from a number of emerging and market leading storage vendors (e.g., NetApp FAS, EMC VNX). As a matter of fact, ESG research indicates that data reduction technologies are quickly rising on the list of most important features and attributes for storage solutions. This survey of decision makers within enterprise-class organizations indicates that 27% would not purchase a scale-out storage solution without data reduction and 51% would strongly prefer a solution with this attribute.[2] Given the dramatic data reduction benefits that have been realized with existing data deduplication technologies, ESG believes that it’s only a matter of time before deduplication becomes a core component of every tier of storage including primary, backup, archive, block, file, unified, solid state disk, and cloud, and moves up the stack into operating systems, applications, and databases.

Introducing Permabit Deduplication Technology

Permabit’s field-proven data deduplication engine is available as a software development kit (SDK), called Albireo, which enables unified data deduplication. The Albireo SDK is unique in its ability to provide a wide variety of deduplication services. The Albireo SDK can be used to:

• Reduce capacity within file, block, or unified storage systems (e.g., NAS appliances or FC disk arrays).
• Process objects at the sub-file or single instance storage (SIS) level.
• Remove duplicates in real-time (inline) or later after data has arrived (post-process) or a hybrid approach which Permabit refers to as “parallel.”
• Scale from a single storage controller to a cluster of storage controllers or a cluster of deduplication appliances communicating with a storage solution over an industry standard Ethernet interface.
• Provide deduplication services that are “end-to-end and everywhere” in the IT stack—from applications and databases running on physical and virtual hosts through operating systems and hypervisors to storage solutions.

Permabit was founded in 2000 by MIT engineers with a goal of developing a scalable enterprise-class archive product with built-in data deduplication. That product, now known as the Permabit Enterprise Archive, is shown toward the left in Figure 3. Industry standard servers are arranged in a grid, with each server acting as either an access or storage node. The storage nodes are packed full of high capacity SATA drives and presented as a network attached file system (NAS). Servers can be added to the grid for increased performance and capacity.

In the Permabit Enterprise Archive product, a global pool with deduplication technology is implemented within a grid of servers. Permabit has extracted this core deduplication technology to create the Albireo SDK. Albireo deduplication advisory services are accessed through an application programming interface (API) provided by Permabit. A storage system with Albireo running within a single controller attached to a number of drive enclosures is shown toward the right in Figure 3.

Albireo indexes chunks of data and provides advisory notifications when duplicates are detected. The software running in the storage system decides whether or not to take the advice. If the advice is taken, it is the responsibility of the storage system software to update references to duplicate data. The Permabit Albireo API supports both block and stream-based access methods. The stream-based API provides content-aware segmentation to optimize deduplication based on file type. Duplicate advisory services can be provided synchronously, or asynchronously using a registered callback. Taken together, the Albireo SDK provides a flexible and powerful array of deduplication services.

ESG Lab Validation

ESG Lab initially evaluated the Albireo SDK during two days of hands-on testing at Permabit headquarters in Cambridge, Massachusetts in late 2009 and performed an audit of resource and performance improvements in 2011. The evaluation began with an overview of how Albireo deduplication advisory services are used within an existing storage system. As shown in Figure 4 the software running within a storage system uses an API call to send new data and addressing information to Albireo. Albireo ingests the data and performs a SHA-256 hash. A two-stage lookup (memory and, if needed, disk) is performed to see if the data has already been stored. If the data is a duplicate, the API returns the location of the pre-existing data. If the data is new, Albireo remembers the SHA-256 hash and the chunk’s location so that it can advise of duplicate data in the future.

ESG Lab evaluated the Albireo SDK using a pair of programs developed by Permabit:

• PBSCAN: This utility uses Albireo advisory services to determine the savings that can be achieved with deduplication. The program is routinely used at prospective customer sites to determine the benefits of deduplication with real-world data sets.
• DD2FS: An open-source user-space file system was modified to illustrate the ease of integrating Albireo into an existing file system.

The test program was implemented as a single process with the Albireo engine running as a separate service. The pbscan utility was implemented with multi-threads working in parallel.  As shown in Figure 5, the test bed used for this phase of the evaluation consisted of a single quad-core Intel Xeon 3 GHz processor with 4 GB of RAM. For test purposes only, and to more easily discern the resource utilization by Albireo, all but one of the processor cores were disabled in BIOS.

While ESG Lab testing was performed with a single processor core, it should be noted that the extreme scalability of the Albireo engine has been proven in production customer environments. Later in this report, we’ll take a look at how Permabit’s underlying deduplication technology is routinely deployed within a grid of multi-core servers.

Capacity Savings

ESG Lab used the Albireo-enabled pbscan utility and real-world application data collected from Permabit’s production IT servers to evaluate the capacity savings that can be achieved with Permabit deduplication advisory services. The results for two data sets are summarized in Figure 6 and Table 1.

What the Numbers Mean

• A 43 GB set of common office productivity files (e.g., documents, spreadsheets, presentations) was reduced by 32.67%.
• Four VMware virtual server images with a total capacity of 157 GB were reduced to only 4.3 GB. VMware virtual server images are often highly redundant, especially when each virtual machine is running the same guest operating system as was the case for this test. In this example, 97% of the capacity required for the VMware images can be saved with Albireo-enabled deduplication.

The experiment was repeated for a pair of Microsoft Hyper-V images and a week’s worth of Microsoft Exchange backup images. One of the backup images was a full backup; the other images were incremental. The results for all of the data types evaluated by ESG Lab are summarized in Figure 7 and Table 2. Note that the savings are shown as a deduplication ratio instead of a percentage of capacity saved.

What the Numbers Mean

• A pair of Microsoft Hyper-V images was reduced by a factor of 2.1:1.
• A single week’s worth of Microsoft Exchange backups (weekly full, daily incremental) was reduced from 2,203 GB to 501 GB, providing an excellent deduplication ratio of 7.4:1.
• A deduplication ratio of 36.2:1 was recorded for four VMware images.

Resource Efficiency

Identifying duplicate data is a resource-intensive operation. First, data needs to be fed into a deduplication engine.  Moving a lot of data can consume a lot of bandwidth. Next, the engine needs an algorithm which can be used to quickly and accurately identify duplicates. Most deduplication solutions use cryptographic hashing functions to identify duplicates, but hashing a lot of data can consume a lot of CPU horsepower. Last, but not least, the deduplication solution needs to maintain an index of previously processed data to find and keep track of duplicates.  Most deduplication solutions use a two stage lookup, with the first lookup occurring in memory and the second occurring on disk. Indexing lots of possibly duplicate data can consume a significant amount of memory. Each of these resource issues can have an effect on the overall performance of a deduplication solution.

Permabit supports two modes of indexing in the latest version of Albireo. Traditional dense indexing is a full chunk index approach that keeps all index entries available in memory to determine if that data chunk has previously been seen. At scale, this approach limits the efficiency of indexing and degrades performance. If the amount of memory is limited, then the ability to scale out is also limited by the amount of available RAM. Sparse indexing improves memory efficiency using a sampling technique that exploits the inherent locality within data streams. It addresses large scale (e.g., hundreds of terabytes) data chunk lookup bottleneck problems as it avoids the limitations of traditional dense indexing. Permabit has developed a hybrid approach using both sparse and dense indexing with a goal of increasing overall deduplication efficiency by 10X or more with only 0.1 bytes of RAM required for each chunk of indexed data.

ESG Lab’s analysis of CPU and memory efficiency began with a review of Permabit’s patented two-stage deduplication detection algorithm.[3] The patent describes a highly efficient two-stage index residing in memory and on disk. This novel approach uses a combination of bit sampling and byte differencing to provide a first stage memory lookup that executes very quickly, consumes very little memory, and has a very low probability of false positives.

A 42 GB set of office productivity files (documents, spreadsheets, presentations, PDFs, etc.) was processed by an Albireo-enabled deduplication utility. The utility was used to quantify the speed and resource overhead of Permabit deduplication advisory services. The utility opened and passed all of the data within each file to the Albireo API. The Albireo API was used to identify and keep track of chunks of data in a Permabit index of duplicate candidates.

Linux utilities were used to record CPU and memory utilization with Albireo services running on a single Intel Xeon processor. Trace output was used to record the average latency for each Albireo deduplication fingerprint API lookup. In addition, trace data was used to record the end-to-end latency associated with Albireo-enabled deduplication including the SHA-256 ingest, the Albireo fingerprint API, the first pass memory index, and a second order index operation for likely duplicate candidates. The results are summarized in Table 3 and Figure 8.

What the Numbers Mean

• The Albireo fingerprint API delivered fast in-memory deduplication lookups that completed in 9 to 17 microseconds.[4]
• A SHA-256 hash and index with a 4 KB chunk size incurred only 43 microseconds of latency. That’s less than 1% of the overhead associated with a typical disk access latency of 5 milliseconds.
• Albireo consumed less than 3.5 bytes of RAM per index entry with traditional dense indexing and 0.1 bytes per entry with sparse indexing.
• A sparse index overhead of 0.1 bytes and a 4 KB chunk size allows deduplication up to 40 TB of data with 1 GB of RAM and up to 2.56 PB of data with 64 GB of RAM.
• Albireo-enabled deduplication, including SHA-256 hashing, consumed approximately half (28% to 59%) of a 3 GHz Xeon processor core.
• A larger chunk size consumed slightly more CPU. This is due to the fact that bigger chunks of data were passing through the CPU-intensive SHA-256 algorithm.

Scalability and Performance

Permabit’s global deduplication algorithm is designed to run within a grid for maximum performance and capacity scalability. Permabit currently supports up to 4.6 PB (4,600 TB) in a single pool of capacity. The Permabit Archive product is continuously tested with Albireo deduplication services running on a grid of three or more servers. A typical entry-level grid at a customer site, comprised of access and storage nodes, has deduplication services running on 11 nodes. The largest grid deployed in production at a customer site is 38 nodes.

The performance scalability of a scale-out deduplication solution is dependent on its ability to handle two resource intensive tasks:  hashing and index lookups. The Albireo architecture was designed to maximize the efficiency and scalability of both. The Albireo architecture can support cryptographic hashing (e.g. SHA-256) distributed over multiple CPUs or offloaded to special-purpose hardware, and delivers scalable index lookup performance using an engine that can be deployed across multiple CPUs within a single server or a grid of Ethernet connected servers.

ESG Lab audited the results of nightly regression tests to assess the performance scalability of Albireo index lookups.  A Permabit developed utility was used to process data that was deterministically unique (i.e., no duplicates) during this test. This type of data was used with a goal of maximizing the stress on the Albireo deduplication advisory engine.

The performance of the Albireo fingerprint API was tested with 4 KB chunks on a single server. The number of chunks processed per second was multiplied by the chunk size to calculate the effective throughput of the Albireo deduplication advisory engine. Similar calculations were performed to assess throughput when processing 64 KB and 128 KB chunks of data. As shown in Figure 9, increasing the chunk size increased deduplication index lookup throughput to 39.74 GB/sec on a single server with four Intel Xeon processor cores.

These results not only highlight the efficiency and speed of the deduplication engine running on a single server, they also demonstrate the power of supporting a programmable chunk size. It also provides the flexibility to choose between maximizing deduplication rates (smaller chunks) or minimizing index overhead and maximizing the throughput of the deduplication engine (larger chunks).   A programmable chunk size can also be used to simplify integration with an existing architecture.

Similar tests were performed as nodes were added to an Albireo grid. As shown in Figure 10, performance scaled in near linear fashion to 412 GB/sec for a sixteen-node grid processing 128 KB chunks of data.

Fault Tolerance

ESG Lab performed a series of tests on a nine node Permabit Enterprise Archive system to determine whether Albireo deduplication advisory services continue running after multiple hardware failures. A long running directory level file copy operation was started. As shown in Figure 11, a node was powered off and a drive was removed on a different node. The system remained available and the copy operation completed without error.

Ease of Integration

ESG Lab evaluated the ease of adding deduplication to an existing storage solution using the FUSE open-source user-space file system framework.[5] The FUSE file system is built over the native ext3 Linux file system. There’s no need to patch or recompile the kernel. Permabit created a file system using FUSE named dd2fs. The dd2fs file system was modified to use Albireo to identify and eliminate duplicates. Six Albireo API calls and 52 lines of supporting code were added to the 1,563 line FUSE demonstration program. Synchronous (inline) and asynchronous (post-processing) deduplication modes were demonstrated.

As shown in Figure 12, the update_block function passes data to be written to the Albireo uds_index_block API. A return value of 1 indicates that the block is a duplicate and that it can share storage with the canonical chunk. After a bit of bookkeeping and error checking, the storage associated with the duplicate block is freed. Other than this key function call, the majority of the Albireo-related code changes were isolated to initialization, shutdown, and handling callbacks when operating in asynch/post-process mode.

The FUSE demo was run to see Albireo-enabled deduplication in action. As shown in Figure 13, a 64 KB file full of random data was written to an empty Albireo-enabled file system. A copy of the file with a new name was added.  The Linux df and du utilities were used to verify that the user’s view of the file system included the space consumed by both files, yet only a single file’s worth of disk capacity was consumed. The Linux md5sum utility was used to verify that the files were the same. As each file was deleted, the file system capacity and underlying disk capacity were checked.

Maturity

ESG Lab performed a high level assessment of Permabit’s software development processes to understand the maturity of the Albireo SDK. The bulk of the code within the Albireo SDK has been used within Permabit’s shipping products for more than seven years. Permabit has been using agile software development processes for more than eight years.

Agile software development refers to a group of software development methodologies based on iterative development where requirements and solutions evolve through collaboration between self-organizing cross-functional teams. The term was coined in the year 2001 with the formulation of the Agile Manifesto.[6] Agile methods generally promote a disciplined project management process that encourages frequent inspection and adaptation; a leadership philosophy that encourages teamwork, self-organization, and accountability; a set of engineering best practices that allow for rapid delivery of high-quality software; and a business approach that aligns development with customer needs and company goals.

The code at the core of Albireo SDK is continuously integrated and tested on a two week iteration cycle. Stories captured in an online Wiki are used to manage requirements. Unit, functional, and stress tests are highly automated and run continuously. Developers write the majority of unit and system level tests. All changes are peer reviewed. Face-to-face cross functional interaction is embraced. Reducing code complexity is valued and recognized.

ESG Lab Validation Highlights

• The Permabit SDK identified potential capacity savings ranging from 33% to 97% for real-world applications including office productivity files, virtual server images, and e-mail backups.
• Fast and resource efficient deduplication indexing was confirmed. Less than 0.1 bytes of memory per index and deduplication advisory throughput of 412 GB/sec was measured on a 16 node quad-core Xeon grid.
• An open-source user-space file system was modified to use Albireo deduplication advisory services using six API calls and only 52 lines of code.
• Synchronous and asynchronous APIs were used to implement inline and post-process data deduplication, respectively.
• Permabit’s agile software development processes were audited.
• Systems in the lab and in the field confirm that Permabit global deduplication advisory services have been deployed on grids of up to 38 servers and validated on grids of 96 servers in Permabit’s test lab.
• An error injection test confirmed that the Albireo deduplication services running within a field-proven Permabit Enterprise Archive solution remain available after both a drive and a server failure.

Issues to Consider

• The Permabit Albireo SDK detects duplicate data, but it does not actually remove it. Removing duplicates and maintaining pointers to duplicate data is implemented within the storage system that uses the Permabit SDK. Data structures, which map and keep track of duplicate data references, are needed to take advantage of the Permabit SDK. This is a trivial consideration for NAS systems which use an inode map to keep track of data on disk. For modern block-based disk arrays this service is often available for thin provisioning or virtualization, but it may be an issue that impacts the complexity and resources associated with Albireo integration.
• An Albireo-enabled storage solution can continue operating even if Albireo becomes unavailable. If and when Albireo becomes unavailable the system is unable to detect new duplicates, but data access is unaffected. This is due to the fact that Albireo only provides deduplication advice, and the storage system uses that advice to eliminate duplicates while maintaining data integrity.
• While the Permabit architecture has been designed to use any hashing algorithm, Permabit has been using the SHA-256 algorithm for years. While the 256-bit SHA2 hashing algorithm virtually eliminates the risk of deduplication induced data corruption due to a hash collision, it does add CPU overhead compared to less rigorous 128-bit algorithms (e.g., MD5).
• The API integration and capacity savings results presented in this report were collected using relatively simple test programs running on a single server. Estimating the effort required to evaluate, architect, and implement a solution using a production storage system is beyond the scope of this report. Similarly, estimating the savings that can be achieved with your customer’s data is beyond the scope of this report.  Testing in your lab, with your storage solution, and with your data is strongly recommended.

The Bigger Truth

One of ESG Lab’s first projects was a 2004 validation of a disk-based backup appliance with built-in data deduplication from Data Domain.[7] Since then, data deduplication has evolved into the hottest, most paradigm shifting technology to hit the storage industry since the UC Berkeley RAID papers were published in 1989. Like RAID, data deduplication quickly permeated the storage market due to its outstanding value proposition. Storage administrators struggling to finish backups within shrinking windows were able to reduce the capacity required to retain backups on disk by 90% or more. The value of this new technology was clearly compelling: data deduplication reduced the cost of disk-based backups, putting it on equal, or better, footing than tape. Backups that finish within a shrinking window and quick ad-hoc restores from disk had suddenly become economically feasible.

In recent years, data deduplication has begun to permeate the storage industry. A number of startups, including Diligent, Sepaton, and Exagrid, followed Data Domain into the disk-based backup appliance market. Since then, all of the major systems vendors have added deduplicating backup appliances to their portfolios. More recently, all of the major backup software vendors have added deduplication to their offerings. Content addressable disk-based solutions with embedded data deduplication technology were introduced in the archive market. Permabit was among the first vendors to enter this growing market. Deduplication has been used to reduce WAN traffic within primary and secondary storage replication solutions. And last, but not least, data deduplication is beginning to take hold in the primary storage market, with NetApp and Microsoft leading the way (Deduplication for FAS and Single Instance Store within Windows Storage Server, respectively).  Deduplication within primary storage solutions has proven to be extremely effective in virtual server and virtual desktop deployments that tend to have lots of data in common.  In ESG’s opinion, it’s simply a matter of time before data deduplication gains wide market acceptance throughout the primary storage market.

Data deduplication technology has driven a number of strategic acquisitions. ADIC purchased deduplication technology from Rocksoft for $63M (ADIC has since been acquired by Quantum). IBM acquired Diligent in a deal rumored to be worth between $160M and $200M. Those acquisitions were dwarfed by EMC’s acquisition of Data Domain, where a bidding war with NetApp drove the value of the deal up to $2.1B in cash.

So what’s the big deal with deduplication? It’s actually rather simple: deduplication reduces storage capacity requirements up to 99%. In other words, IT managers can squeeze up to a hundred times more out of each dollar they spend on disk capacity. Even as data deduplication becomes more of a feature than a product, the value is clearly compelling. While one could argue that the hype and valuations of deduplication solutions in the backup arena have gotten a bit ahead of the market, it’s clear to ESG that we haven’t seen the peak in the archive and primary storage markets yet.

The deduplication market has begun to mature in recent years. As the feature becomes more of a check-off item, vendors are leveraging differences in architectures and implementations to grow market share. Aside from the usual delineations based on price and performance, vendors are competing based on the finer differences between deduplication solutions: object vs. block-based, inline vs. post-process, fixed vs. variable length, and global vs. islands of deduplication. Permabit has extracted the core deduplication technology from a field-proven archiving solution with a goal of delivering a deduplication algorithm that can be used to architect solutions with any of these attributes in mind. In other words, instead of arguing the merits of one implementation versus another, Permabit enables a vendor to implement multiple alternatives and just say “yes”.

ESG Lab has confirmed that Permabit Albireo deduplication advisory services work as advertised. Inline and post-process deduplication support was added to a user space file system with only six Albireo function calls and 52 lines of code. The capacity of real-world data sets were reduced between 33% and 97%. The patented deduplication lookup and indexing algorithm was fast and efficient. Permabit deduplication was observed running on more than one server for a scalable global pool of deduplication and fault tolerance. ESG Lab saw no interruption in access when errors were injected on a nine node Permabit Enterprise Archiving grid.

ESG Lab’s experience with nearly all of the vendors that have brought data deduplication solutions to the market indicates that correctly implementing data deduplication is a difficult task that requires man-years of effort.   Performance, resource efficiency, and scalability have proven to be particularly challenging for a number of vendors. On top of the technical challenges, this relatively new and valuable technology has a growing number of patent portfolios that need to be navigated.

Speaking of patents, Permabit has been awarded a total of 26 patents covering diverse areas in data protection and archive and many more filings are pending in similar areas. The growing portfolio includes patents in the areas of hash-based deduplication for scalable file and object data storage, encrypted deduplication, memory-based snapshots, and many other features of Permabit’s product line. ESG Lab was particularly impressed with the resource efficient two-stage indexing method described in Patent 7,457,813, Storage System for Random Blocks of Data. Highlights of that well-claimed patent are summarized in the resource efficiency section of this report.

ESG Lab is confident that the flexibility provided by the Albireo SDK is unique in the industry. ESG Lab has confirmed that Albireo can be used with file or block storage. It provides deduplication services at the object or sub-file level. It supports inline and post-processing programming models with minimal performance and resource impact. Running over a grid, it can be used to create a global pool of deduplication with predictably scalable performance and rock solid reliability. It provides deduplication capacity savings that are far greater than can be achieved with compression. Stream support can be used to provide content aware data deduplication for objects that are misaligned with the block boundaries of an underlying file system. This allows data stored in container formats (e.g., TAR and ZIP files) to be intelligently deduplicated.

Last, but not least, the Permabit Albireo SDK was designed with quick and easy integration in mind. Based on hands-on experience with the Permabit Albireo SDK, ESG Lab believes that Permabit deduplication can be tested within an existing storage system in a matter of weeks. Given the growing size of the market for capacity reduction and the high cost of developing a deduplication solution, organizations considering the merits of adding data deduplication to an existing storage solution should seriously consider a test drive of Permabit’s field proven, patent protected deduplication algorithm.

Appendix

[2] Source: ESG Research Report, Scale-out Storage Market Trends, December 2010.

[3] US Patent 7457813, Storage System for Random Blocks of Data, Nov 25, 2008.

[4] Confirmed via a review of traces of code execution through the in memory index code path.

[5] fuse.sourceforge.net

[6] Agilemanifesto.org

[7] See: ESG Lab Report, The Data Domain DD200 Restorer, February 2004.

by Lauren Whitehouse

via The state of backup dedupe – Storage Technology Magazine.

Quantum DXi Accent is a no-cost feature add-on to the company’s popular DXi portfolio of appliances for enterprise, midmarket, remote/branch office and small business backup environments, beginning with the DXi6700 Series. DXi Accent installs on the backup media server (in this case, it’s limited to Symantec NetBackup media servers and requires the use of Symantec’s OST).

DXi Accent, like Data Domain Boost, executes the deduplication process earlier in the backup data path. At the media server, DXi Accent chunks the backup data stream into blocks, calculates the hashes for each, and only sends unique data to be stored. This ensures less traffic between the media server and backup target and accelerates the process.

Those evaluating Quantum versus EMC should look at a few differences between DXi Accent and Data Domain Boost:

• Using DXi Accent is not an “all or nothing” scenario as with Data Domain Boost. With DXi Accent, users have a choice to enable or disable the feature on a media server-by-media server basis.
• DXi Accent can be used on a LAN or WAN, while Data Domain Boost is limited to a LAN. Remote backup performed over a WAN from a media server (with DXi Accent but no local DXi appliance) to a central location (where the DXi appliance is located) is supported.
• Quantum’s strategy of bundling high-value features, such as device-to-device replication, direct tape creation, and Symantec OST support—and now DXi Accent—provides Quantum with a price advantage versus Data Domain that has an a la carte licensing approach.

Quantum continues to demonstrate its moxie in the target backup segment. The Quantum DXi portfolio of disk-based backup appliances with deduplication is surely worth a look.

You can read Lauren’s other blog entries at Data Protection Perspectives.