Background

While IT priorities and challenges are often considered with data centers and other centralized corporate resources in mind, it is important to remember that organizations often have distributed locations that have significant and complex computing requirements. In fact, typical IT challenges are often exacerbated in these remote/branch offices due to distance and lack of onsite IT staff. ESG research found that companies face significant challenges when it comes to delivering applications over the WAN from a central location to employees at branch office locations. According to a recent ESG survey,^[1] nearly half of respondents identified improving application performance for end-users as a key initiative (see Figure 1). Improvements to application accessibility and better collaboration capabilities were also high on the list.

Riverbed Steelhead EX + Granite Overview

Riverbed Technology’s Steelhead product family is designed to provide increased application performance and data transfer speeds over the WAN. Steelhead products address four main solution areas.

• Application Acceleration—Steelhead optimizes both TCP and UDP traffic, addresses application-specific latency, delivers LAN-like performance and availability over the WAN, and enables improved collaboration, file sharing, and productivity for distributed enterprises.
• Bandwidth Optimization—enables network managers to achieve better utilization of existing WAN bandwidth by eliminating redundant WAN traffic.
• IT Infrastructure Consolidation—enables consolidation of IT infrastructure from remote offices to a centrally located facility, maintaining performance, availability, and security, as well as reducing capital expenditure and management costs.
• Backup & Replication Acceleration—enables quick and secure backup and replication from branch locations.

The Riverbed Steelhead family of products is designed to optimize WAN traffic between distributed remote and branch office locations and a central data center. Steelhead appliances run the Riverbed Optimization System (RiOS), which is the software platform that enables data-, protocol-, and application-level WAN optimization and allows a central office to consolidate the majority of its remote office server infrastructure, taking the first step toward true infrastructure consolidation. Riverbed Steelhead EX combines WAN optimization capabilities with VMware on the appliance, enabling a branch to virtualize local servers and minimize the bandwidth required by users and applications.

Granite was developed by Riverbed to deliver edge virtual server infrastructure that extends an enterprise storage area network (SAN) out to remote offices. It enables organizations to centralize and consolidate branch office storage at a primary data center. Steelhead EX + Granite combines Riverbed Steelhead and Granite software capabilities with the goal of serving write-intensive and custom applications in the branch with a global storage infrastructure projected from the data center, eliminating storage at remote branch offices previously considered too difficult to consolidate.

ESG Lab Validation

ESG Lab performed hands-on evaluation and testing of the Riverbed Steelhead EX + Granite WAN optimization and storage consolidation appliance at a Riverbed facility in San Francisco, California. Testing was designed to demonstrate how Steelhead EX + Granite improves application performance and availability, as well as delivers infrastructure consolidation and data security for branch locations in a distributed enterprise.

Getting Started with WAN Optimization

Steelhead appliances at remote locations, along with Steelhead Mobile software on remote user laptops, work together with one or more Steelhead appliances in the corporate data center to optimize traffic flowing over the WAN. Riverbed addresses three areas that affect WAN efficiency: application chattiness, data redundancy, and transport protocol inefficiency.

To speed application performance, application-specific optimizations complete transactions locally in the branch on behalf of servers in the data center, eliminating the need to wait for application responses over a WAN connection. To reduce the amount of data sent over a WAN, Steelhead appliances and software intercept and inspect WAN data to determine whether the data or a portion of it has been seen before. When a user attempts to access data already encountered by the local Steelhead datastore, the data is served locally, eliminating the delay of pulling data over the WAN. With this capability, Steelhead appliances allow users and applications to read and manipulate data, while only requesting or sending unique blocks across the WAN. To overcome transport protocol limitations, Steelhead appliances more intelligently scale and pack TCP payloads, significantly reducing round trips and more efficiently transmitting data across the WAN.

Figure 4 shows the test bed used by ESG Lab, which consisted of a simulated data center and remote office. A Network Nightmare WAN simulation device was used to limit bandwidth and inject latency, simulating a transcontinental T1 link. The data center was configured with one NetApp FAS 2050 connected to both Steelhead and Granite core appliances. The remote office had a Steelhead EX + Granite appliance running Steelhead WAN optimization software, Granite block-storage acceleration, and the Riverbed VSP. The connection between the two environments was limited to 1.5 Mbps (T1 equivalent) and had 100 milliseconds of round-trip latency injected to simulate a remote office connecting to a data center up to 3,000 miles away.

ESG Lab Testing

For the first round of tests, Steelhead WAN optimization was used to optimize typical knowledge worker operating tasks. Data reduction and optimization-related metrics were captured using the Steelhead management console as well as wall-clock timing of certain operations. As shown in Figure 5, simple file transfers, Microsoft Exchange messages with attachments, and Microsoft SharePoint transaction performance were measured with and without Riverbed Steelhead WAN optimization enabled. The corresponding results show performance improvements of anywhere from 5X to 50X, depending on the type of transaction.

The data in Table 1 includes the sizes of the objects used in performance testing and the number of seconds to completely execute each operation. The largest time reduction was seen with the transfer of a 65.3 MB file from a remote client to the corporate file server.

Consolidation of Branch Services

Organizations are using server virtualization to simplify their IT infrastructures while reducing costs in their data centers through consolidation. For services required in branch offices, consolidation (minimizing infrastructure in the branches) is also a key goal. However, organizations are faced with a challenge in that certain applications require local compute and storage resources to meet performance requirements for end-users. Steelhead EX features the Riverbed Virtual Services Platform (VSP) that incorporates VMware virtualization technology to consolidate branch servers and applications onto the Steelhead EX appliance.

ESG Lab Testing

ESG Lab began testing the ease of consolidating branch office services and applications using Riverbed Steelhead + Granite technology by simulating the user experience of moving from a traditional branch office with dedicated servers, applications, and local storage to a virtualized Riverbed WAN-optimized and consolidated model, as illustrated in Figure 6.

ESG Lab first virtualized a Microsoft Windows Server 2008 system in the remote location by installing the VM onto the Riverbed VSP. This is accomplished via the Steelhead management interface. The Steelhead EX + Granite appliance can host up to five end-user virtual machines. ESG Lab connected to the Steelhead EX + Granite appliance through the Riverbed Steelhead Management Console to install the virtual server as shown in Figure 7.

Once the virtual machine was loaded and running, ESG Lab simulated a Microsoft SQL server workload using the Iometer load generation utility. SQL Server is an application that, due to the response-time sensitivity of transactional databases, is often hosted on physical servers with local storage in a branch office. In this test, a 4 KB block size was used with a 67% read, 100% random access pattern.

Figure 8 shows the Iometer results displayed during the test. The most important metric to note here is “Average I/O Response Time (ms).” While the back-end connection to the storage array was over a simulated T1 connection with 100 ms of round-trip latency, Iometer reports only 31 milliseconds of latency to disk because data is being written to the local Steelhead blockstore.

It’s important to note here that without Steelhead EX + Granite, ESG Lab was unable to obtain a usable result due to the restricted bandwidth and high latency of the WAN link. In fact, the connection to the LUN in the data center timed out, and the mount failed.

Consolidating Branch Office Storage

Riverbed Granite extends iSCSI block storage from the data center to the remote site in a way that is transparent to users and applications, and that takes advantage of Riverbed Steelhead WAN optimization technology. Granite enables organizations to maintain local servers at branch offices while actually storing and protecting their data within their data centers. Riverbed VSP provides the ability to host those servers directly on the Steelhead EX appliance.

ESG Lab Testing

ESG Lab tested Granite by mounting iSCSI LUNs from the data center on physical branch server nodes as well as within the virtual machines hosted in the Steelhead EX + Granite appliance. To verify the challenge of accessing “unoptimized” iSCSI storage over the WAN, ESG Lab first attempted to mount an iSCSI LUN directly from a traditional branch server to a data center SAN without Granite, and observed that the connection timed out and the mount failed.

Next, ESG Lab tested whether Riverbed Granite could allow the use of iSCSI over the WAN by configuring Granite appliances in both a data center and a remote office location. Figure 9 shows the basic functional design of storage extended with Granite. In effect, there are two iSCSI connections (working from right to left):

• Within the data center (right), between the actual iSCSI SAN target to the Granite core iSCSI initiator.
• Within the remote site (left), between the production server iSCSI initiator to the Steelhead EX + Granite iSCSI target.

Using the same branch server, ESG Lab was able to successfully mount with no errors or timeouts the same iSCSI LUN that had previously failed to mount in the “unoptimized” test.

The combination of Granite with Steelhead WAN optimization makes it possible for a data center LUN to be successfully mounted by a remote office production server—either a physical server located in the branch or one that is virtually hosted within the Riverbed VSP hypervisor.

Next, ESG Lab examined performance and usability. For this test, ESG Lab (working right to left in Figure 9):

1. Configured multiple LUNs within the data center SAN, which is completely unaware of the Steelhead EX + Granite extended storage scenario.
2. Configured the Granite core iSCSI initiator to mount the LUNs that are to be extended.
3. Assigned a specific Steelhead EX + Granite edge device from the Granite core at the data center to extend each LUN. By doing so, the Steelhead EX + Granite device became an iSCSI target and offered the LUN to devices within the remote site.
4. Connected the LUN to the production Windows server using its iSCSI initiator, with the server being completely unaware that the LUN is not within the remote site but actually extended from the data center.

From there, the LUN behaved like any other iSCSI-attached device and could be mounted and utilized. A common concern about remote storage is that the latency associated with initial use or access requests will have an impact on the end-user experience. To observe the behavior, ESG Lab configured an extended LUN within Steelhead EX + Granite that contained a known data set and requested various randomly selected files.

Figure 10 shows the Steelhead EX + Granite Blockstore Metrics report, which measures the amount of “hits” (requested blocks that were already available at the branch) and “misses” (blocks that needed to be sent from the data center) in megabytes. This, in effect, measures the effectiveness of the Granite technology to pre-fetch and/or quickly transmit the necessary blocks, such that the branch server experiences the storage as though it is local.

ESG Lab used AutoCAD software to open a 33.5 MB file (cifs.dwg). The first time the file was accessed across the WAN, the file-open process took 27.7 seconds. The file was closed and the workstation was rebooted to clear client cache. When the workstation was restarted, the same file was opened using AutoCAD, and the file opened in 5 seconds, the same as baseline testing over the LAN.

While Figure 10 shows the Granite technology’s effectiveness in pulling data from the data center to the branch, Figure 11 shows the behavior of data written at the branch being committed to the data center.

• The dark blue line tracks the amount of data being written to the Granite blockstore.
• The light blue areas show data received but not yet written back to the data center storage array, implying minimal latency in transmitting those blocks to the data center.
• The grey areas show data that has been committed to the data center.

Overall, ESG Lab found the Riverbed Granite extended storage solution to be surprisingly easy to configure and utilize. Neither the production server at the remote site nor the data center SAN felt any impact from the extended distance. The iSCSI implementation was intuitive and performed well over a simulated 3,000-mile WAN connection. The combination of Granite with Steelhead technology dramatically accelerated data transfers over low-bandwidth, high-latency connections.

Remote Office Scenarios and Implications

After the servers were virtualized and the SAN storage was extended from the central data center, ESG Lab was ready to explore the performance and resilience of a Riverbed-enhanced remote office when a WAN link goes down and is restored.

ESG Lab Testing

First, ESG Lab set a baseline by opening several large files that resided on an extended iSCSI SAN volume shared on the remote office LAN by a virtual server hosted in the Steelhead EX + Granite appliance without Granite acceleration. ESG Lab opened a 39.4 MB AutoCAD file named “http.dwg” using a client on the remote office LAN. Without Steelhead and Granite optimization, the file opened in 721 seconds, or just over 12 minutes. The AutoCAD application was completely unresponsive while the file was pulled across the WAN.

Next, the same file-open test was performed with Steelhead WAN optimization and Granite active. The first time the file was accessed, the operation completed in 39.1 seconds. After closing the file and opening it a second time, the file opened in 5.0 seconds. As expected, the second open was much faster, being serviced by the blockstore cache on the Steelhead EX + Granite appliance. Saving the file to a new name took 3.1 seconds.

As seen in Figure 12, at 18:07, ESG Lab disconnected the simulated WAN between the remote office and data center, and attempted to open the same file. The file opened successfully, in 5.6 seconds. Next, the file was saved to a new name, which completed in 3.1 seconds. This is comparable to the performance observed when the WAN was connected.

ESG Lab repeated these procedures multiple times, opening files and saving them to new names. Performance was consistent. Figure 12 shows the data writes/commits report from the Steelhead EX appliance.

As shown in Figure 12, as data was written to the network share, the uncommitted bytes that had been accepted by the Steelhead appliance but not yet transmitted to the data center increased. Note that the entire time that the volume was disconnected, the client and server at the remote site remained connected, and the volume remained online.

After approximately 30 minutes, more than 40 MB had been written to the shared volume. ESG Lab then reconnected the WAN and monitored the data writes/commits report. As can be seen in Figure 13, the Steelhead appliance committed the 43.6 MB of data to the NetApp FAS in the data center in about 45 seconds.

The resynchronization was automatic and completely transparent. Users and applications saw no change in connectivity or access when the WAN link was down, nor when it came back up. Figure 14 shows a network traffic summary report for the time period that the resynchronization was executing.

The port number indicates the type of traffic: Port 7951 is traffic flowing between the Steelhead EX + Granite edge device and Steelhead and Granite core devices in the data center. The actual data transmitted across the WAN link was highly optimized, and, of the 43.9 MB of iSCSI data transmitted by the virtual server, only 3 MB was actually transmitted across the WAN, a reduction of 93%.

Data Protection Scenarios and Implications

When extending storage from the data center to the remote office, data protection becomes multifaceted, incorporating not only backup and recovery of production data, but also protection of remote office data from loss or theft.

Securing Data in the Appliance

Riverbed appliances utilize integrated storage to hold cached data in remote locations, designed to enhance the remote user’s experience by providing local access to frequently used data. The Riverbed Steelhead + Granite appliance offers AES encryption (up to and including AES-256) to securely encrypt the data on disk. The AES key for the Granite blockstore is kept in a secure vault area, which is also encrypted using AES-256.

The default key to each appliance vault is unique, derived from a unique identifier of each appliance. The vault key can be changed by organizations to comply with their own security standards. When an appliance boots, the vault key must be provided, or the contents of the blockstore are not accessible. A visual representation of Riverbed encryption is shown in Figure 15.

Backup and Recovery

When considering backup and recovery, multiple data protection scenarios are either enabled or enhanced, including:

• File/application-based protection of the remote data, from the data center
• Block-based protection of the LUNs used by the branch platforms, from the data center

In principle, because server-centric storage utilized at the branch is in fact extended from the data center via Granite (and user-centric data from the branch is stored locally on Granite-extended volumes), Riverbed suggests that data protection can be done entirely from the data center instance of the data.

ESG Lab audited the operating methods that Riverbed uses for storing its data to understand the viability for customers to use their current data-protection methods within a Steelhead EX + Granite deployed configuration.

Traditional File/Application Backups from Guest-VM Branch Servers

For production servers running at the branch, presumably as virtual machines within the Steelhead EX + Granite (VMware Virtual Server) host environment, traditional file- and/or application-centric backups are still achievable.

ESG Lab looked at how a typical backup agent can be installed within a virtualized production OS to send backup data to the requesting backup server located at the data center, as shown in Figure 16. In this configuration, as files are queued to be sent from the production VM to the backup server, Steelhead WAN optimization is designed to recognize the data that already exists at the data center from previous synchronizations. In this case, while both the remote backup agent and the data center backup server believe that the data is being sent across the WAN, only truly unique data segments and reference “pointers” to previously encountered data actually traverse the network.

Based on how ESG Lab tested file transfers with Steelhead EX + Granite, file-centric data movement during backups should be nearly eliminated. Similarly, application-centric backups that generate storage IO as part of the backup process (such as SQL Server log files) will benefit. In those cases, as the data files are prepared for backup, their corresponding blocks within Granite will be committed to the data center and therefore may not need to traverse the WAN during the actual backup.

SAN-based Backup of the Branch from the Data Center

Perhaps the most intuitive and yet subtle method ESG Lab observed was the ability to back up the extended LUN from within the data center. Because the SAN is unaware of the Steelhead/Granite solution, LUNs can be backed up directly using storage-based snapshots and clones—traditional “serverless” backup solutions.

As seen in Figure 17, ESG Lab observed that by backing up the original LUNs from the SAN, all of the production data could be protected in the data center, including virtualized server-centric data and client-specific data that are Granite-extended.

As shown in Figure 17, ESG Lab found serverless backups of remote office volumes to be potentially ideal choices for those customers who already utilize them within their data centers. The IO burden is removed not only from the production resources, but also from the Steelhead and Granite appliances, freeing them up for production IO exclusively. Like any serverless backup, an understanding of the applications in use and the need for consistency and post-backup processing are keys to success.

Riverbed SAN Hardware Snapshot Integration

Along with extending and potentially enhancing customers’ existing backup methodologies, Riverbed has also developed a Riverbed Hardware Snapshot Provider (RHSP) mechanism to directly integrate its storage-extending capabilities with both the Microsoft Volume Shadow Copy Service (VSS) and SAN arrays from EMC, Dell EqualLogic, and NetApp.

Although RHSP was not tested by ESG Lab, a discussion with Riverbed highlighted RHSP capabilities that directly address the need for application consistency with backup. RHSP installs as a plug-in on Windows clients in the branch office. It is used within the VSS process to place a point-in-time marker into the Granite blockstore. This enables a backup agent to quiesce an application to indicate an application-consistent restore point. In turn, this indicator triggers a snapshot on the data center SAN storage array that can then be used for any required restores or subsequent secondary backups to disk or tape in the data center.

ESG Lab Validation Highlights

• ESG Lab used Steelhead appliances to optimize WAN performance and reduced data by up to 50 times, enabling more productive collaboration between remote and central offices.
• When running an OLTP database workload in a Riverbed appliance-hosted virtual machine, ESG Lab observed latency to storage over the WAN decreased by more than 67%, enabling a remote server to mount an iSCSI volume hosted in a distant data center, and making it possible to consolidate business-critical branch services over the WAN.
• ESG Lab found Granite-extended block-storage performance over a simulated transcontinental WAN link to be remarkably viable, rivaling local-attached storage in both throughput and latency.
• ESG Lab validated that Riverbed improved performance by a factor of 18 to 26 times when opening and editing large CAD files across a high-latency, low-bandwidth T1 link, providing WAN access to centralized project files at LAN-like speeds.
• ESG found that customers’ existing backup methodologies were all potentially viable options for remote offices—in ways not achievable without the combination of WAN optimization and storage extension. Without changing their backup mechanisms, customers may find their solutions enhanced because of how Steelhead optimizes the data streams that Granite has already synchronized between sites.

Issues to Consider

• It should be noted that one key to the performance of the extended storage is the built-in blockstore within the Steelhead EX + Granite appliance itself. When designing the storage to be used at the branch, customers should be aware that the Granite solution does not change normal design considerations around capacity or IO performance. Those aspects should still be considered when determining the size of the Granite edge device to place at a particular branch location.
• While many data-protection scenarios are enhanced through this configuration, for SAN-based backup of the branch from the data center, a minimal amount of effort is still required to ensure the boot volumes of the virtualized VMs on VSP within each branch appliance are protected and recoverable. ESG Lab hopes that Riverbed will address this in future releases, so that even in the VSP scenario, an entire VM can be protected at the data center.
• While many backup processes may potentially gain benefit from a Riverbed Steelhead solution with Granite technology, they do so without any awareness of Riverbed’s changes to infrastructure or topology.

The Bigger Truth

Riverbed provides comprehensive WAN optimization solutions, helping organizations share applications and data across global wide-area networks. Riverbed WAN optimization solutions have been proven in the field to give businesses order-of-magnitude increases in the performance and value of their existing IT infrastructure and mission-critical applications, including file sharing, e-mail, backup, document management systems, IT tools, and ERP and CRM solutions.

Riverbed has applied its field-proven WAN optimization technology to provide similar performance gains for SAN-based block data extended to remote offices. Achieving a data reduction of more than 26 to 1 in ESG Lab testing, Steelhead Granite technology not only reduces the amount of bandwidth needed to connect to data center SAN storage, but also provides access to remote users at local speeds, maximizing the productivity of those remote workers. Steelhead EX + Granite also enables organizations to utilize existing investments in data protection hardware and software and secures that data in the data center.

In a truly fluid enterprise, all data and storage resources will be centralized in the data center. When that occurs, organizations will gain the ability to provide desired performance in the branch and the ability to quickly provision systems and storage wherever or whenever they wish. In addition, data protection becomes much easier and more secure for remote offices—it is executed centrally along with all valuable corporate data in the data center.

ESG Lab confirmed, through hands-on testing, that Riverbed’s Steelhead EX + Granite solution is able to reduce remote office network traffic while extending data center SANs with little impact on remote office servers and clients. The solution integrated well with Microsoft Exchange and SharePoint business applications as well as basic file system services and iSCSI block storage, long considered all but impossible to extend over long-distance, low-bandwidth WAN links. Organizations interested in improving the remote user experience while bringing data-center-class performance and protection to their remote offices should seriously consider Riverbed Steelhead EX + Granite.

Appendix

[2] Source: ESG Research Report, 2011 IT Spending Intentions Survey, January 2011.

[3] Source: ESG Research Report, 2011 Remote Office/Branch Office Technology Trends, July 2011.

[4] ESG Research Report, Remote Office/Branch Office Technology Trends, July 2011.

[5] Source: ESG Research Report, 2011 Remote Office/Branch Office Technology Trends, July 2011.

[6] ESG Research Report, Remote Office/Branch Office Technology Trends, July 2011.

Indeed, more organizations are looking to deploy desktop virtualization solutions to deliver a consistent image, complete with all applicable business applications to any device an employee chooses to leverage in order to solve this problem. It enables IT to provide greater levels of flexibility to its employees and yet still maintain control over the data, especially sensitive data, which will still be housed in the centralized data center. However, with most pilots done in a limited data center environment, organizations need to be more aware of the impact this may have on the network, especially if they are planning on deploying it in any remote or branch offices. Like any centralized application, if not done correctly, poor performance and availability issues will lead to user frustration and eventually abandonment, potentially wasting a great deal of money (think about some of the problems with the initial centralized SharePoint deployments, etc.).

In 2012 this may be even more of an issue, as ESG research indicates that, while 25% of respondents to our Remote Office/Branch Office Technology Trends survey indicated that they have already deployed centralized virtual desktop solutions at ROBOs, another 42 % plan to do so over the next 18 months. So now is the time to ensure that your virtual desktop initiative will not be hindered by the WAN. With the proper insight (network management tools) and testing (make sure the POC includes at least one typical ROBO location), problems can be averted. In some cases the WAN link you already have may be adequate, in others, you may need more. In those cases, I would recommend investigating optimizing the WAN link before simply ordering more bandwidth as over time it could save a significant amount from your operational budget. It should be noted that while WAN optimization technology wasn’t originally designed to optimize virtual desktops, many vendors have recognized the need and are rapidly adding support for different desktop virtualization vendors and their specific technologies. The key will be knowing which WAN links need additional support and then matching the appropriate WAN optimization vendors to your desktop virtualization solutions in advance of the formal rollout in order to increase your chances for success.

You can read Bob’s other blog entries at Data Center Continuum.

Unfortunately, this creates a challenge for those that are responsible for managing those networks and more specifically the network’s performance. What I find interesting is that in many recent conversations with network administrators, I was surprised at the number that used the telephone as a network performance management tool. Everyone smiling to themselves knows what I am talking about. What this means is that there are still quite a few organizations that monitor network performance based on very simple metrics -like the number of phone calls complaining about performance. No calls = Good. Lots of calls = Bad. To be fair, they might actually be calling to complain about an application’s performance, but somehow it usually rolls downhill to the network. Even those I spoke to that had tools tended to use them for historical reporting rather than proactive monitoring and management.

There are a lot of things putting stress on corporate networks these days – way more devices requesting access and moving files, greater scale and complexity of the networks, virtual servers demanding more and dynamically shifting, the transition from 1 to 10 GbE, more latency sensitive VoIP, video and desktop virtualization apps, etc. With everything that is going on, now is time for organizations take a more proactive approach to network management.

The key to more effective and proactive management is first getting visibility – more specifically end-to-end visibility- of the entire network. This would include everything from highly virtualized server environments and networked storage environments, to a modest 1 GbE network at a remote office. Once you have created the ability to collect information from all these locations, you can then decide what network tools are required to collect, analyze and deliver meaningful information to the business. If you’re not sure how to do this, check out solutions from vendors like Anue, Gigamon, NetOptics, Virtual Instruments (networked storage) and VSS Monitoring.

So while visions of your networks are dancing through your heads this holiday season, make sure that those visions include a way to have end-to-end visibility. Even better, make a resolution for 2012 to ensure you have complete visibility into your network. That way you can fix problems before they happen and either eliminate the phone calls or just have them forwarded to the party (server/storage, etc.) responsible for the problem!

You can read Bob’s other blog entries at Data Center Continuum.

Why is all this important? Consider the fact that 30% of respondents to an ESG research study indicated that managing latency-sensitive applications such as video and IP telephony was among the biggest networking challenges facing organizations supporting IT requirements for ROBO locations. This issue will only become more problematic as organizations rely more and more upon communication and collaboration technologies such as video and rich content to bridge geographically disparate branches. In addition to adding native stream splitting over HTTP video support,  Riverbed has also included VDI support for Citrix ICA over SSL and earlier this year announced that it would be the first company to join the newly launched Teradici Network Solutions Partner Program with the intent of helping customers better address IT infrastructure performance, including VDI for VMware View environments.

Also, while the majority of application traffic may leverage TCP, there is still a need and market for optimizing UDP traffic, especially in DR environments.  The IPv6 is an important checkbox that will help Riverbed compete in government deals and certain geographies, and ensure a future-proof environment for its customers.

According to ESG research, nearly three-quarters (74%) of organizations manage applications and IT services centrally at a corporate site and deliver them over the WAN to remote and branch office locations (ROBOs) rather than managing them locally. This centralization and consolidation of IT leads to greater data center network traffic and enhanced importance of WAN performance. Unsurprisingly, therefore, WAN performance management (37%) and monitoring/managing WAN traffic (34%) ranked one and two, respectively, as the biggest networking challenges facing organizations supporting IT requirements for ROBO locations. In order to meet this demand, Riverbed has integrated Cascade Shark functionality into the Steelhead appliance, in what may be an industry first, enabling on-demand packet capture analysis, branch monitoring, and troubleshooting, without the need for dedicated probe appliances.

So getting to the Bigger Truth – if your organization has an initiative for Video, VoIP, or VDI, looking to enhance its DR environment and is currently a Riverbed customer, you should really take a look at RiOS 7. Not a Riverbed customer but are embarking on any of those initiatives, you should probably look at how WAN optimization solutions and RiOS 7 could help.

See this ESG Brief for more information.

You can read Bob’s other blog entries at Data Center Continuum.

Fortunately, there is technology available to give remote and branch office users a LAN-like experience over the WAN. It is commonly referred to as WAN optimization technology and has been widely used to help compress traffic and accelerate application performance to remote and branch offices. In fact, according to a recent ESG survey, purchasing WAN optimization technology was one of the top three steps taken to improve application performance at remote sites. Why is it in the top three? That’s pretty simple: it works. Remote users can access applications, transfer files, and get LAN-like performance over the WAN. According to the report, the more remote locations an organization has (50+), the more likely it is to have deployed WAN optimization technology.

As organization rely more on video, deploy desktop virtualization solutions, and look to leverage cloud or SaaS models, the need to accelerate the traffic and ensure acceptable performance will only increase. For more on this topic, check out the ESG Research Brief: WAN Optimization Usage at the Branch Office http://www.enterprisestrategygroup.com/2011/10/wan-optimization-usage-at-remotebranch-offices/ (premium access required). So, while WAN optimization technology won’t be able to send you leftover brownies over the WAN, at least you will get the email about them a lot quicker!

Read more of Bob’s blog entries at Data Center Continuum.

Introduction

Background

When IT organizations were asked about the impact of server virtualization on their storage infrastructure, 35% responded that it has increased their use of SAN-based storage.[1] This can be explained not only by the shear explosion of virtual servers that demand more storage resources, but also the trend towards moving more production resources to virtualized servers. This number will only grow as virtualization continues to move into the data center and support business critical applications.

One of the benefits of virtualization is that it obscures the physical complexity of the underlying hardware from applications. But that abstraction also obscures the causes of performance problems and outages, making it difficult to diagnose and resolve issues as troubleshooting requires input from multiple support teams in IT (applications, operations, server, networking, and storage).

VirtualWisdom

Virtual Instruments delivers a unique solution that provides real-time operational, utilization, and performance monitoring as well as root-cause diagnostics for the Fibre Channel SAN. VirtualWisdom passively mirrors traffic flowing across the SAN to examine an out-of-band copy of the traffic, combining the results with monitoring data gathered from both VMware ESX/ESXi servers and the SAN switches to present a comprehensive set of granular, accurate monitoring data.

As shown in Figure 2, VirtualWisdom deploys multiple data collectors, which Virtual Instruments calls “probes”, both in hardware and software, to collect data from servers to SAN. The combined data is hosted on the VirtualWisdom server and provides a central store for analysis and real-time monitoring of the SAN infrastructure.

The VirtualWisdom probes include the following:

• SAN Performance Probe (ProbeFC8) is an appliance that monitors in real-time every frame header on a fibre channel SAN. Connected to the SAN via a SANInsight Traffic Access Point (TAP), the probe measures end-to-end SCSI performance and latency as well as errors at all layers of the FC stack: link, port, and device utilization.
• SAN Availability Probe (ProbeSW) is software that resides on the VirtualWisdom server and collects monitoring data from the SAN switches via SNMP.
• Virtual Server Probe (ProbeVM) is software that also resides on the VirtualWisdom server and collects VMware server information through APIs in vCenter.

Data is collected in a database on the VirtualWisdom server and can be accessed via two client applications. VirtualWisdom Views allows administrators to access SAN metrics in table form, either in real-time or playback mode. It also provides the interface to configure probes for the SAN environment.

The VirtualWisdom Dashboard is a configurable application that allows administrators to monitor multiple metrics in the entire SAN fabric. Each dashboard view can be customized to display metrics from various elements such as virtual servers, switches, and SAN links. Custom dashboards and views can also be created specifically for SAN administrators, server administrators, and application owners.

VirtualWisdom provides a solution that facilitates the following key benefits:

• Virtual infrastructure optimization. Performance metrics at not only the virtual server level, but also at the switch and storage array levels, helps virtual server administrators more accurately identify the causes of performance problems and provide a more intelligent plan for provisioning and load balancing applications deployed on virtual machines.
• SAN optimization. With a comprehensive view of SAN traffic and a historical data trail, SAN and storage administrators can quickly pinpoint the root causes of application service interruptions. This allows them                to quickly determine if the SAN is the root cause of an outage or slow application performance.
• Private cloud optimization. IT organizations looking to move services to a private cloud can use the monitoring and reporting capabilities of VirtualWisdom to help fulfill service level agreements related to performance and availability and minimize the costs associated with providing the high level SLAs required for cloud services.

ESG Lab Validation

ESG Lab performed hands-on evaluation and testing of the VirtualWisdom product line at Virtual Instruments headquarters in San Jose, California. Testing was designed to demonstrate the capabilities of VirtualWisdom in ease of implementation and management, SAN utilization and performance optimization, and real-time monitoring and analysis.

Getting Started – Ease of Use

The test bed used by ESG Lab is illustrated in Figure 3. A VirtualWisdom SAN Performance Probe, Model ProbeFC8, was attached to a SANInsight Fibre Channel Traffic Access Point (TAP) to receive out-of-band SAN traffic for continuous monitoring and reporting. The data was sent to the VirtualWisdom server every second for collection and analysis. SAN Availability Probe and Virtual Server Probe software were installed on the VirtualWisdom server to monitor the Fibre Channel switch fabric and virtual server environment.

ESG Lab Testing

Installation of the server software is wizard-driven and designed to install and configure all software components including the VirtualWisdom server, software probes, and the VirtualWisdom database. ESG Lab was able use the wizard to install the software in only ten steps.

Once the server software was installed, ESG Lab used two programs, Views and Dashboard, to configure the SAN monitoring environment and examine SAN traffic metrics. Using the Views program, ESG Lab was able to see all available probes. As shown in Figure 4, ESG Lab leased (accessed) a probe for use with the VirtualWisdom server simply by dragging and dropping the probe name into the VirtualWisdom database.

SAN Performance Probes are visible immediately in the portal view. SAN Availability Probes for switches are configured using SNMP and need credentials to make the SNMP queries for each switch. Virtual Server Probes use vCenter APIs and require administrator credentials to communicate with them.

As shown in Figure 5, ESG Lab was able to successfully configure multiple SAN Availability Probes to access switches in the SAN fabric using the Switch Configuration view under the Fabric Management setup.

Once the probes were leased and configured as part of the VirtualWisdom server, ESG Lab was able to navigate to the Fabric Management section and examine the logical topology of the SAN fabric. Figure 6 shows the logical fabric from switches to storage controllers and configured LUNs.

Since the devices in the view are editable, ESG Lab was able to create nicknames to identify devices more easily.

Probes collecting statistics at the SCSI and physical link level sent data to the VirtualWisdom server every minute. Using the Views application, ESG Lab configured several tables to examine link and SCSI traffic. In Figure 7, ESG Lab configured a table view to show real-time link data statistics. Four ports on the probe were utilized for testing, and the table shows statistics for events such as link and SCSI frames/sec, loss of sync, loss of signal, and reset events.

SAN Utilization and Performance Optimization

Visibility into performance metrics for a SAN is a crucial element in planning and responding to utilization and performance issues in a virtual server environment. Using the VirtualWisdom Dashboard, administrators can not only identify virtual servers with overutilization issues, they can also quickly assess the traffic load placed on the SAN infrastructure to discover storage that can absorb additional virtual server resources.

ESG Lab Testing

ESG Lab tested the performance of the virtual server environment by first creating a widget, which is a Dashboard component, to measure the CPU utilization of two virtual machines on two separate storage LUNs. As shown in Figure 8, virtual server esx4-cl1a exceeding the threshold for CPU utilization with an average utilization of 84.68%, while esx4-cl1b was within acceptable limits utilization at 30.76%. ESG Lab used vMotion to move a virtual machine, ESX4-VMotionTestVM-XP, to another virtual server on another storage LUN to alleviate the high CPU utilization.

Mapping the two virtual servers back to the storage LUNs (LUN 2 and LUN 33), ESG Lab created a widget that monitored the MB per second as vMotion was used to move the virtual image. As Figure 9 shows, spikes in MB/sec can be seen on LUN 33 as vMotion was started.

ESG Lab examined the average CPU usage on both esx4-cl1a and esx4-cl1b and found both were to be under the acceptable threshold at 57.23% and 30.99%, respectively, after a successful move of virtual image. Figure 10 shows the new metrics for both virtual machines.

Real-time Reporting and Analysis

VirtualWisdom Dashboard is an application that provides a real-time summary of virtual and physical infrastructure, including metrics collected from the three types of probes. Bringing this data together into a consolidated view allows administrators to quickly identify and isolate performance bottlenecks Fibre Channel traffic errors related to a LUN, HBA, server, or application.

ESG Lab Testing

ESG Lab examined the dashboard application which provides a comprehensive view of performance metrics for the SAN environment and found numerous data points associated with traffic originating from the virtual servers to the storage LUNs. The dashboard is preconfigured with multiple tabs that summarize all three probe types and trends for the SAN Performance Probes. ESG Lab was able to configure widgets to examine multiple metrics for the SAN, switches, and virtual machines, and set thresholds to determine where potential performance problems occurred.

As shown in Figure 11, ESG Lab was able to observe the top LUNs for Read Exchange Completion Time (ECT), showing LUNs that are exceeding the acceptable threshold even if virtual servers associated with those LUNs didn’t show unacceptable latencies. In addition, ESG Lab examined data that collected other metrics such as reservation conflicts, maximum pending exchanges on SAN links and HBAs, and top read and write ECTs on HBAs.

Next, ESG Lab examined the performance trends for the fibre channel switches, shown in Figure 12. ESG Lab was able to monitor not only HBAs with the highest reads and writes, but the same traffic on ISL and SAN ports.

Using the metrics in the switch summary page, ESG Lab was able to quickly identify a switch, QAVM_B300_2, with IO traffic above the acceptable threshold. ESG Lab was also able to examine errors in physical links and SCSI traffic including loss of sync, loss of signal, high discards, and link resets at the HBA, switch, and SAN ports.

ESG Lab created a custom tab in the dashboard application that examined performance of the SAN infrastructure from an application point of view. Using SAP as an example, ESG Lab was able to see metrics on the server such as average CPU and memory utilization. In addition, ESG Lab measured buffer to buffer credits and frames/sec on the switch attached to the virtual servers. On the storage side, ESG Lab saw multiple metrics from MB/sec on the ports to errors such as aborts and busy. Using a custom view, as seen in Figure 13, ESG Lab was able to gather a wide variety of metrics focused only on the portion of the SAN fabric serving the SAP application.

ESG Lab Validation Highlights

• Installation and configuration of hardware and software probes was quick and easy, and ESG Lab was monitoring SAN traffic in minutes.
• Using the Views application, ESG Lab was able to monitor physical link and SCSI metrics starting from the virtual servers and ending at the storage LUNs.
• Identifying and remediating high utilization on virtual servers was simple and easy. With a view of underutilized storage, virtual images could be moved to spread processing load without adding new storage resources.
• The VirtualWisdom Dashboard provided a holistic view of a SAN environment and presented comprehensive metrics that allowed administrators to see end-to-end traffic and identify the causes of high latency.

Issues to Consider

• TAP installation is the only disruptive step of implementing VirtualWisdom, as the live links must be interrupted while the TAPs are inserted. Multi-path capabilities for failover can help mitigate this interruption.  However, best practice would be to plan for insertion of TAPs at the same time as patch panel installs, as new switches and new storage is deployed, or during normal maintenance cycles.
• Configuration of dashboard summaries that target specific metrics takes a significant amount of training. Professional services could be required to help customers best understand all the configuration requirements and how to collect and present the data desired.

The Bigger Truth

Investments in storage infrastructures have experienced tremendous growth over the past few years, and the pace of that growth is likely to accelerate. ESG research found that half of all IT organizations plan to spend more on storage infrastructure over the next 18 months.[2] One of the main drivers behind this emphasis on storage is the adoption of server virtualization. Where once virtualization was confined to development environments and light processing servers, it has moved into the data center and become a serious consideration for mission critical applications along with services in the cloud.

The increased demands on virtualization have a ripple effect on other parts of the infrastructure, and storage is no exception. The solution, however, can be costly as companies pour more capital resources into storage investments to accommodate the data requirements created by virtualization. In fact, 36% of IT organizations cite the capital cost of new storage infrastructure as their primary challenge relating to virtualization.[3]

Companies that find themselves in a never-ending catch up game of acquiring storage to meet virtualization demands can alleviate those budgetary drains with the SAN instrumentation solution provided by VirtualWisdom. Understanding the end-to-end performance of a SAN environment can help companies not only better utilize existing resources and plan more effectively for storage growth, but help mitigate risks from outages that can impact the business.

ESG Lab examined all the components provided by VirtualWisdom, and found a comprehensive solution that was not only easy to implement, but provided a unique end-to-end view of a SAN infrastructure. The wealth of data elements captured and collected in easy to read views was impressive, all while creating no performance impact on the SAN environment itself.

Bringing all the elements of a SAN environment together (link monitoring, switch monitoring, virtual server monitoring), Virtual Instruments has built a complete SAN monitoring solution that effectively provides the visibility needed to proactively and intelligently manage SAN resources. Application owners, virtualization administrators, and storage administrators can all benefit from the holistic approach to performance optimization provided by Virtual Instruments, and return real capital savings back to the business.

Appendix

[2] Source: ESG Research Report, 2011 IT Spending Intentions Survey, January 2011.

[3] Source: ESG Research Report, The Evolution of Server Virtualization, November 2010.

Judging by the progress in this area, centralization appears extremely successful but alas, one especially difficult technology remains anchored to branch and remote offices–PCs. This is a real problem since PC management and security has always been a “one step up, two steps back” proposition. Remote PC security alone remains a bear. In a recent ESG Research report, IT professionals were asked to define their biggest information security issues around remote/branch office support. Managing remote PC security and configurations topped the list of challenges. Obviously, persistent PC security problems remain.

So what are large organizations doing to address this? You may be surprised to learn that a growing number of firms are ready to can the whole PC enchilada and replace physical remote PC configurations with desktop virtualization alternatives. In fact, 25% of organizations are already using desktop virtualization technologies to serve remote/branch workers, 22% plan to do so in the next 12 months, and 20% plan to do so in the next 24 months.

This makes a ton of sense–standard desktop images, centralized configuration and patch management, server-based storage of sensitive PC data, etc. I can see desktop virtualization gaining momentum in the future as Bring Your Own Device (BYOD) policies evolve further. What else does change mean? Here are a few thoughts:

1. Desktop virtualization represents a new use case for WAN optimization vendors. The primary job of WAN optimization controllers is accelerating Sharepoint, Exchange, and file access. Lots of vendors do this pretty well but desktop virtualization requires new protocol support and may open the market for new equipment or new vendors. Citrix is keenly aware of this and is already tightly coupling XenDesktop (and XenApp) with NetScaler.
2. Virtual desktops may not come from corporate HQ. My guess is that many firms will look at the transition from physical to virtual desktops and at least investigate SaaS provider options as an alternative to owning all of the servers, storage, networking equipment, etc. This may mean that branch offices become dual-homed with WAN connections to corporate data centers and direct Internet connections for cloud connectivity. This could be a profound change to typical branch office networks.
3. Virtualization will likely spread to tablets, smart phones, and remote workers. If virtualization can ease endpoint management and improve security, why stop at remote office Windows PCs? More and more endpoints will simply render graphics over the network rather than receive software updates and store sensitive data on local memory and disk.

You can read Jon’s other blog entries at Insecure About Security.

These additions make sense, especially considering Riverbed’s relationship with Akamai. Aside from accelerating traffic between Akamai sites, these new asymmetric acceleration solutions could prove to be very useful in getting content to end-users. Riverbed’s positioning on only targeting the high growth virtual ADC or vADC markets, however, doesn’t resonate with me as much. Both F5 and Citrix offer virtual editions of their products and they are already well entrenched in the ADC space, so I believe that Riverbed will have its work cut out for it and may have to focus on more than just the virtualization angle to gain significant market share. That said, Zeus brings 1,500 customers with it, so it’s not like Riverbed is starting from scratch either. Aptimize provides some pretty slick first time viewed optimization benefits, overcoming the initial caching delay. Riverbed explains the acquisitions simply as Fast = Steelhead, Faster = Steelhead & Zeus, and Fastest = Steelhead, Zeus, & Aptimize. Cascade will provide visibility across the entire environment.

Riverbed may have an unique advantage–combining these new products with Cascade and Steelhead could present an integrated application performance solution. It is interesting to note that Riverbed has decided to keep Zeus as its own business unit. While this will keep it very focused on vADC (earning a healthy payout), it also has the potential to hinder integration efforts. Riverbed did articulate that, while an integrated solution was possible in the future, it wanted each company’s capability to be able to stand on its own to better fit the needs of the customer.

Riverbed will also need to focus on educating, training, and certifying its sales force and channel partners on these new offerings. More importantly, they need to know where in the enterprise they should be selling this to (application teams, not the networking guys). The Zeus sales force should help, but in order to really have an impact, Riverbed will need to scale. That, however, is something I am pretty sure it knows how to do.

Read more of Bob’s blog entries at Data Center Continuum.