The Challenges

The use of server virtualization technology is on the rise among organizations of all sizes and in all industries around the world.  In a recent ESG survey of current and planned users of the technology, 52% of organizations had already deployed, while 48% plan to do so.[1] Given the impressive economic benefits of server virtualization, the glut of affordable and under-utilized processing power, and growing power and cooling issues in the data center, ESG predicts that the brisk adoption of server virtualization will continue for the foreseeable future.

ESG research indicates that the vast majority (87%) of organizations that have deployed server virtualization have done so in conjunction with networked storage.  Compared to islands of direct attached hard drives, utilization is greatly increased when applications share a pool of networked storage. Applications deployed on virtual machines sharing a pool of storage are more mobile and available than those deployed on direct attached hard drives.

While the benefits of server virtualization and networked storage are clearly compelling, IT managers are faced with a number of challenges as they try to manage a consolidated mix of real-world applications running on a virtualized infrastructure.  As shown in Figure 1, the top two concerns are performance and a general lack of information and best practices.  This holds true across organizations of all sizes, regardless of the number of virtual servers deployed.  That users would be so concerned with the performance of their infrastructures makes sense given the fact that 46% of virtualization users report that they currently run “Tier 1” applications on virtual machines and 33% plan to in the future.

The Solution

This ESG Lab report examines the performance of real-world application workloads running in a virtualized and consolidated IT environment that leverages the following technologies:

• IBM System Storage DS5020 Express and DS3950 Express storage systems: With high performance that is optimized for mixed workloads, the DS5020 Express and the DS3950 Express were designed for modular scalability (capacity and/or performance), high availability, and advanced functionality including copy services and remote replication.
• IBM System x3950 M2 servers: The IBM System x3950 M2 is a rack-optimized server with extraordinary performance and scalability that is ideally suited for virtualized environments.
• QLogic 8 Gb FC Dual-port HBA for IBM System x: The QLogic 8 Gb Dual-port HBA for IBM System x is designed to deliver sustained high throughput and reliability in highly available virtual environments.

• VMware vSphere: Building on the power of VMware Infrastructure, VMware vSphere transforms IT infrastructures into a private cloud which enables the automated delivery of IT infrastructure as a service.

The capabilities of the IBM servers and storage that were used during this evaluation are summarized in Figure 2. The IBM System x3950 M2 supports up to 96 Intel Xeon 7400 processor cores, 1 TB of DDR-2 memory and 28 PCI-Express x8 expansion slots. The IBM System Storage DS5020 Express and the DS3950 Express support up to 112 drives (FC, SATA, or mixed configurations) and are equipped with up to 4 GB of cache and 8 GB/sec of internal bandwidth.  The DS3950 Express is a variant of the DS5020 Express that is available in two pre-configured models.  The DS5020 Express can be custom configured and adds support for full disk encryption (FDE).  The DS5020 Express supports up to eight FC host interfaces and the DS3950 Express supports up to four FC host interfaces.

The Results

This report examines the performance capabilities of IBM System Storage DS5020 Express and DS3950 Express storage systems running a mix of real-world applications in a VMware vSphere-enabled virtual server environment powered by a pair of IBM System x3950 M2 servers.   In particular, this report explores how:

• An eight socket System x3950 M2 achieved an excellent VMmark mixed workload score of 33.85@24 Tiles.
• A single DS5020 attached to a pair of System x3950 M2 servers running a mix of real-world application workloads in 16 virtual machines supports up to:
  • 8,252 mailboxes using the Microsoft Exchange Jetstress utility
  • and 3,580 small database IOs per second using the Oracle Orion utility
  • and 468 MB/sec of throughput for large OLAP Oracle Orion operations
  • and 3,324 simulated web server IOPs
  • and 366 MB/sec of throughput for simulated backup/scan/index jobs
  • with the predictably fast response times and scalability
  • Within a vSphere enabled infrastructure, the DS5020 Express achieved a maximum aggregate throughput of 3.1 GB/sec during bandwidth intensive throughput testing and 1.12 GB/sec during mixed application workload testing.

The predictably fast, mixed workload performance scalability of the virtualized environment tested by ESG Lab is summarized in Figure 3.   The results will be explored in detail later in this report, but for now it should be noted that the performance of the DS5020 Express scaled extremely well as a mix of real-world application workloads ran in parallel on up to 16 virtual machines.

The balance of this report explores how mixed workload testing was accomplished, what the results mean, and why they matter to your business.

ESG Lab Validation

The real-world performance capabilities of the IBM DS5020 Express were assessed by ESG Lab at an IBM facility located in Tucson, Arizona.   The methodology presented in this report was designed to assess the performance capabilities of a single IBM DS5020 Express storage system shared by multiple virtual servers running a mix of real-world application workloads.  The cooperation of VMware, IBM and QLogic was key to the success of this project.  In particular, this project benefitted from VMware’s expertise in helping customers plan for the deployment of business-critical applications in virtual server environments and IBM’s long heritage of success in the modular storage systems market.

VMmark

Conventional server benchmarks were designed to measure the performance of a single application running on a single operating system inside a single physical computer. SPEC CPU2000 and CPU2006 are well known examples of this type of server benchmarking tool. Much like traditional server benchmarks, conventional storage system benchmarks were designed to measure the performance of a single storage system running a single application workload.  The SPC-1 benchmark, developed and managed by the Storage Performance Council with IBM playing a key role, is a great example. SPC-1 was designed to assess the performance capabilities of a single storage system as it services an online interactive database application.

Traditional benchmarks running a single application workload can’t help IT managers understand what happens when a mix of applications is deployed together in a virtual server environment. To overcome these limitations, VMware created a mixed workload benchmark called VMmark.  VMmark uses a tile-based scheme for measuring application performance and provides a consistent methodology that captures both the overall scalability and individual application performance of a virtual server solution. As shown in Figure 4, compared to a traditional benchmark, which tests a single application running on a single physical server, VMmark measures performance as a mix of application workloads is run in parallel within virtual machines deployed on the same physical server.

The novel VMmark tile concept is simple, yet elegant.  A tile is defined as a mix of industry standard benchmarks that emulate common business applications (e.g., e-mail, database, web server).   The number of tiles running on a single machine is increased until the server runs out of performance.  A score is derived so that IT managers can compare servers with a focus on their performance capabilities when running virtualized applications.

The IBM System x3950 M2 used during this ESG Lab Validation has an excellent published VMmark score of 33.85@ 24 tiles.[2]At a high level, this means that the IBM System x3950 M2 did 33.85 times more work than the dual processor, single core server that VMware used as a reference when VMmark was first released in 2007.

A Mixed Real-world Storage Benchmark Methodology

While VMmark is well suited for understanding the performance of a mix of application running on a single server, it was not designed to assess what happens when a mix of applications are run on multiple servers  sharing a single storage system.  VMmark tends to stress server internals more than it does the storage system. The methodology presented in the balance of this report was designed to stress the storage system more than the servers. Taking a cue from the VMmark methodology, a tile-based concept was used.  As shown in Figure 5, each tile is composed of a mixture of four application workloads.  Two physical servers, each configured with eight virtual machines, were used to measure performance as the number of active tiles was increased from one to four.

The difference between the server-focused VMmark benchmarking and storage-focused ESG Lab benchmarking is shown in Figure 6.  Note how VMmark testing is performed with a single server, often attached to multiple storage systems. As a matter of fact, the IBM System x3950 M2 VMmark results presented earlier in this report were achieved with a pair of IBM System Storage DS4700 arrays.[3] In other words, when vendors publish VMmark results, they make sure there is plenty of storage available so they can record the highest VMmark server score.  This provides IT managers with a fair comparison of the performance capabilities of competitive server technologies.

ESG Lab storage-focused benchmarking uses a different approach. Instead of testing with a single server and more than enough storage, multiple servers are attached to a single storage system.  Rather than running application level benchmarks which stress the CPU and memory of the server, lower level industry standard benchmarks are used with a goal of measuring the maximum mixed workload capabilities of a single storage system.

Mixed Workloads

Industry standard benchmarks were used to emulate the IO activity of four common business application workloads:

• E-Mail: The Microsoft Jetstress utility was used to generate e-mail traffic.  Similar to the Microsoft LoadSimm utility used in the VMmark benchmark, Jetstress simulates the activity of typical Microsoft Exchange users as they send and read e-mails, make appointments, and manage to-do lists.  The Jetstress utility is, however, a more lightweight utility than LoadSimm. Using the underlying Jet Engine database, Jetstress was designed to focus on storage performance.
• Database: The Orion utility from Oracle was used to generate database traffic. Much like Jetstress, Orion is a lightweight tool that is ideally suited for measuring storage performance.  Orion was designed to help administrators understand the performance capabilities of a storage system, either to uncover performance issues or to size a new database installation without having to create and run an Oracle database. Orion is typically used to measure two types of database activity: response-time sensitive online transaction processing (OLTP) and bandwidth sensitive online analytic processing (OLAP).
• Web Server: The industry standard Iometer utility was used to generate web server traffic. The IO definition was composed of random reads of various block sizes.  The web server Iometer profile used for this test was originally distributed by Intel, the author of Iometer. Iometer has since become an open source project.[4] Iometer tests were performed on Windows physical drives running over VMware raw mapped devices.
• Scan/read: The Iometer utility was used to generate a single stream of read traffic.  Operations that tend to generate this type of large block sequential traffic include scan and index operations, long running database queries, backup operations, bulk data uploads, and copies. One 256 KB sequential read workload was included in each tile to add a throughput intensive component to the predominantly random IO profile of interactive e-mail, database, and web server applications.  As most experienced database and storage administrators have learned, a throughput intensive burst in IO traffic can drag down the performance for interactive applications, causing performance problems for end-users.  Adding a few streams of throughput intensive scan/read traffic was used to determine whether interactive performance would remain predictably responsive as the amount of mixed IO utilization increased.

Each of the four workloads ran in parallel, with the Jetstress e-mail test taking the longest to complete (approximately three hours).   The settings for each of the industry standard benchmarks are documented in the appendix.

Test Bed

VMware vSphere version 4.0 was installed on a pair of powerful IBM System x3950 M2 servers, each with four six-core processors and a pair of QLogic 8 Gb Fibre Channel host bus adapters.  A DS5020 Express with 112 15K RPM FC drives was connected to the servers through QLogic dual port 8 Gb host bus adapters and a FC switch as shown in Figure 7.

Drive Layout

The DS5020 drive configuration is summarized in Table 1. Four Exchange database volumes were configured. Each of the Exchange database volumes was configured with an eight drive RAID-10 database volume and a four drive RAID-10 log volume.   The Oracle, web server and scan/read workloads ran against four drive RAID-10 volumes. The operating system volumes (Vmdk/OS) were configured using a 3+1 RAID-5 layout.   Volume ownership was balanced across the dual controllers within the DS5020 Express and distributed evenly over the eight host interfaces.   The volumes were spread evenly over two VMware host groups with a multipath policy of most recently used (MRU). [5]

Configuring Virtual Machines

Each of the Microsoft Exchange machines was configured with four virtual CPU cores, 32 GB of RAM, a virtual disk over VMFS for the operating system, and two mapped raw LUNs.  DS5020 Express disk capacity was used for all storage capacity including VMware virtual disk files (VMDK), Windows Server 2008 operating system images, application executables, and application data.   All of the application data volumes under test were configured as mapped raw LUNs (also known as raw device mapped, or RDM volumes).  The configuration of one of the four virtual machines used for e-mail testing is shown in Figure 8. Note how three hard disks have been configured: one virtual disk for the operating system and two mapped raw LUNs for the Exchange database and Logs.

The Results

In a way, storage system benchmark testing is like an analysis of the performance of a car. Specifications including horsepower and acceleration from 0 to 60 are a good first pass indicator of a car’s performance.  But while specifications provide a good starting point, there are a variety of other factors that should be taken into consideration including the condition of the road, the skill of the driver, and gas mileage ratings.  Much like buying a car, a test drive with real-world application traffic is the best way to determine how a storage system will perform in real-world conditions.

Characterization

Performance analysis began with an examination of the low level aggregate throughput capabilities of the test bed.  This testing was performed using the Iometer utility running within the eight virtual machines that were used later during mixed workload testing.  The eight virtual machines accessed DS5020 Express storage through eight 8 Gbps FC interfaces.

Iometer access definitions, which measured the maximum throughput from disk, were used for this first pass analysis of the underlying capabilities of the DS5020 Express.[6] Similar to a dynamometer horsepower rating for a car, maximum throughput was used to quantify the power of a turbo-charged DS5020 Express storage engine. As shown in Figure 9, ESG Lab recorded a maximum throughput of 3.1 GB/sec.

What the Numbers Mean

• Much like the horsepower rating of a car, the throughput rating of a storage system is a good indicator of the power of a storage system’s engine.
• Storage throughput is a measure of the bandwidth available to the system. Throughput can be measured on a stream or aggregate basis.  A stream is represented by one application or user communicating through one IO interface to one device.  Aggregate throughput is a measure of how much data the storage system can move on a whole for all applications and users.
• ESG Lab throughput characterization was performed using the industry standard Iometer utility as 32 streams performed large sequential reads from eight logical devices through eight FC interfaces.[7]
• ESG Lab recorded a peak aggregate throughput of 3.1 GB/sec in a VMware vSphere environment.
• Forty-two percent of the throughput was delivered from DS5020 Express cache; the balance was serviced from disk.
• When comparing the performance capabilities of two servers in a virtual server environment, the server with more cache tends to perform better.    ESG Lab is confident that a similar pattern holds true for storage systems.  A storage system with more cache—and better caching algorithms—should perform better in a virtual server environment.
• ESG Lab characterization testing indicates that the DS5020 Express has more than enough cache and front end bandwidth to meet the needs of virtualized applications requiring up to 112 disk drives for capacity.
• ESG Lab is convinced that the patented caching algorithms of the DS5020 Express provide a significant performance boost during mixed application virtualized application testing.

Virtual Machine Utilization

Mixed application testing began with a quick analysis of server memory and CPU utilization to make sure that there were no bottlenecks between virtualized applications and the DS5020 Express.  Memory and CPU utilization as reported by the VMware Infrastructure Manager are shown in Figure 10.

These screenshots were taken during the peak activity phase of the four tile test.  With memory and CPU utilization at less than 10%, there was no obvious bottleneck between virtualized applications and the DS5020 Express.

Mixed Real-world IOPS Scalability

I/Os per second, or IOPS, is a measure of the number of operations that a storage system can perform in parallel. When a system is able to move a lot of IOPS—from disk and from cache— it will tend to be able to service more applications and users in parallel. Much like the horsepower rating for a car engine, the IOPS rating for a storage controller can be used as an indicator of the power of a storage system engine.

While IOPS out of a cache is typically a big number and can provide an indication of the speed of the front end of a storage controller, IOPS from disk is a more useful metric when determining the real-world performance of a storage system servicing a mix of business applications.  For example, e-mail and interactive database applications tend to be random in nature and therefore benefit from good IOPS from disk.  With that said, a mix of real-world applications tends to have random and sequential IO traffic patterns that may be serviced from disk or from cache.

ESG Lab measured IOPS performance as reported by the DS5020 Express as the number of virtual machines running mixed real-world application workloads was increased from four through sixteen.  With a mix of random and sequential IO over 112 disk drives, the goal was not to record a big IOPS number; the goal with this exercise was an assessment of the scalability of the DS5020 Express as an increasing number of applications are consolidated onto a single virtualized platform.  The IOPS scalability during the peak period of mixed workload activity is shown in Figure 11.

What the Numbers Mean

• IOPS varied throughout the mixed workload test with peaks occurring during the Orion small IOPs phase and towards the end as the Jetstress utility performed a database consistency check.
• A peak of 18,963 and a steady state of 13,218 IOPS were recorded during the four tile run.
• IOPS scaled in a near-linear fashion as mixed real-world application traffic increased from four through sixteen virtual servers.

Handling Throughput Spikes with Ease

As noticed during IOPS monitoring, peaks of throughput activity could be correlated to the periodic behavior of real-world applications. Two bursts of aggregate throughput were observed: the first during the Oracle large MBPS test, which simulates a throughput intensive OLAP application, and the second during the Jetstress database consistency check. A VMware vSphere view of mixed workload performance on one of the servers is shown in Figure 12.

What the Numbers Mean

• An aggregate throughput level of 1.12 GB/sec was recorded as mixed, real-world applications were run on 16 virtual machines sharing a single DS5020 Express storage system (560 MB/sec for one of the two physical servers is shown in Figure 12).
• As throughput intensified during the Oracle Orion OLAP test phase, bandwidth utilization for other mixed workloads operating in parallel remained steady.

Mixed Application Performance Scalability

Having looked at the IOPS and throughput ratings of the turbo-charged DS5020 Express  engine, here’s where the rubber meets the road as we examine performance at the application level. The output from each of the industry standard benchmark utilities was analyzed to determine the performance scalability and responsiveness of real-world applications running in a consolidated virtual environment.

Microsoft Exchange

The Microsoft Jetstress tool was used to see how many simulated e-mail users could be supported by the DS5020 Express resources allocated for the Exchange application.  The number of IOPS and response time for each database and log volume was recorded at the end of each Jetstress run.   A response time goal of 20 milliseconds or less for DB reads and 5 milliseconds or less for log writes is required to pass the test.  These values are defined by Microsoft as a limit beyond which end-users will feel that their e-mail system is acting slowly.

ESG used the following IBM guidelines from an IBM report describing the results of an IBM System Storage DS4800 Mailbox Jetstress Analysis report to interpret the results:

In an enterprise Exchange 2007 environment, performance is usually designed around a 0.5 IOPS user profile, which is equivalent to a very heavy Exchange user. While disk performance varies, generally you should calculate based on a one hundred IOPS per disk metric, which is a conservative starting point, and tune from there for your specific environment.[8]

Microsoft Jetstress logs were used to determine the number of IOPS and response times as the number of active virtual machines was increased from four through sixteen.[9] Based on a 0.5 IOPS user profile, the number of IOPS was used to calculate the number of supported Exchange users.  The number of supported mailboxes as the number of virtual machines was increased from four to sixteen is shown in Figure 13, Figure 14, and Table 2.

What the Numbers Mean

• The single tile mixed application test supported 2,358 Exchange users with an average DB disk response time of 14 milliseconds.
• Performance scaled in a near-linear fashion to 8,254 users while the DS5020 Express was busy processing and servicing other applications concurrently.
• As the number of simulated e-mail users was increased, the DS5020 Express provided excellent response times that are well within Microsoft’s guidelines.  For example, the Microsoft guideline for a database read volume is 20 milliseconds as shown by the dotted line in Figure 14.
• The four tile test, which produced 4,127 IOPS over 32 database drives, delivered 129 IOPS per drive—well above the conservative IBM guideline of 100 IOPS per drive.

Oracle Orion

The Oracle Orion utility was used to measure small transfer (8 KB) IOPS and response time and large transfer (1 MB) throughput.  The small results are used to predict the performance and scalability of response time sensitive interactive database applications (e.g., OLTP). The large results are used to predict the performance of throughput intensive database mining and decision support systems (DSS).

ESG used the following guidelines from presentations presented at Oracle OpenWorld in November 2007 to interpret the results:

Target 5-10 millisecond for response time critical IO. Start by assuming 30 IOPS per disk for OLTP and 20 MB/sec per disk in DSS. This is way below the theoretical value but allows for media repair etc.[10]

For new or non-existing applications, use business rules or data model transaction profiles flow to understand “what is a transaction,” and then extrapolate for transactions per second or hour. Optionally you can use the numbers we have seen in our consulting gigs. Note that these are just guideline values. Use the following as basic guidelines for OLTP:

Low transaction system – 1,000 IOPS or 200 MBytes/s

Medium transaction system – 5,000 IOPS or 600 Mbytes/s

High-end transaction system – 10,000 IOPS or 1 Gbyte/s <- almost rarely achievable and usually TPC-C type workloads[11]

The results for the four tile Orion test are summarized in Table 3. A sample Orion report is shown in the Appendix.

What the Numbers Mean

• The four tile test achieved a grand total of 3,580 small IOPS and 468 large MBPS while the system was simultaneously running a mix of real-world application workloads.
• Using Oracle’s back of the envelope sizing guidelines, this level of IO activity is significantly higher than a  typical “low transaction system” and nearly represents a “medium transaction system.”
• The total number of small IOPS processed during the busy four tile test yielded an excellent rate of 224 small IOPS per drive, which dwarfs the extremely conservative Oracle planning guideline of 30 IOPS per drive.
• Orion reported an average latency of 5.44 milliseconds for the small IOPs workload.  Given the Oracle guidance of 5 to 10 milliseconds, ESG Lab believes that these are excellent results—especially given the mix of IO intensive workloads that were being serviced by the DS5020 Express in parallel.

Web Server and Scan/Read

Performance results as reported by the Iometer utility for the web server and scan/read workloads executing within virtual machines during the four tile test are listed in Table 4.

What the Numbers Mean

• Given the cache friendly, read-only nature of web server IO traffic, ESG Lab believes that these results indicate that the DS5020 Express has the horsepower required to service tens of thousands of simultaneous page requests.
• ESG Lab believes that a file system workload would produce results that are approximately similar to the web server workload used for this test.
• Each of the four scan/read streams sustained at least 87 MB/sec of throughput for the entire duration of the mixed workload test.  A stream of this magnitude could service the data needs of a number of simultaneous backup streams, a very aggressive scan and index job, or a throughput intensive database table scan—with no perceivable performance impact on applications that are running parallel.

Much like the electrical system in your home, figuring out how many appliances you can run in parallel before blowing a fuse is not a function of the number of wires behind the walls. What matters more is the design of the circuits used to distribute the right amount of power to appliances when needed.  ESG Lab testing indicates that the DS5020 Express engine delivers the right amount of power to virtualized applications when needed.

DS3950 Express Performance Analysis

The DS3950 Express supports up to four FC host interfaces as compared to the DS5020 Express, which supports up to eight FC host interfaces. Otherwise, the components and architecture of the DS3950 Express are exactly the same as the DS5020 Express. ESG Lab tested the DS5020 Express with only four active FC host interfaces connected to the IBM System x3950 M2 servers with a goal of analyzing the performance difference between the DS5020 Express and the DS3950 Express. The results are shown in Figure 15 and Table 5.

What the Numbers Mean

• The simulated DS3950 Express with four active host paths delivered 78.82% of the throughput and 92.17% of the IOPS from cache compared to the DS5020 Express with eight active paths.
• The mostly random ESG Lab mixed workload performed roughly the same.  This is due to the fact that the most important performance consideration for mostly random business application workloads is the number of disk drives operating in parallel. In this case, the same number of drives was tested (112).

ESG Lab Validation Highlights

• 3.1 GB/sec of aggregate throughput was sustained during characterization testing within a VMware vSphere environment.
• Mixed real-world application workloads running simultaneously within sixteen virtual machines deployed over two IBM System x3950 M2 servers provided the performance needed to concurrently support:
  • 8,252 mailboxes using the Microsoft Exchange Jetstress utility
  • and 3,580 small database IOs per second using the Oracle Orion utility
  • and 468 MB/sec of throughput for large OLAP Oracle Orion operations
  • and 3,324 simulated web server IOPs
  • and 366 MB/sec of throughput for simulated backup/scan/index jobs
  • with the predictably fast response times and scalability
• Excellent IOPS per drive were recorded (e.g., 224 for the Oracle OLTP test).
• As the number of virtual machines sharing a single DS5020 was increased, performance scaled in a near linear fashion with predictably fast response times (16 to 17 millisecond for Jetstress DB reads, 5.35 to 5.55 milliseconds for Oracle Orion small IOPS).
• The DS5020 had horsepower to spare for rebuilds and advanced functions including copy services and remote replication.

Issues to Consider

• Generally accepted best practices and predominantly default VMware and IBM System Storage settings were used during the design of this test.  As expected after any benchmark of this magnitude, deep analysis of the results indicates that tuning would probably yield slighter higher absolute results. Given that the goal of this test was not to generate a big number,  ESG Lab is confident that the results presented in this report meet the objective of estimating performance scalability and responsiveness as a growing number of virtual machines share a consolidated pool of DS5020 Express storage.
• For applications requiring extreme performance beyond that which is provided by FC and SATA drives, ESG Lab believes that the DS5020 Express is an ideal architecture for the selective use of solid state disk (SSD) devices.  While mixed workload testing was not performed with SSD devices,   ESG Lab is confident that SSD devices could be used to improve the performance of highly referenced database indexes and temp files.
• The test results/data presented in this document are based on industry-standard benchmarks deployed together in a controlled environment. Due to the many variables in each production data center environment, it is still important to perform capacity planning and testing in your own environment to validate a storage system configuration.

The Bigger Truth

Server virtualization is being deployed by a growing number of organizations to lower costs, improve resource utilization, provide non-disruptive upgrades, and increase availability. Each benefit is fundamentally enabled by de-coupling servers, applications, and data from specific physical assets.  Storage virtualization takes those very same benefits and extends them from servers to the underlying storage domain—bringing IT organizations one step closer to the ideal of a completely virtualized IT infrastructure.

While the benefits of a completely virtualized infrastructure are obvious to most IT managers, performance is a real concern.   Server, storage, and network administrators are looking for answers to a number of questions:

• Can we meet performance service level agreements for a mix of business-critical applications?
• Does the storage system have the horsepower to serve mixed, real-world applications?
• Can the storage system scale to accommodate future growth and consolidation?

IBM approached ESG Lab with an ambitious goal of answering these questions. A performance benchmark was designed to measure the performance capabilities of a storage system subjected to an IO intensive mix of virtualized business applications. Taking a cue from the VMmark benchmark from VMware, a “tile” concept was used during the design of this test.  Each “tile” was composed of four applications, each running in its own virtual machine.    The server horsepower of a pair of IBM System x3950 M2 servers, with an excellent published VMmark score of 33.85@24 tiles, was used to drive up to four tiles and sixteen virtual applications in parallel.   ESG believes that the results of this storage-focused benchmark complement the excellent server-focused results of the IBM System x3950 M2 VMmark test.

IBM has more than a decade of experience delivering modular FC-attached storage systems designed to meet the cost-optimized performance demands of medium-sized organizations, mid-tier applications, remote departments, and near-line applications. The IBM DS5000 series builds on the heritage of the previous generation DS4000 series disk system with more than 87,000 systems and 511 petabytes shipped to date.  The engine under the hood of DS5000 Series has been turbo-charged to meet the real-world performance demands of virtualized applications.  With twice the host bandwidth and three times the internal bandwidth of the previous generation DS4700, the DS5020 Express is designed to deliver the high performance, low latency, and balanced scalability needed to meet the demanding performance needs of a mix of real-world applications sharing a consolidated infrastructure.

ESG Lab testing began with a confirmation that the DS5020 Express test bed can deliver up to 3.1 GB/sec of raw aggregate throughput in a VMware vSphere environment.  This result was an early indicator that the IBM DS5020 Express has the internal bandwidth and processing power needed to serve a mix of real-world application workloads. The results of the mixed workload tests were even more impressive. A single DS5020 Express simultaneously supported 8,252 simulated Exchange users and 3,580 Oracle Orion small database IOs per second and 468 MB/sec of throughput for large OLAP Oracle Orion operations and 3,324 simulated web server IOPs and 366 MB/sec of throughput for bandwidth intensive streams of read traffic—all while delivering predictably fast response times.  Testing on the DS5020 Express with the same number of drives and less host connections indicates that the IBM System Storage DS3950 Express delivers similar levels of performance for the mix of applications tested by ESG Lab.

ESG Lab is pleased to report that the combination of IBM System Storage DS5020 Express,  IBM System x3950 M2 servers  and QLogic 8 Gb Fibre Channel host bus adapters delivers the performance needed to meet the needs of a mix of real-world business applications running within a VMware vSphere enabled virtual infrastructure.

Appendix

This is an example of the output created by the Jetstress utility. It shows the performance for one of four Jetstress tests running in parallel. Specifically, this report was created by the Jetstress utility running on a virtual machine within the fourth tile of the four tile test.

Microsoft Exchange Server Jetstress

Performance Test Result Report

Test Summary

Database Sizing and Throughput

Jetstress System Parameters

Disk Subsystem Performance

Host System Performance

10/16/2009 1:51:50 PM — Performance logging begins (interval: 30000 ms).
10/16/2009 1:51:50 PM — Verifying database checksums …
10/16/2009 2:55:36 PM — E: (100% processed)
10/16/2009 2:55:36 PM — Performance logging ends.
10/16/2009 2:55:36 PM — C:\Downloads\Jetstress\Logs\DBChecksum_2009_10_16_13_51_50.blg has 127 samples.
10/16/2009 2:55:37 PM — C:\Downloads\Jetstress\Logs\DBChecksum_2009_10_16_13_51_50.html is saved.
10/16/2009 2:55:37 PM — Verifying log checksums …
10/16/2009 2:55:37 PM — F:\ (2 logs passed)
10/16/2009 2:55:37 PM — C:\Downloads\Jetstress\Logs\Performance_2009_10_16_11_49_42.blg has 488 samples.
10/16/2009 2:55:37 PM — Creating test report …
10/16/2009 2:55:40 PM — Volume E: has 0.0150 for Avg. Disk sec/Read.
10/16/2009 2:55:40 PM — Volume F: has 0.0004 for Avg. Disk sec/Write.
10/16/2009 2:55:40 PM — Volume F: has 0.0000 for Avg. Disk sec/Read.
10/16/2009 2:55:40 PM — Test has 0 Maximum Database Page Fault Stalls/sec.
10/16/2009 2:55:40 PM — Test has 0 Database Page Fault Stalls/sec samples higher than 0.
10/16/2009 2:55:40 PM — C:\Downloads\Jetstress\Logs\Performance_2009_10_16_11_49_42.xml has 479 samples queried.

This is an example of the output created by the Oracle Orion utility. It shows the performance for one of eight Orion tests running in parallel. Specifically, this report was created by the Orion utility running on a virtual machine within the fourth tile of the four tile test.

ORION VERSION 10.2.0.1.0

Commandline:

-run advanced -testname vmware -num_disks 5 -size_small 8 -size_large 1024 -type rand -simulate raid0 -write 30 -duration 150 -matrix basic

This maps to this test:

Test: vmware

Small IO size: 8 KB

Large IO size: 1024 KB

IO Types: Small Random IOs, Large Random IOs

Simulated Array Type: RAID 0

Stripe Depth: 1024 KB

Write: 30%

Cache Size: Not Entered

Duration for each Data Point: 150 seconds

Small Columns:,      0

Large Columns:,      0,      1,      2,      3,      4,      5,      6,      7,      8,      9,     10

Total Data Points: 36

Name: \\.\f:           Size: 1924071424

1 FILEs found.

Maximum Large MBPS=108.27 @ Small=0 and Large=9

Maximum Small IOPS=882 @ Small=25 and Large=0

Minimum Small Latency=5.42 @ Small=1 and Large=0

This is an example of the output created by the Iometer utility after a web server test run. This example shows the performance of the four web server tests which ran in parallel during the mixed workload four tile test.

This is an example of the output created by the Iometer utility after a scan/read test run. It shows the performance of the four scan/read tests which ran in parallel during the mixed workload four tile test.

The following excerpts were extracted from the IBM DS5020 Storage System Profile Summary.

PROFILE FOR STORAGE SUBSYSTEM: ESG_DS5020 (Fri Oct 02 05:53:29 PDT 2009)

SUMMARY——————————

Number of controllers:              2

High performance tier controllers:  Enabled

Number of arrays:                   24

RAID 6:                             Enabled

Total number of logical drives used:     25

Number of standard logical drives:    24

Number of access logical drives:      1

Total number of logical drives allowed:  1024

Drive Limit Management:

Number of drive slots discovered:  112

Number of drive slots allowed:     112

FlashCopy Logical Drives:                                 Enabled

Number of flashcopies used:                            0

Number of flashcopies allowed:                         2

Number of flashcopies allowed per base logical drive:  2

Remote Logical Drive Mirroring:  Disabled/Deactivated

Number of mirrors used:       0

Number of mirrors allowed:    0

VolumeCopy:                   Disabled

Number of copies used:     0

Number of copies allowed:  0

Number of drives:           112

Mixed drive types:          Enabled

Current media type(s):      Hard Disk Drive (112)

Current interface type(s):  Fibre (112)

Total hot spare drives:     0

Standby:                 0

In use:                  0

Drive Security:           Disabled

Security key identifier:  None

Storage Partitioning:             Enabled

Number of partitions used:     2

Number of partitions allowed:  128

Number of logical drives allowed per partition:  256

Access logical drive:  LUN 31,31,31 (see Mappings section for details)

Default host OS:       DEFAULT (Host OS index 0)

Current configuration

Firmware version:                    07.60.08.00

NVSRAM version:                      N1814D20R1060V08

EMW version:                         10.60.G5.05

AMW version:                         10.60.G5.05

NVSRAM configured for batteries:          Yes

Start cache flushing at (in percentage):  80

Stop cache flushing at (in percentage):   80

Cache block size (in KB):                 16

Media scan frequency (in days):                Disabled

Failover alert delay (in minutes):             5

Feature enable identifier:                     3030303934203030313235204A93DE36

Feature pack:                                  DS5020 Model 20, 24, 28

Feature pack submodel ID:                      121

Storage Subsystem world-wide identifier (ID):  60080E500017B6BA000000004A93DE34

CONTROLLERS——————————

Number of controllers: 2

Controller in Enclosure 85, Slot A

Status:                      Online

Current configuration

Firmware version:         07.60.08.00

Appware version:       07.60.08.00

Bootware version:      07.60.08.00

NVSRAM version:           N1814D20R1060V08

Replacement part number:     37781-03

Model name:                  4988

Board ID:                    4988

Submodel ID:                 121

Product ID:                  1814      FAStT

Revision:                    1060

Replacement part number:     37781-03

Part number:                 37781-03

Serial number:               SQ91100094

Vendor:                      IBM

Date of manufacture:         June 2, 2009

Trunking supported:          No

Data Cache

Total present:            1709 MB

Total used:               1709 MB

Processor cache:

Total present:            339 MB

Cache Backup Device

Status:                   Optimal

Type:                     USB flash drive

Location:                 Controller A, Connector USB 1

Capacity:                 1,960 MB

Product ID:               eUSB

Part number:              Not Available

Serial number:            200902190239A7D8

Revision level:           8715

Manufacturer:             SMART

Date of manufacture:      Not available

Host Interface Board

Status:                   Optimal

Location:                 Slot 1

Type:                     Fibre channel

Number of ports:          2

Board ID:                 0902

Replacement part number:  L2-25043-03

Part number:              PN L2-25043-03

Serial number:            SN SQ91100387

Vendor:                   VN LSI

Date of manufacture:      June 1, 2009

Date/Time:                   Fri Oct 02 05:54:13 PDT 2009

Associated Logical Drives (* = Preferred Owner):

BK_01*, BK_04*, JS_01*, JS_03*, JS_2L*, JS_4L*, OR_02*, OR_04*, OS_01*, OS_03*,

WB_02*, WB_03*

STANDARD LOGICAL DRIVES——————————

SUMMARY

Number of standard logical drives: 24

See other Logical Drives sub-tabs for premium feature information.

NAME   STATUS   CAPACITY    RAID LEVEL  ARRAY  MEDIA TYPE       INTERFACE TYPE

BK_01  Optimal  557.791 GB  10          BK_01  Hard Disk Drive  Fibre channel

BK_02  Optimal  557.793 GB  10          BK_02  Hard Disk Drive  Fibre channel

BK_03  Optimal  557.793 GB  10          BK_03  Hard Disk Drive  Fibre channel

BK_04  Optimal  557.793 GB  10          BK_04  Hard Disk Drive  Fibre channel

JS_01  Optimal  1.089 TB    10          JS_01  Hard Disk Drive  Fibre channel

JS_02  Optimal  1.089 TB    10          JS_02  Hard Disk Drive  Fibre channel

JS_03  Optimal  1.089 TB    10          JS_03  Hard Disk Drive  Fibre channel

JS_04  Optimal  1.089 TB    10          JS_04  Hard Disk Drive  Fibre channel

JS_1L  Optimal  557.793 GB  10          JS_1L  Hard Disk Drive  Fibre channel

JS_2L  Optimal  557.793 GB  10          JS_2L  Hard Disk Drive  Fibre channel

JS_3L  Optimal  557.793 GB  10          JS_3L  Hard Disk Drive  Fibre channel

JS_4L  Optimal  557.793 GB  10          JS_4L  Hard Disk Drive  Fibre channel

OR_01  Optimal  557.793 GB  10          OR_01  Hard Disk Drive  Fibre channel

OR_02  Optimal  557.793 GB  10          OR_02  Hard Disk Drive  Fibre channel

OR_03  Optimal  557.793 GB  10          OR_03  Hard Disk Drive  Fibre channel

OR_04  Optimal  557.793 GB  10          OR_04  Hard Disk Drive  Fibre channel

OS_01  Optimal  836.689 GB  5           0S_1   Hard Disk Drive  Fibre channel

OS_02  Optimal  836.689 GB  5           OS_02  Hard Disk Drive  Fibre channel

OS_03  Optimal  836.689 GB  5           OS_03  Hard Disk Drive  Fibre channel

OS_04  Optimal  836.689 GB  5           OS_04  Hard Disk Drive  Fibre channel

WB_01  Optimal  557.793 GB  10          WB_01  Hard Disk Drive  Fibre channel

WB_02  Optimal  557.793 GB  10          WB_02  Hard Disk Drive  Fibre channel

WB_03  Optimal  557.793 GB  10          WB_03  Hard Disk Drive  Fibre channel

WB_04  Optimal  557.793 GB  10          WB_04  Hard Disk Drive  Fibre channel

DETAILS

Logical Drive name:            BK_01

Logical Drive status:       Optimal

Capacity:                   557.791 GB

Logical Drive ID:           60:08:0e:50:00:17:b6:ba:00:00:1a:5e:4a:c0:a8:24

Subsystem ID (SSID):        12

Associated array:           BK_01

RAID level:                 10

Secure:                     No

Media type:                 Hard Disk Drive

Interface type:             Fibre channel

Enclosure loss protection:  No

Preferred owner:            Controller in slot A

Current owner:              Controller in slot A

Segment size:                                       512 KB

Capacity reserved for future segment size changes:  Yes

Maximum future segment size:                        2,048 KB

Modification priority:                              High

Read cache:                            Enabled

Write cache:                           Enabled

Write cache without batteries:      Disabled

Write cache with mirroring:         Enabled

Flush write cache after (in seconds):  10.00

Dynamic cache read prefetch:           Enabled

Enable background media scan:          Enabled

Media scan with redundancy check:      Disabled

Pre-Read redundancy check:             Disabled

MAPPINGS (Storage Partitioning – Enabled (2 of 128 used))——————-

Logical Drive Name    LUN  Controller  Accessible by         Logical Drive status

BK_01                 4    A           Host Group VMware_01  Optimal

BK_03                 10   B           Host Group VMware_01  Optimal

JS_01                 2    A           Host Group VMware_01  Optimal

JS_03                 8    A           Host Group VMware_01  Optimal

JS_2L                 3    A           Host Group VMware_01  Optimal

JS_4L                 9    A           Host Group VMware_01  Optimal

OR_02                 1    A           Host Group VMware_01  Optimal

OR_04                 7    A           Host Group VMware_01  Optimal

OS_01                 0    A           Host Group VMware_01  Optimal

OS_03                 6    A           Host Group VMware_01  Optimal

WB_02                 5    A           Host Group VMware_01  Optimal

WB_04                 11   B           Host Group VMware_01  Optimal

BK_02                 4    B           Host Group VMware_02  Optimal

BK_04                 10   A           Host Group VMware_02  Optimal

JS_02                 2    B           Host Group VMware_02  Optimal

JS_04                 8    B           Host Group VMware_02  Optimal

JS_1L                 3    B           Host Group VMware_02  Optimal

JS_3L                 9    B           Host Group VMware_02  Optimal

OR_01                 1    B           Host Group VMware_02  Optimal

OR_03                 7    B           Host Group VMware_02  Optimal

OS_02                 0    B           Host Group VMware_02  Optimal

OS_04                 6    B           Host Group VMware_02  Optimal

WB_01                 5    B           Host Group VMware_02  Optimal

WB_03                 11   A           Host Group VMware_02  Optimal

Access Logical Drive  31   A,B         Host deimos           Optimal

Access Logical Drive  31   A,B         Host phobos           Optimal

Access Logical Drive  31   A,B         Storage Subsystem     Optimal

[2] The full disclosure for the IBM System x3950 M2 report is available at http://www.vmware.com/files/pdf/vmmark/VMmark-IBM-2009-06-16-x3950M2.pdf. To learn more about VMmark, including a full list of published results, go to http://www.vmware.com/products/vmmark/

[3] The IBM System Storage DS4700 is the previous generation of the IBM System Storage DS5020 examined by ESG Lab in this report.

[4] Web server Iometer (www.sourceforge.net/projects/iometer) workload definitions are included in a results file excerpt as Figure 13.

[5] For more detail, see the Appendix.

[6] The configuration and methodology that was used during characterization testing is described in the Appendix.

[7] For more on the characterization configuration and methodology please see the Appendix.

[8] IBM System Storage DS4800 Exchange Server 2007 15,000 Mailbox JetStress Analysis, David Hartman and David West, November 2007, http://www-03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP101123

[9] A sample JetStress log is included in the Appendix as Figure 11.

[10] Current trends in Database Performance, Andrew Holdsworth, Oracle OpenWorld, Nov 2007, http://www.oracle.com/technology/deploy/performance/pdf/PerfTrends_Holdsworth.pdf

[11] Back of the Envelope Database Storage Design, Nitin Vengurlekar, RAC/ASM Development, Oracle Open World, Nov 2007, http://www.oracle.com/technology/products/database/asm/pdf/back%20of%20the%20env%20by%20nitin%20oow%202007.pdf

[12] See Figure 13 for workload details.