The Challenges

The use of server virtualization technology is on the rise among organizations of all sizes and in all industries around the world.  In a recent ESG survey of current and planned users of the technology, 52% of organizations had already deployed, while 48% plan to do so.[1] Given the impressive economic benefits of server virtualization, the glut of affordable and under-utilized processing power, and growing power and cooling issues in the data center, ESG predicts that the brisk adoption of server virtualization will continue for the foreseeable future.

ESG research indicates that the vast majority (87%) of organizations that have deployed server virtualization have done so in conjunction with networked storage.  Compared to islands of direct attached hard drives, utilization is greatly increased when applications share a pool of networked storage. Applications deployed on virtual machines sharing a pool of storage are more mobile and available than those deployed on direct attached hard drives.

While the benefits of server virtualization and networked storage are clearly compelling, IT managers are faced with a number of challenges as they try to manage a consolidated mix of real-world applications running on a virtualized infrastructure.  As shown in Figure 1, the top two concerns are performance and a general lack of information and best practices.  This holds true across organizations of all sizes, regardless of the number of virtual servers deployed.

The Solution

This ESG Lab report examines the performance of real-world application workloads running in a virtualized and consolidated IT environment that leverages the following technologies:

• IBM System Storage DS5020 Express and DS3950 Express storage systems: With high performance that is optimized for mixed workloads, the DS5020 Express and the DS3950 Express were designed for modular scalability (capacity and/or performance), high availability, and advanced functionality including copy services and remote replication.
• IBM BladeCenter HS22 servers: The IBM BladeCenter HS22 is a highly efficient server with extraordinary scalability that delivers the ability to add more processing power, memory, or IO needed in virtualized environments.
• Microsoft Windows Server 2008 R2: With a built-in and updated Hyper-V platform and Live Migration, organizations can increase the efficiency and flexibility of their server infrastructure with Windows Server 2008 R2.
• QLogic 8 Gb SAN Switch Module and CFFh and CIOv form factor FC Expansion cards for IBM BladeCenter: Providing  up to 20 ports of SAN connectivity,  the QLogic 8 Gb Fibre Channel switch and expansion card infrastructure is designed to deliver sustained high throughput and reliability in highly available virtual environments.

The capabilities of the IBM servers and storage that were used during this evaluation are summarized in Figure 2. The IBM BladeCenter H supports up to 14 blade servers, each populated with up to eight Intel Xeon 5500 processor cores and 98 GB of DDR-3 RAM.  The IBM System Storage DS5020 Express and the DS3950 Express support up to 112 drives (FC, SATA, or mixed configurations) and are equipped with up to 4 GB of cache and 8 GB/sec of internal bandwidth.  The DS3950 Express is a variant of the DS5020 Express that is available in two pre-configured models.  The DS5020 Express can be custom configured and adds support for full disk encryption (FDE).  The DS5020 Express supports up to eight FC host interfaces and the DS3950 Express supports up to four.

The Results

This report examines the performance capabilities of IBM System Storage DS5020 Express and DS3950 Express storage systems running a mix of real-world applications in a Microsoft Hyper-V R2 enabled virtual server environment powered by a pair of IBM BladeCenter HS22 servers.   In particular, this report explores how:

• A single DS5020 attached to a pair of BladeCenter HS22 servers running a mix of real-world application workloads in 16 Microsoft Hyper-V R2 enable virtual machines supports up to:
  • 19,992 mailboxes using the Microsoft Exchange 2010 Jetstress utility
  • and 3,362 small database IOs per second using the Microsoft SQLIO utility
  • and 850 MB/sec of database throughput for large scan operations using the SQLIO utility
  • and 3,319 simulated web server IOPs
  • and 407 MB/sec of throughput for simulated backup/scan/index jobs
  • with the predictably fast response times and scalability
  • Within a Microsoft enabled virtual server infrastructure, the DS5020 Express achieved a maximum aggregate throughput of 3.28 GB/sec during bandwidth intensive performance testing and 1.13 GB/sec during mixed application workload testing.

The predictably fast, mixed workload performance scalability of the virtualized environment tested by ESG Lab is summarized in Figure 3.   The results will be explored in detail later in this report, but for now it should be noted that the performance of the DS5020 Express scaled extremely well as a mix of real-world application workloads ran in parallel on up to 16 virtual machines.

The balance of this report explores how mixed workload testing was accomplished, what the results mean, and why they matter to your business.

ESG Lab Validation

The methodology presented in this report was designed to assess the performance capabilities of a single IBM DS5020 Express storage system shared by multiple Microsoft Hyper-V R2 enabled virtual machines running a mix of real-world application workloads.  The cooperation of Microsoft, IBM and QLogic played a vital role in the success of this project.  In particular, this project benefitted from Microsoft’s expertise in helping customers plan for the deployment of business-critical applications in consolidated environments and IBM’s long heritage of success in the modular storage systems market.

A Mixed Real-world Storage Benchmark Methodology

Conventional server benchmarks were designed to measure the performance of a single application running on a single operating system inside a single physical computer. SPEC CPU2000 and CPU2006 are well known examples of this type of server benchmarking tool. Much like traditional server benchmarks, conventional storage system benchmarks were designed to measure the performance of a single storage system running a single application workload.  The SPC-1 benchmark, developed and managed by the Storage Performance Council with IBM playing a key role, is a great example. SPC-1 was designed to assess the performance capabilities of a single storage system as it services an online interactive database application.

The IBM DS5020 used during this ESG Lab Validation has a published SPC-1 result of 26,080 SPC-1 IOPS with a price-performance value of $8.08 per SPC-1 IOP. [2] This is an excellent result for a modular storage system of this class.   The DS5020 also has a published Microsoft Exchange Solution Review Program (ESRP) report which indicates that the DS5020 supports up to 28,000 simulated Microsoft Exchange 2007 mailboxes.

While each of these results can be used to predict how a  server and storage solution will perform when running a single application workload,  neither was designed to help  IT managers understand what happens when a mix of applications are deployed together in a virtual server environment.

Compared to a traditional benchmark which tests a single application running on a physical server, ESG Lab tested a mix of virtualized applications running at the same time.  The methodology shown in Figure 4  was designed to assess the performance of a mix of application workloads, including database and e-mail applications, running on hypervisor enabled virtual servers sharing a single pool of SAN attached storage.

A cell concept was used during ESG Lab mixed workload testing.  As shown in Figure 5, each cell is composed of a mixture of four application workloads.  Two physical servers, each configured with eight virtual machines, were used to measure performance as the number of active cells was increased from one to four.

Mixed Workloads

Industry standard benchmarks were used to emulate the IO activity of four common business application workloads:

• E-Mail: The Microsoft Jetstress 2010 utility was used to generate e-mail traffic.  Jetstress simulates the activity of typical Microsoft Exchange users as they send and read e-mails, make appointments, manage to-do lists, etc.  The Jetstress utility is a lightweight tool that is ideally suited for predicting storage performance in a Microsoft Exchange environment.  The JetStress utility uses the jet engine database that Microsoft Exchange uses internally.  JetStress 2007 is the tool that was used in the IBM DS5020 Microsoft ESRP test that was mentioned previously in this report.   ESG Lab tested with JetStress 2010 running in Hyper-V R2 enabled virtual machines running Microsoft Server 2008 R2 as a guest operating system.
• Database: The Microsoft sqlio utility was used to generate database traffic. Much like Jetstress, sqlio is a lightweight tool that is ideally suited for measuring storage performance.  Sqlio was designed to help administrators understand the performance capabilities of a storage system, either to uncover performance issues or to size a new database installation without having to create and run a SQL Server   database. Sqlio was used to simulate two types of database activity: response-time sensitive online transaction processing (OLTP) and bandwidth sensitive decision support systems (DSS).

• Web Server: The industry standard Iometer utility was used to generate web server traffic. The IO definition was composed of random reads of various block sizes.  The web server Iometer profile used for this test was originally distributed by Intel, the author of Iometer. Iometer has since become an open source project.[3] Iometer tests were performed on Hyper-V R2 enabled virtual hard disks running within virtual machines running Windows 2008 R2 as the guest operating system.  Windows NTFS file systems were not configured for access as a drive letter. Instead, Iometer testing was performed at the underlying Windows physical drive level (e.g. , Physical Drive 3).
• Scan/read: The Iometer utility was used to generate a single stream of bandwidth intensive sequential read traffic.  Operations that tend to generate this type of large block sequential traffic include scan and index operations, long running data base queries, backup operations, bulk data uploads, and copies. One 256 KB sequential read workload was included in each cell to add a throughput intensive component to the predominantly random IO profile of interactive e-mail, database, and web server applications.  As most experienced database and storage administrators have learned, a throughput intensive burst in IO traffic can drag down the performance for interactive applications, causing performance problems for end-users.  Adding a few streams of throughput intensive scan/read traffic was used to determine whether interactive performance would remain predictably responsive as the amount of mixed IO utilization increased.

Each of the four workloads ran in parallel for approximately two hours.  The Jetstress e-mail test ended with a mandatory database consistency check which completed roughly an hour after the rest of the mixed workloads had completed.   The settings for each of the industry standard benchmarks are documented in the appendix.

Test Bed

Microsoft Windows Server 2008 R2 was installed on a pair of powerful IBM BladeCenter HS22 servers, each with a pair of quad-core processors and a QLogic 8 Gb Fibre Channel CFFh and CIOv expansion card providing four ports of connectivity per blade.  A DS5020 Express with 112 15K RPM FC drives was connected to the servers through four QLogic 8 Gb SAN Switch Module for IBM BladeCenter as shown in Figure 6.

Drive Layout

The DS5020 drive configuration is summarized in Table 1. Four Exchange database volumes were configured. Each of the Exchange database volumes was configured with an eight drive RAID-10 database volume and a four drive RAID-10 log volume.   The SQL Server, web server and scan/read workloads ran against four drive RAID-10 volumes. The guest operating system volumes were configured using a 3+1 RAID-5 layout.   Volume ownership was balanced across the dual controllers within the DS5020 Express and distributed evenly over the eight host interfaces.   The volumes configured for access over two redundant host interfaces and accessed through the IBM redundant disk array controller (RDAC) driver. [4]

Configuring Virtual Machines

Each of the Microsoft Jetstress virtual machines was configured with two virtual CPU cores and 12 GB of RAM.  DS5020 Express disk capacity was used for all storage capacity including the Windows Server 2008 R2 guest operating system,   application executables and application data.   All of the application data volumes under test were configured as virtual hard disks (VHDs).  The configuration of one of the four virtual machines used for e-mail testing is shown in Figure 7. Note how three hard disks have been configured: one for the guest operating system and two for the Exchange database and Logs.

The Results

In a way, storage system benchmark testing is like an analysis of the performance of a car. Specifications including horsepower and acceleration from 0 to 60 are a good first pass indicator of a car’s performance.  But while specifications provide a good starting point, there are a variety of other factors that should be taken into consideration including the condition of the road, the skill of the driver, and gas mileage ratings.  Much like buying a car, a test drive with real-world application traffic is the best way to determine how a storage system will perform in real-world conditions.

Characterization

Performance analysis began with an examination of the low level aggregate throughput capabilities of the test bed.  This testing was performed using the Iometer utility running within eight of the virtual machines that were used later during mixed workload testing.  The eight virtual machines accessed DS5020 Express storage through eight 8 Gbps FC interfaces.

Iometer access definitions which measured the maximum throughput from disk were used for this first pass analysis of the underlying capabilities of the DS5020 Express.   Similar to a dynamometer horsepower rating for a car, maximum throughput was used to quantify the power of a turbo-charged DS5020 Express storage engine. As shown in Figure 8, ESG Lab recorded a maximum throughput of 3.3 GB/sec.

What the Numbers Mean

• Much like the horsepower rating of a car, the throughput rating of a storage system is a good indicator of the power of a storage system’s engine.
• Storage throughput is a measure of the bandwidth available to the system. Throughput can be measured on a stream or aggregate basis.  A stream is represented by one application or user communicating through one IO interface to one device.  Aggregate throughput is a measure of how much data the storage system can move on a whole for all applications and users.
• ESG Lab throughput characterization was performed using the industry standard Iometer utility as 32 streams performed large sequential reads (1 MB) from eight logical devices through eight FC interfaces.
• ESG Lab recorded a peak aggregate throughput of 3.3 GB/sec in a Microsoft Hyper-V R2 enabled virtual server environment.
• Approximately fifteen percent of the throughput was delivered from DS5020 Express cache; the balance was serviced from disk.
• When comparing the performance capabilities of two servers in a virtual server environment, the server with more cache tends to perform better.    ESG Lab is confident that a similar pattern holds true for storage systems.  A storage system with more cache—and better caching algorithms—should perform better in a virtual server environment.
• ESG Lab characterization testing indicates that the DS5020 Express has more than enough cache and front end bandwidth to meet the needs of virtualized applications requiring up to 112 disk drives for capacity.
• ESG Lab is convinced that the patented caching algorithms of the DS5020 Express provide a significant performance boost during mixed application virtualized application testing.

Virtual Machine Utilization

Mixed application testing began with a quick analysis of server memory and CPU utilization to make sure that there were no bottlenecks between virtualized applications and the DS5020 Express.  Memory and CPU utilization as reported by Microsoft Windows 2008 R2 performance monitor is shown in Figure 9.

These screenshots were taken during the peak activity phase of the four cell test.  With memory and CPU utilization at less than 10%, there was no obvious bottleneck between virtualized applications and the DS5020 Express.

Mixed Real-world IOPS Scalability

I/Os per second, or IOPS, is a measure of the number of operations that a storage system can perform in parallel. When a system is able to move a lot of IOPS—from disk and from cache— it will tend to be able to service more applications and users in parallel. Much like the horsepower rating for a car engine, the IOPS rating for a storage controller can be used as an indicator of the power of a storage system engine.

While IOPS out of a cache is typically a big number and can provide an indication of the speed of the front end of a storage controller, IOPS from disk is a more useful metric when determining the real-world performance of a storage system servicing a mix of business applications.  For example, e-mail and interactive database applications tend to be random in nature and therefore benefit from good IOPS from disk.  With that said, a mix of real-world applications tends to have random and sequential IO traffic patterns that may be serviced from disk or from cache.

ESG Lab recorded the number of IOPS as reported by the DS5020 Express as the number of virtual machines running mixed real-world application workloads was increased from four through sixteen.  With a mix of random and sequential IO over 112 disk drives, the goal was not to record a big IOPS number; the goal with this exercise was an assessment of the scalability of the DS5020 Express as an increasing number of applications are consolidated onto a single virtualized platform.  Mixed workload IOPS scalability is shown in Figure 10.

What the Numbers Mean

• IOPS varied throughout the mixed workload test with peaks occurring towards the beginning during the SQL Server small IOPs phase and then again towards the end as the Jetstress utility performed a database consistency check.
• A peak of 16,572 IOPS was reported by the DS5020 performance monitor during the four cell run.
• The average number of IOPS during each three hour run is shown in Figure 10.
• The average number of IOPS scaled in a near-linear fashion as mixed real-world application traffic increased from four through sixteen virtual servers.

Mixed Application Performance Scalability

Having looked at the IOPS and throughput ratings of the turbo-charged DS5020 Express engine, here’s where the rubber meets the road as we examine performance at the application level. The output from each of the industry standard benchmark utilities was analyzed to determine the performance scalability and responsiveness of real-world applications running in a consolidated virtual environment.

Microsoft Exchange

The I/O efficiency and performance efficiency of Microsoft Exchange has improved significantly over the years. Architectural improvements in Exchange 2010 including a new store schema, large database IO sizes,  improved read/write coalescing, improved pre-read support and increased cache effectiveness have reduced the number of I/O’s per user up to 70% compared to Exchange 2007.[5] ESG Lab typically uses a value of 0.5 IOPS per mailbox to emulate a typical worker when testing with JetStress 2007.  A value of 0.12 IOPS per mailbox was used during JetStress 2010 testing to reflect the 70% reduction in IOPS compared to Exchange 2007.

The Microsoft Jetstress 2010 utility was used to see how many simulated e-mail users could be supported by the DS5020 Express during mixed workload testing.  The number of IOPS and response time for each database and log volume was recorded at the end of each Jetstress run.   A response time goal of 20 milliseconds or less for database reads is required to pass the test.  These values are defined by Microsoft as a limit beyond which end-users will feel that their e-mail system is acting slowly. [6]

Microsoft Jetstress logs were used to determine the number of IOPS and response times as the number of active virtual machines was increased from four through sixteen.[7] Based on a 0.12 IOPS user profile, the number of IOPS was used to calculate the number of supported Exchange users.  The number of supported mailboxes as the number of virtual machines was increased from four to sixteen is shown in Figure 11, Figure 12 and Table 2.

What the Numbers Mean

• The single cell mixed application test supported 7,050 Exchange users with an average DB disk response time of 7 milliseconds.
• Performance scaled in a near-linear fashion to 19,992 users while the DS5020 Express was busy processing and servicing other applications concurrently.
• As the number of simulated e-mail users was increased, the DS5020 Express provided excellent response times that are well within Microsoft’s guidelines.  For example, the Microsoft guideline for a database read volume is 20 milliseconds as shown by the dotted line in Figure 12.
• The I/O efficiency improvements in Exchange 2010 reduces the cost of delivering e-mail support in mixed virtual server environments.  In this case, ESG Lab supported up to 19,992 mailboxes on four virtualized Exchange 2010 servers in a mixed workload environment—twice the expected number of supported mailboxes within an Exchange 2007 environment.

Microsoft SQL Server

The Microsoft sqlio utility was used to measure the performance of small (8 KB) random and large (256 KB) sequential read workloads. The small random results were used to predict the performance and scalability of response time sensitive interactive database applications (e.g. OLTP).  The large sequential results were used to predict the performance of throughput intensive database mining and decision support systems (DSS).

ESG used the following Microsoft TechNet guidelines to interpret the results:

Disk Latency – Lower values are better but this can vary and is dependent on the size and nature of the IO’s being issued. On well-tuned IO subsystems, ideal values would be:

• 4–20 ms for Data on OLTP systems (ideally 10 ms or less)
• 30 ms or less on DSS (decision support system) type[8]

The results for the four cell sqlio test are summarized in Table 3. A sample sqlio log is shown in the Appendix.

What the Numbers Mean

• The four cell test achieved a grand total of 3,362 OLTP IOPS and 850 MB/sec of aggregate DSS throughput while the system was simultaneously running a mix of real-world application workloads.
• The DS5020 delivered excellent response times that are well under the 20 and 30 Microsoft TechNet limits.
• Database architects often use a value of 150 IOPS per drive for planning purposes. The total number of small IOPS processed during the busy four cell test yielded an excellent rate of 210 IOPS per drive.
• ESG Lab believes that these are excellent results—especially given the mix of IO intensive workloads that were being serviced by the DS5020 Express in parallel.

Web Server and Scan/Read

Performance results for the web server and scan/read workloads executing within virtual machines during the four cell test are listed in Table 4.

What the Numbers Mean

• Given the cache friendly, read-only nature of web server IO traffic, ESG Lab believes that these results indicate that the DS5020 Express has the horsepower required to service tens of thousands of simultaneous page requests.
• ESG Lab believes that a file system workload would produce results that are approximately similar to the web server workload used for this test.
• Each of the four scan/read streams sustained approximately 100 MB/sec of throughput for the entire duration of the mixed workload test.  A stream of this magnitude could service the data needs of a number of simultaneous backup streams, a very aggressive scan and index job, or a throughput intensive database table scan—with no perceivable performance impact on applications that are running parallel.

Much like the electrical system in your home, figuring out how many appliances you can run in parallel before blowing a fuse is not a function of the number of wires behind the walls. What matters more is the design of the circuits used to distribute the right amount of power to appliances when needed.  ESG Lab testing indicates that the DS5020 Express engine delivers the right amount of power to virtualized applications when needed.

DS3950 Express Performance Analysis

The DS3950 Express supports up to four FC host interfaces as compared to the DS5020 Express, which supports up to eight FC host interfaces. Otherwise, the components and architecture of the DS3950 Express are exactly the same as the DS5020 Express. ESG Lab tested the DS5020 Express with only four active FC host interfaces connected to the IBM BladeCenter HS22 servers with a goal of analyzing the performance difference between the DS5020 Express and the DS3950 Express. The results are shown in Figure 13 and Table 5.

What the Numbers Mean

• The simulated DS3950 Express with four active host paths delivered 78.82% of the throughput and 92.17% of the IOPS from cache compared to the DS5020 Express with eight active paths.
• The mostly random ESG Lab mixed workload performed roughly the same.  This is due to the fact that the most important performance consideration for mostly random business application workloads is the number of disk drives operating in parallel. In this case, the same number of drives was tested (112).

ESG Lab Validation Highlights

• 3.28 GB/sec of aggregate throughput was sustained during characterization testing within a Microsoft Hyper-V R2 environment.
  • A single DS5020 attached to a pair of BladeCenter HS22 servers running a mix of real-world application workloads in 16 Microsoft Hyper-V R2 enable virtual machines supports up to:
  • 19,992 mailboxes using the Microsoft Exchange 2010 Jetstress utility
  • and 3,362 small database IOs per second using the Microsoft SQLIO utility
  • and 850 MB/sec of database throughput for large scan operations using the SQLIO utility
  • and 3,319 simulated web server IOPs
  • and 407 MB/sec of throughput for simulated backup/scan/index jobs
  • with the predictably fast response times and scalability
• Excellent IOPs per drive were recorded (e.g., 210 for the SQL Server OLTP test).
• As the number of virtual machines sharing a single DS5020 was increased, performance scaled in a near linear fashion with predictably fast response times (7 to 10 milliseconds for Jetstress DB reads).
• The DS5020 had horsepower to spare for rebuilds and advanced functions including copy services and remote replication.

Issues to Consider

• Generally accepted best practices and predominantly default Microsoft and IBM System Storage settings were used during the design of this test.  As expected after any benchmark of this magnitude, deep analysis of the results indicates that tuning would probably yield slighter higher absolute results. Given that the goal of this test was not to generate a big number,  ESG Lab is confident that the results presented in this report meet the objective of estimating performance scalability and responsiveness as a growing number of virtual machines share a consolidated pool of DS5020 Express storage.
• For applications requiring extreme performance beyond that which is provided by FC and SATA drives, ESG Lab believes that the DS5020 Express is an ideal architecture for the selective use of solid state disk (SSD) devices.  While mixed workload testing was not performed with SSD devices,   ESG Lab is confident that SSD devices could be used to improve the performance of highly referenced database indexes and temp files.
• The test results/data presented in this document are based on industry-standard benchmarks deployed together in a controlled environment. Due to the many variables in each production data center environment, it is still important to perform capacity planning and testing in your own environment to validate a storage system configuration.

The Bigger Truth

Server virtualization is being deployed by a growing number of organizations to lower costs, improve resource utilization, provide non-disruptive upgrades, and increase availability. Each benefit is fundamentally enabled by de-coupling servers, applications, and data from specific physical assets.  Storage virtualization takes those very same benefits and extends them from servers to the underlying storage domain—bringing IT organizations one step closer to the ideal of a completely virtualized IT infrastructure.

While the benefits of a completely virtualized infrastructure are obvious to most IT managers, performance is a real concern.   Server, storage, and network administrators are looking for answers to a number of questions:

IBM approached ESG Lab with an ambitious goal of answering these questions. A performance benchmark was designed to measure the performance capabilities of a storage system subjected to an IO intensive mix of virtualized business applications running in a Microsoft Windows Server 2008 R2 environment. A cell concept was used during the design of this test.  Each cell was composed of four applications, each running in its own virtual machine.    The server horsepower of a pair of IBM BladeCenter HS22 servers was used to drive up to four cells and sixteen virtual applications in parallel.

IBM has more than a decade of experience delivering modular FC-attached storage systems designed to meet the cost-optimized performance demands of medium-sized organizations, mid-tier applications, remote departments, and near-line applications. The IBM DS5000 series builds on the heritage of the previous generation DS4000 series disk system with more than 87,000 systems and 511 petabytes shipped to date.  The engine under the hood the DS5000 Series has been turbo-charged to meet the real-world performance demands of virtualized applications.  With twice the host bandwidth and three times the internal bandwidth of the previous generation DS4700, the DS5020 Express is designed to deliver the high performance, low latency, and balanced scalability needed to meet the demanding performance needs of a mix of real-world applications sharing a consolidated infrastructure.

ESG Lab testing began with the configuration and deployment of Microsoft enabled virtual machines.  Bare metal Hyper-V R2, which is built into Windows Server 2008 R2, provided the underlying virtualization technology.  Wizard driven configuration with a familiar look and feel made it easy to configure, clone and manage the virtualized environment.  The combination of high performance IBM DS5020 storage, the enhanced high availability of Windows Server 2008 R2 (e.g. Live Migration), and the improved I/O performance of Hyper-V R2 and Exchange 2010 created a solid foundation for mixed workload testing.

The first set of tests revealed that the DS5020 Express can deliver up to 3.28 GB/sec of raw aggregate throughput in a Microsoft Hyper-V R2 environment.  This result was an early indicator that the IBM DS5020 Express has the internal bandwidth and processing power needed to serve a mix of real-world application workloads. The results of the mixed workload tests were even more impressive. A single DS5020 Express simultaneously supported 19,992 simulated Exchange 2010 users and 3,362 SQL Server OLTP IOPs  and 850 MB/sec of SQL Server DSS throughput and 3,319 simulated web server IOPs and 407 MB/sec of throughput for bandwidth intensive read traffic—all while delivering predictably fast response times.

ESG Lab is pleased to report that the combination of IBM System Storage DS5020 Express, IBM BladeCenter HS22 servers and QLogic 8 Gb Fibre Channel switch and expansion card infrastructure can be used with confidence to meet the performance needs of a mix of real-world business applications running in a Microsoft Windows Server 2008 R2 environment with built-in Hyper-V R2 virtual server support.

Appendix

This is an example of the output created by the Jetstress utility. It shows the performance for the Jetstress running in the first VM (JS01) during the single cell/four VM test.

The following parameters that were used during sqlio testing were derived from Microsoft SQL Server best practices (http://msdn.microsoft.com/en-us/library/cc966412.aspx):

echo ***run 1 of 2: OLTP reads***

sqlio -BN -o8 -kR -frandom -b8 -LS -s2700 -Fsqlioparms.txt >>sqlresults.txt

echo ***run 2 of 2: DSS scans ***

sqlio -BN -o8 -kR -fsequential -b256 -LS -s2700 -Fsqlioparms.txt >>sqlresults.txt

sqlioparms.txt:  e:\sqlio_test.dat 1 0×0 100000

A sample of the results of sqlio testing follows. This example shows the performance of one of four sqlio tests which ran in parallel during the four cell/16 VM test.

************** esg lab mixed workload sqlio testing begins*************

Wed 03/10/2010 1:55 PM

***run 1 of 2: OLTP read***

sqlio v1.5.SG

using system counter for latency timings, 3579545 counts per second

parameter file used: sqlioparms.txt

file e:\sqlio_test.dat with 1 thread (0) using mask 0×0 (0)

1 thread reading for 2700 secs from file e:\sqlio_test.dat

using 8KB random IOs

enabling multiple I/Os per thread with 8 outstanding

buffering set to not use file nor disk caches (as is SQL Server)

using specified size: 100000 MB for file: e:\sqlio_test.dat

initialization done

CUMULATIVE DATA:

throughput metrics:

IOs/sec:   989.98

MBs/sec:     7.73

latency metrics:

Min_Latency(ms): 0

Avg_Latency(ms): 7

Max_Latency(ms): 304

histogram:

ms: 0  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24+

%:  0  1  6  12 14 14 11 7  6  5     4   4   3   2    2   1   1   1   1   1    1   1    0   2

***run 2 of 2: DSS scan***

sqlio v1.5.SG

using system counter for latency timings, 3579545 counts per second

parameter file used: sqlioparms.txt

file e:\sqlio_test.dat with 1 thread (0) using mask 0×0 (0)

1 thread reading for 2700 secs from file e:\sqlio_test.dat

using 256KB sequential IOs

enabling multiple I/Os per thread with 8 outstanding

buffering set to not use file nor disk caches (as is SQL Server)

using specified size: 100000 MB for file: e:\sqlio_test.dat

initialization done

CUMULATIVE DATA:

throughput metrics:

IOs/sec:   1127.42

MBs/sec:   281.85

latency metrics:

Min_Latency(ms): 1

Avg_Latency(ms): 6

Max_Latency(ms): 1232

histogram:

ms: 0  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24+

%:  0  6 13  12  9  7  8  7   7    6    5   4   3   2   1    0   0   0   0   0   0    0   0   1

************** esg lab mixed workload sqlio testing ends***************

This is an example of the output created by the Iometer utility during mixed workload testing. This example shows the performance of the web server and scan/read virtual machines during the single cell/4 VM test.

The following excerpts were extracted from the IBM DS5020 Storage System Profile Summary.

PROFILE FOR STORAGE SUBSYSTEM: ESG_DS5020 (Fri Oct 02 05:53:29 PDT 2009)

SUMMARY——————————

Number of controllers:              2

High performance tier controllers:  Enabled

Number of arrays:                   24

RAID 6:                             Enabled

Total number of logical drives used:     25

Number of standard logical drives:    24

Number of access logical drives:      1

Total number of logical drives allowed:  1024

Drive Limit Management:

Number of drive slots discovered:  112

Number of drive slots allowed:     112

FlashCopy Logical Drives:                                 Enabled

Number of flashcopies used:                            0

Number of flashcopies allowed:                         2

Number of flashcopies allowed per base logical drive:  2

Remote Logical Drive Mirroring:  Disabled/Deactivated

Number of mirrors used:       0

Number of mirrors allowed:    0

VolumeCopy:                   Disabled

Number of copies used:     0

Number of copies allowed:  0

Number of drives:           112

Mixed drive types:          Enabled

Current media type(s):      Hard Disk Drive (112)

Current interface type(s):  Fibre (112)

Total hot spare drives:     0

Standby:                 0

In use:                  0

Drive Security:           Disabled

Security key identifier:  None

Storage Partitioning:             Enabled

Number of partitions used:     2

Number of partitions allowed:  128

Number of logical drives allowed per partition:  256

Access logical drive:  LUN 31,31,31 (see Mappings section for details)

Default host OS:       DEFAULT (Host OS index 0)

Current configuration

Firmware version:                    07.60.08.00

NVSRAM version:                      N1814D20R1060V08

EMW version:                         10.60.G5.05

AMW version:                         10.60.G5.05

NVSRAM configured for batteries:          Yes

Start cache flushing at (in percentage):  80

Stop cache flushing at (in percentage):   80

Cache block size (in KB):                 16

Media scan frequency (in days):                Disabled

Failover alert delay (in minutes):             5

Feature enable identifier:                     3030303934203030313235204A93DE36

Feature pack:                                  DS5020 Model 20, 24, 28

Feature pack submodel ID:                      121

Storage Subsystem world-wide identifier (ID):  60080E500017B6BA000000004A93DE34

CONTROLLERS——————————

Number of controllers: 2

Controller in Enclosure 85, Slot A

Status:                      Online

Current configuration

Firmware version:         07.60.08.00

Appware version:       07.60.08.00

Bootware version:      07.60.08.00

NVSRAM version:           N1814D20R1060V08

Replacement part number:     37781-03

Model name:                  4988

Board ID:                    4988

Submodel ID:                 121

Product ID:                  1814      FAStT

Revision:                    1060

Replacement part number:     37781-03

Part number:                 37781-03

Serial number:               SQ91100094

Vendor:                      IBM

Date of manufacture:         June 2, 2009

Trunking supported:          No

Data Cache

Total present:            1709 MB

Total used:               1709 MB

Processor cache:

Total present:            339 MB

Cache Backup Device

Status:                   Optimal

Type:                     USB flash drive

Location:                 Controller A, Connector USB 1

Capacity:                 1,960 MB

Product ID:               eUSB

Part number:              Not Available

Serial number:            200902190239A7D8

Revision level:           8715

Manufacturer:             SMART

Date of manufacture:      Not available

Host Interface Board

Status:                   Optimal

Location:                 Slot 1

Type:                     Fibre channel

Number of ports:          2

Board ID:                 0902

Replacement part number:  L2-25043-03

Part number:              PN L2-25043-03

Serial number:            SN SQ91100387

Vendor:                   VN LSI

Date of manufacture:      June 1, 2009

Date/Time:                   Fri Oct 02 05:54:13 PDT 2009

Associated Logical Drives (* = Preferred Owner):

BK_01*, BK_04*, JS_01*, JS_03*, JS_2L*, JS_4L*, OR_02*, OR_04*, OS_01*, OS_03*,

WB_02*, WB_03*

STANDARD LOGICAL DRIVES——————————

SUMMARY

Number of standard logical drives: 24

See other Logical Drives sub-tabs for premium feature information.

NAME   STATUS   CAPACITY    RAID LEVEL  ARRAY  MEDIA TYPE       INTERFACE TYPE

BK_01  Optimal  557.791 GB  10          BK_01  Hard Disk Drive  Fibre channel

BK_02  Optimal  557.793 GB  10          BK_02  Hard Disk Drive  Fibre channel

BK_03  Optimal  557.793 GB  10          BK_03  Hard Disk Drive  Fibre channel

BK_04  Optimal  557.793 GB  10          BK_04  Hard Disk Drive  Fibre channel

JS_01  Optimal  1.089 TB    10          JS_01  Hard Disk Drive  Fibre channel

JS_02  Optimal  1.089 TB    10          JS_02  Hard Disk Drive  Fibre channel

JS_03  Optimal  1.089 TB    10          JS_03  Hard Disk Drive  Fibre channel

JS_04  Optimal  1.089 TB    10          JS_04  Hard Disk Drive  Fibre channel

JS_1L  Optimal  557.793 GB  10          JS_1L  Hard Disk Drive  Fibre channel

JS_2L  Optimal  557.793 GB  10          JS_2L  Hard Disk Drive  Fibre channel

JS_3L  Optimal  557.793 GB  10          JS_3L  Hard Disk Drive  Fibre channel

JS_4L  Optimal  557.793 GB  10          JS_4L  Hard Disk Drive  Fibre channel

OR_01  Optimal  557.793 GB  10          OR_01  Hard Disk Drive  Fibre channel

OR_02  Optimal  557.793 GB  10          OR_02  Hard Disk Drive  Fibre channel

OR_03  Optimal  557.793 GB  10          OR_03  Hard Disk Drive  Fibre channel

OR_04  Optimal  557.793 GB  10          OR_04  Hard Disk Drive  Fibre channel

OS_01  Optimal  836.689 GB  5           0S_1   Hard Disk Drive  Fibre channel

OS_02  Optimal  836.689 GB  5           OS_02  Hard Disk Drive  Fibre channel

OS_03  Optimal  836.689 GB  5           OS_03  Hard Disk Drive  Fibre channel

OS_04  Optimal  836.689 GB  5           OS_04  Hard Disk Drive  Fibre channel

WB_01  Optimal  557.793 GB  10          WB_01  Hard Disk Drive  Fibre channel

WB_02  Optimal  557.793 GB  10          WB_02  Hard Disk Drive  Fibre channel

WB_03  Optimal  557.793 GB  10          WB_03  Hard Disk Drive  Fibre channel

WB_04  Optimal  557.793 GB  10          WB_04  Hard Disk Drive  Fibre channel

DETAILS

Logical Drive name:            BK_01

Logical Drive status:       Optimal

Capacity:                   557.791 GB

Logical Drive ID:           60:08:0e:50:00:17:b6:ba:00:00:1a:5e:4a:c0:a8:24

Subsystem ID (SSID):        12

Associated array:           BK_01

RAID level:                 10

Secure:                     No

Media type:                 Hard Disk Drive

Interface type:             Fibre channel

Enclosure loss protection:  No

Preferred owner:            Controller in slot A

Current owner:              Controller in slot A

Segment size:                                       512 KB

Capacity reserved for future segment size changes:  Yes

Maximum future segment size:                        2,048 KB

Modification priority:                              High

Read cache:                            Enabled

Write cache:                           Enabled

Write cache without batteries:      Disabled

Write cache with mirroring:         Enabled

Flush write cache after (in seconds):  10.00

Dynamic cache read prefetch:           Enabled

Enable background media scan:          Enabled

Media scan with redundancy check:      Disabled

Pre-Read redundancy check:             Disabled

MAPPINGS (Storage Partitioning – Enabled (2 of 128 used))——————-

Logical Drive Name    LUN  Controller  Accessible by         Logical Drive status

BK_01                 4    A           Host Group Msft_01  Optimal

BK_03                 10   B           Host Group Msft_01  Optimal

JS_01                 2    A           Host Group Msft_01  Optimal

JS_03                 8    A           Host Group Msft_01  Optimal

JS_2L                 3    A           Host Group Msft_01  Optimal

JS_4L                 9    A           Host Group Msft_01  Optimal

OR_02                 1    A           Host Group Msft_01  Optimal

OR_04                 7    A           Host Group Msft_01  Optimal

OS_01                 0    A           Host Group Msft_01  Optimal

OS_03                 6    A           Host Group Msft_01  Optimal

WB_02                 5    A           Host Group Msft_01  Optimal

WB_04                 11   B           Host Group Msft_01  Optimal

BK_02                 4    B           Host Group Msft_02  Optimal

BK_04                 10   A           Host Group Msft_02  Optimal

JS_02                 2    B           Host Group Msft_02  Optimal

JS_04                 8    B           Host Group Msft_02  Optimal

JS_1L                 3    B           Host Group Msft_02  Optimal

JS_3L                 9    B           Host Group Msft_02  Optimal

OR_01                 1    B           Host Group Msft_02  Optimal

OR_03                 7    B           Host Group Msft_02  Optimal

OS_02                 0    B           Host Group Msft_02  Optimal

OS_04                 6    B           Host Group Msft_02  Optimal

WB_01                 5    B           Host Group Msft_02  Optimal

WB_03                 11   A           Host Group Msft_02  Optimal

Access Logical Drive  31   A,B         Host deimos           Optimal

Access Logical Drive  31   A,B         Host phobos           Optimal

Access Logical Drive  31   A,B         Storage Subsystem     Optimal

[2] http://www.storageperformance.org/benchmark_results_files/SPC-1/IBM/A00081_IBM_DS5020-Express/a00081_IBM_DS5020-Express_SPC1_executive-summary-r1.pdf

[3] Web server Iometer (www.sourceforge.net/projects/iometer) workload definitions are included in a results file excerpt as Figure 13.

[4] For more detail, see the Appendix.

[5] http://download.microsoft.com/download/D/1/B/D1BE3AEC-A9CD-4459-99F1-B28867FAE20B/Exchange2010TCP_parte8.pdf

[6] http://technet.microsoft.com/en-us/library/bb738152(EXCHG.80).aspx

[7] A sample JetStress log is included in the Appendix.

[8] Sqlio is an unsupported tool provided by Microsoft that is available for download at: http://www.microsoft.com/downloads/details.aspx?familyid=9a8b005b-84e4-4f24-8d65-cb53442d9e19&displaylang=en