Overall Storage Market

Two mega trends (or market demands) drive the storage market.  While not new, their overriding importance is noteworthy:

Value: is both an aim and necessary result, and is itself driven by three other significant trends: consolidation (just look at the resurgence of the mainframe for proof of that), virtualization, and the continuing dramatic increases in capacity demand.

Simplicity: this can be seen as the offspring of a successful marriage of intelligence and automation. While there may be (indeed, often should be) more ‘smarts’ under the covers, the storage system itself should be a breeze to understand and to operate.

If a vendor or product does not have a useful contribution in either one of these areas—and ideally, they should contribute to both—then that vendor or product will probably die. Products that are overly specialized, are techno-geek ‘crack,’ or that require an army of specialists to tweak all the knobs will be increasingly irrelevant as storage continues to become more of a flexible and automated servant of IT and less of a rigid and frustrating shackle restraining it. We must remember that storage is no more and no less than a tool—no-one makes money by having storage in grand isolation. It is a means to an end, and not an end in itself.

There are signs of cautious optimism returning across the IT market, with budgets nudging upward and slightly less emphasis on cost alone. As buyers begin to refocus on broader business IT and operational priorities, ESG’s Spending Intentions Survey has good news for the storage sector this year with a jump of 42% in respondents expecting to spend more on storage than in the preceding year (54% of respondents said they will spend more on storage in 2010 v. 2009 compared to only 38% that expected storage expenditure growth in 2009 over 2008).[1]

Block Storage Dynamics and Change in 2010

The recent economic gloom has certainly emphasized efficiency as an important goal for storage vendors and users; in good times, it’s easy to overlook things like lax utilization levels, perhaps a little ‘terabyte-ostentation,’ or even a touch of known over-provisioning. After all, there are bigger IT fish to fry: keeping applications up, controlling risk, and beating the competition. Surely, a few extra dollars spent on spinning rust hardly matters! But now the cat is out of the bag—none of the ‘bigger IT fish’ have gone away, but now the business demand for more efficient storage infrastructures is added. Therefore, the majority of the ‘top ten’ market dynamics for block storage in 2010 listed below concern efficiency in one way or another. Some of the individual elements outlined are stronger continuations of existing themes, some are things that will really gain traction this year, and a few items are in the very early stages but will begin to color users’ judgment.  Listed in no particular order, these dynamics will be central to shaping both the discussions and the decisions around block storage in 2010.

1. Tiering Matters: Although talked about for decades, a truly dynamic storage hierarchy returns to relevance with a vengeance. Why it’s happening now is something of a chicken and egg situation: it’s more needed as the volumes and complexity of data types increase and it’s finally becoming doable at all levels thanks to better hardware and—most important—software from both the emerging and established players. The winning formula combines granularity and automation in order to achieve optimum value and simplicity. Having a working storage hierarchy has often been described as a matter of operational elegance when the reality is that it’s always been all about the money—and that’s a bigger focus now that it has been for years. There is also a likelihood that the rise of an SSD + SATA combination will lead to a squeeze on Fibre Channel.
2. Solid State is Solid: Just about every block storage vendor is backing solid state so extensive market success is a matter of when, not if. The key to success in 2010 is providing tools and guidance so that IT can effectively use the power of solid state. Generally speaking, that means serving IO and not capacity (which will continue to spin for many years to come). Solid state will also continue to gain traction in more forms than SSD as the market understanding and ability to use all forms of solid state grow in concert with continuing price declines.
3. Capacity – Management and Control: Every time ESG surveys the market, ‘coping with data/storage/capacity growth’ is a top issue for IT. The joint emphases of reducing waste while also better utilizing what is actually purchased makes two storage technologies continue to grow strongly this year:
   1. Virtualization: Its logic is inescapable, even if only for the ability to thinly provision, which for the majority of users and the majority of time is a slam-dunk way for them to get more bang for their storage buck. In addition to saving money, storage virtualization adds value with tools such as snapshots, replication, improved DR, and so on.
   2. Deduplication: This is another logical addition for many users. Its adoption rate is poised to increase further and faster in 2010, with a vast array of choice for users to consider—hardware appliance or software tool being the main one. As a common-sense concept, it is rapidly becoming table stakes and a feature rather than a product or market.
4. Unified Storage: The same two driving market forces—value and simplicity—are rapidly turning unified storage from an interesting idea into a big deal as well as a growing reality. Software managed storage pools are the way of the future, with the data being presented to the compute layer as SAN, NAS, or objects as needed. The old approach of application specific storage (try imagining one PC for PPT, another for Word, and so on) was born of expediency and generous budgets, but is crazy. Instead, applications and users should be able to share a pool of storage resources; taking a broader view, this move to unified storage ties in with all the ‘converged IT / stack’ focus—whether from VCE/Acadia, HP, IBM, or others.
5. General Purpose Storage: This market dynamic is similar to unified storage, but concerns product attributes rather than access. Much as we have different vehicles for different purposes (cars, trucks, minivans, etc.) we’ve grown up with different storage platforms for different purposes. However, there can be considerable value in multi-purpose tools, both operationally (sharing) and economically (better usage/negotiation). This is true whether it manifests as a crossover vehicle or a general purpose storage platform. Such tools will—by design— never excel at everything (that’s why Ferraris and big trucks still exist), but they can do the majority of what the majority of users need. Expect more such offerings because the growth of ‘full-stack’ IT products also mitigates in favor of ostensibly simple (even if complex underneath) pools of available storage.
6. Data Protection: Reducing the risk of data loss remains at the top of IT managers’ lists in all its forms: security, availability, backup, compliance, and so on. The most valuable thing in IT is the data—always was and always will be. Everything else is a replaceable resource. This means balancing the best RPO and RTO that an application demands and that a user can afford. Vendors that improve ease and level of affordable data protection will fare better than others. In terms of in-box data protection, the market will realize that some traditional RAID levels are complex (rebuilds take forever!) and that realistic alternatives, such as the chunking / striping approach or a simple multi-copy approach (using the value of ever-cheaper higher capacity drives) are emerging.
7. Cloud –‘Tiers but no Reign:’ This topic is included because it appears impossible to avoid. There is massive noise and hype, but it’s really just a consumption model: a bureau on a wire, if you like. While the ‘cloud’ may represent some particular values and opportunities, a consumption model does not change an application’s needs or the associated storage attributes to support it. Whether storage is fast or slow, fat or thin, replicated or not, the needs will be the same whether operated within a user’s own IT or somewhere else. Putting risk and security to one side, the only issue will once again be economics; we may therefore see some users taking advantage of a ‘cloud tier’ in their overall hierarchy, but that’s about the limit of the impact on storage per se.
8. Commodity-Based and ‘Back to Basics:’ With increased focus on ‘collapsed stacks’ and ‘convergence,’ advanced storage functionalities can move to horizontal management layers, including VMware, Hyper V, etc. This increases the importance of having the optimum ‘foundational storage,’ which in turn drives focus on improving the ‘basic’ aspects of raw ‘commodity’ storage such as reducing vibration (to drive up performance); increasing reliability; and facilitating easier monitoring, reporting, and [self] repair.
9. Broad Economic Efficiencies: This bears repeating: whether motivated by CAPEX, OPEX, or green undertakings, ‘the cat is out of the bag’ regarding poor and lax storage management, provisioning, and use. From power consumption to spin down and better provisioning to de-dupe: efficiency matters.
10. Vendor Positioning: Most noticeably, the market shake-out will continue. Acquisitions aren’t done, nor are failures. Less immediately obvious, but of longer-term importance, is the potential for tougher times for ‘pure-play’ storage vendors. General systems vendors will become more dominant over the next few years (based on IT commoditization and convergence), which may mean less opportunity for ‘pure-plays’ without added offerings or significant differentiation. It seems likely that the storage software stack will move to [cheap, standard] servers, which will turn the focus of storage further toward a commodity angle and squeeze margins.

The Bigger Truth

It is ironic (but also inevitable given everything else outlined in this paper), that just as storage gets ever more important, we find more and more that no-one wants to be bothered with it at a detailed level every day. But perhaps that’s a good sign. After all, we used to crank cars, adjust the choke manually, and drive stick-shifts, but few do any of those any more even though cars are ever more essential to so many aspects of our lives. It matters (in a positive sense) that storage doesn’t matter like it used to! Invisibility and automated flexibility are the names of the game and the outlook for 2010 shows signs of these being broadly achievable. To stave off too much hype, there are a couple of real world caveats that should be remembered!

1. Semantics are surprisingly important: things aren’t all the same just because they share a name. For example, tiering, thin provisioning, SSD integration, and deduplication implementations all vary widely. Caveat emptor.
2. Inertia is an ever-present market restraint: partly driven by buying cycles, partly by varying user and/or application needs, partly by natural human conservatism or skepticism, and—yes—partly by politics, sloth, or even mismatched employee measurement and reward systems. In the end, not everything changes overnight or is based on logic.

All this said, block storage is a crucial market segment, running many key ‘tier one’ applications, and it looks set to be vibrant in 2010. While often placed in a ‘not the sexiest, but it’s really important’ category, there are significant changes entering the block segment and being embraced in 2010. As we gradually move to a unified and converged world, so the importance of block storage, file storage, or whatever storage will wane; but storage as a whole remains crucial because it holds the crown jewels of IT: the data. The continuing battle will be over price (of course) and the degree of sophistication to work with the operating and/or virtualizing system to provide storage to applications in a manner that is easily flexible in all possible dimensions. Vendors and products that advance both value and simplicity will win.