Energy costs in Qatar are exceptionally low (less than 2c per KWh) and this can contribute to the perception that there is a weak ROI on green energy efficient investments. However, this perception fails to take into account the lost opportunity of re-selling valuable energy resources abroad rather than consuming them wastefully on the domestic market.

So we had enough of the whys, now how do we go green? The simple magic words are “energy efficiency” and “high utilization.” We could choose to have 1,000 servers operating at 5% average efficiency or 100 servers operating at 50% average efficiency. Many organizations choose to operate one application on one server, so 1,000 applications means 1,000 servers. By introducing virtualization we can run multiple applications on shared servers reducing costs, improving efficiency and being green all at the same time.

Virtualization can enable a perfect storm of efficiency; we reduce the number of servers, thereby reducing both the capital cost and operation costs of those servers. Virtualization enables reductions in both data center capital costs and the operational cost of supplying electricity and cooling. One often forgotten additional advantage is the reduction in software licensing and maintenance costs because we are managing a smaller IT estate.

Storage is an important issue to focus on to improve efficiency; one enterprise disk of storage consumes as much as 1MWh (megawatt-hour) over its useful life. CIOs and COOs often find it very difficult to delete data that is no longer useful. Studies show that the chances of needing a document or a spreadsheet reduce exponentially across time, so the chances of you needing that 7-year-old spreadsheet ever again are close to zero. Best practice establishes that we should set a data retention policy and ruthlessly delete data that goes beyond that time.

Data deduplication can be an additional useful tool toward reducing data storage costs and increasing storage efficiency. It is a specific form of compression where redundant data is eliminated, typically to improve storage utilization. Deduplication is able to reduce the required storage capacity since only the unique data is stored. This in turn can reduce the overall footprint inside the data center. Deduplication can help by squeezing out between 10 to 20 percent more storage space just by getting rid of duplicated data.

Another valuable solution is to upgrade the data center equipment into modern equipment that has larger capacity, e.g., multiple old servers can be replaced by one modern server combining the benefit of energy efficiency and smaller space requirements. Upgrading old data center equipment can roughly increase energy efficiency by 3-4 times.

Cooling problems are clearly a major growing concern for data center managers. As per Gartner Research , it is estimated that data centers typically waste more than 60% of their energy just in cooling their equipment. Traditional cooling techniques are inadequate both economically and operationally. The solutions stemming from newer technologies are district cooling and hot aisle containment.

MEEZA data centers at QSTP leverage the massive investment made by the Qatar Foundation in district cooling, offering extremely efficient large scale plants that could not be replicated elsewhere in the country. Expert MEEZA engineers routinely virtualize customer applications, reducing the need for large numbers of servers and dramatically reducing overall energy consumption. MEEZA leads the way in IT sustainability by demonstrating best practice in green IT.

Read more of Steve’s blog entries at The Hot Aisle.

Businesses decisions are driven by three constraints, affordability, risk and time to deliver. Generally, suppliers to business leverage this understanding to deliver something that their customer needs yet is constrained from doing themselves. So, for example, if you need a 24 x 7 security patrol for your company premises, you can either choose to employ and manage a team of security officers or you can outsource the service to a professional security firm.

The outsourced security firm can be an effective business choice because it reduces risk and you can determine what you need by defining a service level (such as a full perimeter patrol every 2 hours and check all doors and windows are secure at 7PM) rather than deciding to employ a team of employees to deliver security. Service levels are the key business driver here because they force us to think about the problem before we think about the solution.

Much the same is true of Information Technology–companies can choose to think of IT as hardware, software and people that somehow come together to help business execute or, alternatively, as a set of underpinning business services with service level agreements and requirements. Companies that start off thinking about the problem –  what they are trying to deliver –  generally do a better job than those who leave it all to chance.

By thinking of IT as a set of services that underpin your core business processes (selling cars or homes, banking, insurance, liquefying gas) you can start aligning your business and IT requirements and make significantly better investment decisions. Research shows that the most successful and profitable businesses have mature business processes underpinned by mature IT processes. No surprise then that here in Qatar, IT Infrastructure Library (ITIL) training is extremely popular as fast growing businesses look to grow their IT and business maturity.

The basis of ITIL Is that IT becomes a set of services delivered as standard processes with service level agreements in a structured and repeatable way. Businesses are looking to make IT repeatable, standard and reliable with defined costs and reduced risk.

So in the same way that security, cleaning, and facilities management have long been recognized as being suitable for outsourcing as a managed service, many parts of IT delivery are equally suitable. Managed storage, managed network, managed e-mail and managed data center services are common across the world. These reflect the IT outsourcers’ ability to build repeatable capability at low cost by leveraging scale and investment in process and technology.

The characteristics of a service that is suitable for outsourcing are:

• Definable by a service level
• Requirement to scale up and down depending on demand
• Benefits from delivery by a mature specialist organization with defined processes
• Benefits from volumes of scale above your own requirements

Reliable IT delivery is becoming business critical with outages often meaning that customers take their business elsewhere or employees cannot work. IT outages cost money and damage brand reputation so careful management and delivery of IT is critical. Service levels align business needs to IT delivery ensuring that the right levels of service design and service operation are put in place to avoid problems.

For businesses to truly reap the advantages that IT can provide, there needs to be this focus on service levels, outcomes and deliverables. Running IT like a business will enable IT to help businesses prosper and grow.

Read Steve’s other blog entries at The Hot Aisle.

I guess it is unsurprising that ASHRAE, the American Society of Heating, Refrigerating, and Air-Conditioning Engineers, writes standards that are about refrigeration–after all, turkeys don’t vote for Thanksgiving.

Here is the article in its entirety:

Setting efficiency goals for data centers

For the past decade, we have been working to make our data centers as efficient as possible; we now use less than half the energy to run our data centers than the industry average. In the open letter below, I am very happy to welcome a group of industry leaders who collectively represent most of the world’s most advanced data center operators. -Urs Hoelzle, SVP, Operations and Google Fellow

Recently, the American Society of Heating, Refrigerating and Air-Conditioning Engineers (ASHRAE) added data centers to their building efficiency standard, ASHRAE Standard 90.1. This standard defines the energy efficiency for most types of buildings in America and is often incorporated into building codes across the country.

Data centers are among the fastest-growing users of energy, according to an EPA report, and most data centers have historically been designed and operated without regard to energy efficiency (for details, see this 2009 EPA Energy Star survey). Thus, setting efficiency standards for data centers is important, and we welcome this step.

We believe that for data centers, where the energy used to perform a function (e.g., cooling) is easily measured, efficiency standards should be performance-based, not prescriptive. In other words, the standard should set the required efficiency without prescribing the specific technologies to accomplish that goal. That’s how many efficiency standards work; for example, fuel efficiency standards for cars specify how much gas a car can consume per mile of driving but not what engine to use. A performance-based standard for data centers can achieve the desired energy saving results while still enabling our industry to innovate and find new ways to improve our products.

Unfortunately, the proposed ASHRAE standard is far too prescriptive. Instead of setting a required level of efficiency for the cooling system as a whole, the standard dictates which types of cooling methods must be used. For example, the standard requires data centers to use economizers — systems that use ambient air for cooling. In many cases, economizers are a great way to cool a data center (in fact, many of our companies’ data centers use them extensively), but simply requiring their use doesn’t guarantee an efficient system, and they may not be the best choice. Future cooling methods may achieve the same or better results without the use of economizers altogether. An efficiency standard should not prohibit such innovation.

Thus, we believe that an overall data center-level cooling system efficiency standard needs to replace the proposed prescriptive approach to allow data center innovation to continue. The standard should set an aggressive target for the maximum amount of energy used by a data center for overhead functions like cooling. In fact, a similar approach is already being adopted in the industry. In a recent statement, data center industry leaders agreed that Power Usage Effectiveness (PUE) is the preferred metric for measuring data center efficiency. And the EPA Energy Star program already uses this method for data centers. As leaders in the data center industry, we are committed to aggressive energy efficiency improvements, but we need standards that let us continue to innovate while meeting (and, hopefully, exceeding) a baseline efficiency requirement set by the ASHRAE standard.

Chris Crosby, Senior Vice President, Digital Realty Trust
Hossein Fateh, President and Chief Executive Officer, Dupont Fabros Technology
James Hamilton, Vice President and Distinguished Engineer, Amazon
Urs Hoelzle, Senior Vice President, Operations and Google Fellow, Google
Mike Manos, Vice President, Service Operations, Nokia
Kevin Timmons, General Manager, Datacenter Services, Microsoft

Read more of Steve’s blog entries at The Hot Aisle.

I have spent the last 12 months working with some of the smartest and best funded marketing people on the planet.  I have been working with the the really big, household name, IT Infrastructure vendors. I learned lots, lots about marketing, lots about honest analysis and lots about human nature. I learned that marketing isn’t that tough to do but it can be terribly hard to do it well and be constantly effective.

Boiled down to its basics, marketing is about understanding a business problem really well whilst at the exact same time being paranoid that you are completely wrong about that business problem in every respect.  After you get the paranoia right, everything else is just process. Sure there is a ton of creative stuff that needs to get done to do it well but if you have smart people and a pile of cash, that is never a problem.

So step one in our marketing 101 lesson is understanding the problem. Marketing people sometimes call this Getting to the Insight and you have to get it right. Get it wrong and everything else you do is likely to be completely useless and may even be counter productive.

Once you get the Insight you need to work out who has the problem. Is it SMEs, Enterprises, Startups, the Medical Profession? Marketing people call that the Segment and the process is called segmentation.

Now we have the Insight and the Segment, we need to work out a set of solutions that solve the problem for each sector. Notice I said solutions, not solution. We need solutions because each segment may need a different solution to the same problem.

We are almost there. Now we need a set of Messages that explain and position the Solutions to the Segments we identified earlier.

Insight -> Segment -> Solutions -> Messages

So this is what I have been doing at ESG, helping marketing departments understand the problem–that is, get to the Insight that matters that will make the vendor successful. I have also been trying my hardest to disprove insights, often with help from research and polls. Generally as long a you can’t disprove an insight, it’s OK. Strangely enough, it is usually almost impossible to prove insights, because no one ever has complete market visibility.

These insights need to be backed up with research and customer validation–we need to be paranoid. Insights are nebulous and time bound. What might have been an amazing insight at one time won’t be that way always – once the problem is solved for the relevant segments, the insight goes away. Challenging the insights that vendors rely on to make major product investments is important and it needs to be verified constantly. IT insights can decay or morph very quickly, much more quickly than fast moving consumer goods but not quite as quickly as fashion and apparel items.

Understanding what customers think today about IT products and services is crucial but even more important is being right about how they will be thinking next quarter, next year, next decade. If we understand how insights change over time we can adjust our segmentation, alter our solutions and correct our messages. If we fail to foresee the changes, we fail to correct our messages, and we get a misalignment between customer brand perception and marketing department messaging.

The IT business moves so fast that is often a fatal mistake.

Read Steve’s other blog entries at The Hot Aisle.

Overview

Currently, enterprises adopt horizontal standardization, deploying common technologies at the storage and network layers. Storage offers block, file, and object access over a standard physical driver layer. Networks offer standard Ethernet- and IP-based physical drivers to the physical platforms.

The cloud changes this model by introducing a new virtual to physical layer into the IO layer; this adds complexity and obscures the underlying shared services, but delivers application mobility. In much the same way that (to date) enterprises have adopted a multi-vendor approach to operating system and hardware selection in order to provide support for their mix of legacy and best of breed application portfolio, today’s cloud-enabled enterprise is just as likely to need to support multiple hypervisors. This has significant implications for the selection and configuration of shared service layers.

The Changing Landscape

Legacy Windows platforms have been consolidated onto VMware farms for many years now as enterprises try to regain control over the server sprawl they built up over the last decade. VMware has been successfully deployed as an after-market fix that has enabled enterprises to execute on a physical to virtual consolidation without making code or environmental changes to their applications.

The next wave of virtualization is now starting, with new applications developed from the outset to support virtualization. This new world offers significant additional benefits to the consolidated/virtualized farms of today. IT will be able to migrate certain applications, live, between servers and within a data center without user impact. This offers the prospect of compute on demand with dynamic provisioning, improved availability (migrate on failure), and greater operational flexibility.

Key Technologies

The key virtualization technology here is the hypervisor and there are a number of competing platforms.

• VMware ESX
• Citrix Xen
• Red Hat KVM
• Microsoft Hyper-v
• Virtual Desktop Infrastructure (VDI) – [Xen, Hyper-v, or VMware]

Technology Diversity

From an end-user perspective, there are four conflicting drivers for the choice of hypervisor technology.

• Oracle VM (which uses Xen) is the recommended and fully supported virtualization platform for the 4,000 Oracle applications on the market and is likely to receive significant support from CIOs.
• Hyper-v is the recommended and fully supported virtualization platform for Windows platforms and services from Microsoft.
• VDI often leverages Citrix Xen, but can also use Hyper-v or VMware at the discretion of the integrator.
• VMware has a large install base of consolidated platforms.

Adoption of a single hypervisor is becoming an increasingly unlikely prospect for the enterprise, so developing a strategy and technology set that will fully support disparate hypervisors across the shared services layers of network and storage is essential.

What Are We Trying to Achieve?

Given that we must support multiple physical platforms and hypervisors simultaneously, we need to determine at the outset what is desirable and what is essential in our cloudscape.

Desirable. It might be desirable to transparently migrate applications between hypervisors (inter-hypervisor), but this is unlikely to be an immediate and essential requirement even if it were possible to achieve at present.

Essential. It is certainly essential to be able to migrate applications within a hypervisor environment (intra-hypervisor) as this enables load sharing and operational flexibility as well as an opportunity to improve availability and business continuity. Today, this is possible (with varying degrees of sophistication depending on the hypervisor), but it is dependent on fully functional virtual IO capability at the storage and network layers.

What Do We Need at the Network Layer?

Modern network ports are configured with more than MAC and IP addresses. It is common to configure QoS (quality of service), VLAN, and ACL parameters; these switch configuration parameters must be maintained as applications are migrated from one physical server to another server connected to a different switch port.

Uniquely, Blade Network Technologies provides support for all of the important hypervisor technologies, including VMware VMotion, Xen XenMotion, and Microsoft Hyper-v Live Migration. Its VMready is switch-resident software that provides the ability to transparently migrate a virtualized machine from one physical server to another whilst maintaining all of the essential quality, connectivity, and security settings.

In fact, VMready delivers against the desirable and essential network layer requirements for live virtual machine migration, both intra-hypervisor and inter-hypervisor. All that’s needed is to have the hypervisor capability in place to make live inter-hypervisor migrations a reality.

What Do We Need at the Storage Layer?

At the storage layer, SAN addresses (also known as WWN – World Wide Number) are virtualized across all major hypervisor platforms using N_Port ID Virtualization (NPIV). NPIV enables a single physical host bus adapter (HBA) to log in to a switch multiple times. The hypervisor then manages the connections between virtual HBAs in the guest operating system and these virtual NPIVs.

The benefit of this standards-based approach is that zoning, LUN masking, and QoS  are all based on the WWN so as long as the guest operating system maintains the same WWN migration between physical servers. The key is that all HBAs and all SAN switches must support NPIV; SAN controllers however, treat NPIV-created WWNs transparently. The hypervisor must also be NPIV-aware so that it can manage the virtual fabric ports. In the case of VMware, it is important to recognize that RDMs should be used in preference to the VMFS file system so that LUN masking, zoning, and QoS functionality can be related to specific VMs rather than to all VMs in a cluster. In addition to mitigating the security complications and risk that can arise from a configuration error if all VMs in the cluster share the same storage, VM-related storage frees storage/VM pairs from being restricted to a pool of clustered storage, all having to share the same values.

Analysis

It is likely that enterprises will choose to support multiple hypervisors within their private clouds and perhaps others in the public cloud, posing support and integration problems. How are we to deliver compatibility between multiple hypervisors and an existing legacy physical estate whilst maintaining our horizontal standardization in the storage and network layers?

Network layer virtual machine migrations are not just about moving IP and MAC addresses; they are much more significant and complex. The network switch needs to be included in the migration—unless you are interested in running in lowest common denominator mode with no QoS, VLAN, or ACL parameters.

Choosing the right open technologies and the best fit configuration options will enable enterprises to leverage shared services layers across multiple hypervisors.

The Bigger Truth

The second wave of virtualization is starting. It is essential that organizations select technology directions at the shared services layer that are open enough to support live VM migration. Blade Network Technologies, through VMready, has recognized that the point of network connectivity is now the application and not a MAC or IP address. By taking this approach, Blade Network Technologies has been able to deliver a flexible and universally applicable solution for highly virtualized multi-vendor platforms.

via ‘Kill’ Applications? Sorry, I’m not convinced | ZDNet UK.

Smart CIOs understand applications need to be treated differently depending on their value to the business. There are four key types: invest, operate, contain, and kill.

‘Invest’ applications generally make up around 10-15% of the full estate. These are the applications the CEO knows about–the ones that when they get better, faster, or more functional have a direct impact on business value. Examples are key CRM systems or MRP platforms, applications that underpin vital business processes and touch customers. These applications are not terribly cost sensitive, so when CTOs look to virtualise to take out cost, CIOs resist. Virtualisation is useful for invest applications, but only if it improves agility, increases speed of deployment, adds functionality, or reduces risk.

‘Operate’ applications represent 40-80% of the estate. No matter how much better we make them, they don’t improve business performance. Examples might be e-mail or document management, internal HR systems, or archiving systems. They need to be reliable and cheap. Virtualisation works here as a method of taking out costs. So does outsourcing and software as a service (SaaS) delivery.

‘Contain’ applications are those we wish we didn’t have–old stuff that’s expensive to run and difficult to change or manage. We get the amount of these we deserve: under-invest and the category grows. They have one other characteristic: they are difficult to re-platform and change to ‘operate’ status. Although they’re important to the organisation, they don’t typically make the business run any better if we improve them. We just want them to run silently for as long and as cheaply as possible. ‘Operate’ applications that have not had proper investment, love, and attention will eventually move into this category.

‘Kill’ applications are always a nightmare. These are the ones that are impossibly expensive to run and maintain. By definition, they only represent a tiny handful of the estate (perhaps 1-5%). They are impossibly difficult to change. Often, the guys who wrote and maintained the code are retired (or dead). No one else you know still has the hardware, except the Natural History Museum, and the vendor no longer supports the operating system. These might have been ‘contain’ applications that just wouldn’t stay contained or ‘invest’ applications where you didn’t invest (silly you). There is only one thing to do with a ‘kill’ application: bin it. You know it’ll cause pain and disruption, as well as costing a lot of money, but it has to be done.

Smart CIOs know this already and take a pragmatic approach to their applications, understanding instinctively where to spend money and where to bleed a previous investment. And really smart CIOs never reach the point where they need a kill category.

Read more of Steve’s blog entries at The Hot Aisle.

I have been offered a fantastic opportunity to lead an enormous IT undertaking with the Qatar Foundation in Doha to create a visionary 21st century cloud IT operation for the Middle East and North Africa.  I will be leading that effort directly and, as a result, I will be unavailable to take briefings or to provide direct services to clients for the foreseeable future.  Please continue to leverage the expertise and talents of the other fine ESG analysts in this regard.

I will continue to keep you updated on our progress, challenges, and thoughts on data centre and IT operations via the Hot Aisle and the ESG site.

Read more of Steve’s blog entries at The Hot Aisle.

A clunker starts off with a ceiling height that is too low for hot air to separate out and migrate towards the CRAC units without too much mixing. The plenum under the raised floor is shallow and clogged with cables and other detritus choking off airflow from the CRACs. The floor tiles are perforated and have low airflow characteristics. The cabinets are all lined up like a schoolroom: front to back to front to back. You could cook turkeys in the back row. The CRAC units are low capacity and that capacity is exhausted. Naturally, the boss wants you to install some 10KW racks in a hurry for a critical business project.

What can you do?  Say “no way?” Offer a co-location option in a commercial facility as an option? Start looking for a new job?

I bumped into a possible solution a few days back on Twitter when I connected with Mary Hecht-Kissell (@PR_Strategies) who looks after Coolcentric. The problem set, defined above, that makes a clunker data centre is all about getting enough cold into servers to remove the excess heat. Every element in the clunker conspires to make delivering more cold air virtually impossible. That’s where the Coolcentric solution makes a difference. It delivers cold water right up against the servers. It adds additional cooling capacity that enables that set of additional 10KW (or more) racks to be installed in a data centre that seemed like a lost cause. It’s a fairly simple piece of technology that has been well engineered to be retrofitted to most types of existing cabinets. It’s a water-cooled door.

The water-cooled door is fitted onto the back of the rack so that the hot air exhausting out of the cabinet gets chilled immediately and very efficiently. Liquids are about 4000 times more efficient at removing heat from a server than air, so these water-cooled doors can remove significantly more heat with very low pumping energy.

One smart way to think about it is that the water-cooled door acts like a mini, contained hot aisle for environments (like our clunker data centre) where cabinet alignment, roof height, and plenum problems make hot aisle containment impossible.

Sounds like a pretty decent alternate to semtex!

Read more of Steve’s blog entries at The Hot Aisle.

Mike is an industry veteran, having been through the normal form, build, and sell cycle a number of times–most notably with Illustra into Informix and SleepyCat Software into Oracle Corp. During the 3 quarters following the A-round, Mike has been able to build a credible and valuable company that adds significantly to the Apache Hadoop distribution, without funded competition.

In that time, Cloudera has built a 30 person firm–the Cloudera Distribution for Hadoop–built a support and professional services capability, and created a training and certification business. He also made the very smart move of recruiting Doug Cutting out of Yahoo, the original author of the Hadoop system.

Hadoop’s initial use case has been in managing and processing Internet scale web at Yahoo and Facebook, but is now seeing significant levels of interest in other markets where processing large scale data in real time is a competitive advantage. These include financial services, government security, credit card fraud, genomics, digital media (3D), and national scale telecommunications firms.

Mike sees the development of Hadoop as a platform as the key area for now, with enhancements to add enterprise level management capabilities “to avoid the need to have a team of Stanford graduates to hand crank the system.” To address just this issue, they have developed the Cloudera Desktop that enables the centralised management of internal and public Hadoop clusters.

Later, Mike expects to see ISVs delivering enterprise solutions based on a Hadoop platform that enables hitherto impossible feats of analytic prowess. Early enterprise adopters in this space are likely to be able to leapfrog the competition with smarter decisions and more insightful products and services that serve customers better.

Nice guy, smart company, killer product.