The trend toward “consumerization” marches onward in IT; more and more end-users are choosing their own hardware platforms and software applications in lieu of the IT-sanctioned business tools provided by their companies. These end-users are looking to tackle issues like data sharing, portability, and access from multiple intelligent endpoint devices, creating a conundrum for IT as it needs to balance business enablement, ease of access, and collaborative capacity with the need to maintain control and security of information assets.

This report looks at SaaS offerings focused on sharing and collaboration and purposely excludes those services focused primarily on data protection and backup.

The full report–including all data, cross-cuts, and market analysis–is available to qualified business and IT users who can register for free on our site, contact us for subscription information, or contact me for more info.

An overview of the 36-page report, demographic information, and a rough respondent profile follow.

Findings

• E-discovery pain is most acute and actionable for higher-revenue, serial-litigant enterprise companies
• Most corporations are not tracking e-discovery spending or the productivity or efficiency of document review from law firms.
• Enterprise litigants are pressuring law firms for alternate fee arrangements, cost itemization, specific tools, and methods
• Corporate information governance and litigation readiness (especially defensible deletion) are priorities, but not yet realities.
• Enterprises use diverse technologies for litigation response, moving toward platforms, search, and ECM systems
• Data growth, dispersion, and new formats pose challenges to collection and preservation
• Defensibility efforts are focused on notifying users and process supervision

Demographics

Respondents represented the legal departments of 48 mid-market and enterprise companies of 500 or more employees, across a variety of industries. 75% were General Counsel, in-house attorneys, or C-level legal executives

Respondent profile

• 58% had revenue of $1BN or more in 2010
• 50% had 5,000 or more employees
• 52% had over 50 matters in 2010
• 53% of those tracking e-discovery spent $1M or more on it in 2010
• 62% spent $1M or more on outside counsel in 2010
• 48% expected ESI to be involved in 40% or more of 2011 matters
• 46% saw a year-over-year increase in legal and regulatory matters from 2009 to 2010

We’ll have more previews of more report highlights to come. Again, for full access to all data and analysis, qualified business and IT users can register for free on our site, contact us for subscription information, or contact me for more info.

On the product side, it will mean some serious portfolio rationalization.  In archiving, Autonomy possesses its Zantaz archiving line, including Digital Safe, the acquired Information Governance assets of CA, and now Mimosa following the recent acquisition of Iron Mountain’s digital assets, while HP has its own Integrated Archiving Platform.  In records management, HP has TRIM where Autonomy has Meridio and iManage content management from its acquisition of Interwoven.

But like the Brady Bunch, this group must somehow form a family.  More details on the earnings call today.  Hold onto your e-discovery hats!

This is particularly true as legal work has shifted in-house to Corporate Counsel.  Many enterprise tools in e-discovery today have been re-positioned from roots in data management, archiving, search, or forensics. They’re often not legal-friendly, not designed for legal workflow, and are still generally aimed primarily at IT stakeholders who live close to the data.  Confusingly, all of these tools perform “Early Case Assessment”–a step you will not find on the Electronic Discovery Reference Model (EDRM).

“What exactly do these tools do,” you may ask. It’s a good question. But a better one to start with is: “what do in-house attorneys do, and how do they need to do it?”

Initial Case Assessments: the attorney’s definition

Our latest e-discovery report starts by examining the attorney use case, based on ESG’s conversations with General Counsel and their descriptions of handling incoming legal and regulatory matters–a process many (included EDRM.net) refer to as “Initial Case Assessment.”

An initial case assessment as practiced internally by legal departments facing legal, regulatory, or high-profile internal investigations often includes some or all of the following objectives:

• Evaluating the claims and merits of the case
• Understanding the subject matter, topic, people, history, and time period involved
• Determining the known and potential unknown facts of the matter
• Reviewing relevant case law, precedent, and jurisdictional issues pertaining to the matter
• Estimating potential cost exposure, including e-discovery parameters and costs
• Weighing the risk and consequences of an unfavorable outcome
• Developing preliminary defense options and case strategy
• Establishing and provisioning resource assignments

Early Case Assessment: the software market’s definition

“Isn’t this the same as Early Case Assessment,” you might ask.  Not necessarily, since ECA has become more of a software marketing catch-all for accessing, investigating, analyzing, filtering, and culling data. ECA is often positioned for the purpose of culling or deduplicating irrelevant data, towards the significant potential ROI of limiting downstream review and production costs.  Yet its popularity has led to confusion, as ECA holds no concrete location in the legal workflow or the EDRM, and ECA capabilities are applied to a broad set of tools in the market, including:

• Data management, forensic investigation, or search solutions
• Specific ECM systems or archives to export, search, and filter data from those tools
• Appliances for processing, filtering, and uploading data to review tools
• Middleware overlays between repositories, e-discovery systems, and case or matter management
• Attorney review and analysis tools
• …and many applications and data environments in between

ECA is not a concrete step on the EDRM, but increasingly seems to span all of them.  It has roots in the searching, filtering, and navigation attorney tools typically used in the review stage, but as e-discovery tools have evolved (particularly for in-house use), ECA capabilities seem to move earlier and earlier in the process.

Harnessing technology to meet your needs

But back to the original question–what do these tools do? More importantly, how can they help attorneys do their jobs better, and help businesses create a more repeatable and even predictable process for litigation and regulatory response?

Electronic discovery technology plays several roles in evaluating a case from an early stage and honing a strategy to pursue it. Most importantly, it offers the ability to access and analyze potential evidence. But many tools can also help filter and cull data, search and navigate it more intelligently, and forecast estimates for project cost, scope, and potential timelines, which help in determining discovery burden and negotiating eventual production requirements.

ESG’s latest Market Landscape Report examines:

• The attorney’s process of Initial Case Assessment.
• The different capabilities of tools on the market for performing it.
• The current vendors and solutions available in the market.
• The organizational business goals and technical requirements in selecting and integrating e-discovery solutions, plus the challenges to adoption.

The goal of this Market Landscape Report is to help corporations, in conjunction with their external law firms and service providers, expand their organizational “initial case assessment” processes to incorporate e-discovery technology, and include insight gleaned from electronically stored information in their case planning, budgeting, and strategies–all with the aim of transforming e-discovery into a more defensible, repeatable, and even predictable business process.

This report is available to ESG Premium Subscribers on our website here with login. Free ESG website access is also available to qualified IT and Business Professionals with registration.

At EMC World 2011, the company opted to build (not block) momentum, breathing new life into Documentum and the rest of its Information Intelligence Group (IIG) family with several new announcements including:

• OnDemand versions of the IIG portfolio in the works, built on a VCUBE architecture available interchangeably on- and off-premises.

• A new social collaboration platform integrating Documentum with Cisco Quad to link content management with collaboration and compliance.

• Partnership between Box.net and Documentum, providing more mobility and consistency for users in the cloud, along with unified retention capabilities.

• A new iPad app for Documentum users, with plans to unify the UI for more IIG apps over time.

• A new partnership program that lets partners build, sell, deploy, and support solutions created with IIG products.

Here, we see EMC ready to “meet and raise” in the “post-PC era” game of IT consumerization and choice computing, while lowering the barrier to actually running the software through new cloud delivery models–a critical piece seemingly absent last year for IIG as the rest of EMC began its journey to the cloud.  Pricing was not yet revealed; this will be a fine line for the company to navigate between increasing customer adoption with traditional “cloudy” subscription pricing and maintaining its own bottom line.

A new social collaboration partnership between Documentum and Cisco Quad goes further in attracting the “new user” through a platform for building new business applications with collaborative workflow. Not bad, considering that Documentum has classically been a repository-driven solution; in fact, it was originally released without any interface when it first emerged in the 90s. Could it be that IIG is doubling down against SharePoint only a year after trying to play nice with integration for SourceOne at EMC World 2010? Collaboration tools (including SharePoint) ranked second only to content management in ESG’s enterprise information management spending trends for 2011–here users get more of both, plus compliance.

Overall, EMC World 2011 saw IIG getting more of the recharge promised by last year’s name change and management shifts. CEO Joe Tucci and heir apparent Pat Gelsinger have already given EMC’s storage business a jolt with multi-billion dollar deals for Data Domain and Isilon. IIG is starting to look more innovative, building bridges with other EMC groups and partners while forging its own identity.

• Pictures are worth a thousand words – think of the recent arguments that photos from Google Earth contributed to protests in the Middle East.

• They’re also worth millions of data points–Hans Rosling’s statistics demonstrations using Trendalyzer show how data visualizations are changing how we ingest information, how we think about the world, and how we educate people.  The tool is now available in Google Docs as the Motion Chart.

Life after Text

I wrote last week about changing business and technology for a post-paper world, but the reality is that we are not just post-Paper, but post-Text.

“People are becoming less text-based altogether,” my mother told me the other day. She’s an academic librarian, and had heard a talk from speaker Stephen Abram, who says we must become “nextheads” not “textheads,” because “the dominance of educated persons who decode and internalize ‘text’ well is ending.”

Software of all kinds is adapting to a less text-based user, and a less-technical business user in many cases.  The rise of “search-based applications” means we’ve at least surpassed the “Google it” stage of applying technology to our needs.  And we’ve seen other usability gains through better design and workflow, better search, more accessible charts, visualizations, and dashboards for business users as well.

Instead of “reading the manual,” users now expect technology interfaces (and their designers) to “read their minds,” anticipate their needs, and even respond to their touch or presence.

E-discovery–new usability for the legal team

This brings me to e-discovery–which is frequently one of those search-based applications.  Attorneys have begun embracing new technology and demanding better usability, seeking e-discovery tools with more “legal-friendly” interfaces.  The goal is more speed in execution, and more legal self-sufficiency without IT’s intervention in the process.

But because so many e-discovery tools were re-purposed from IT-focused offerings like archiving or forensic collection, redesigning their interfaces and workflows for business users in a legal context is a nontrivial undertaking for vendors.  And legal users too often get the blame for all this, for being “non-technical” or even “technophobic.”

ESG is at work now on our Market Landscape report on Initial Case Assessment, a term attorneys use for analyzing the cost, risk, and merits of a potential case.  Our topic is e-discovery tools and features targeting legal users, particularly General Counsel as part of an in-house process.

We’re not post-text, but there are much better dashboards and visualizations emerging, as well as better workflow, integration, and configuration options for setting up projects.  The legal field, once accustomed to paper evidence, shelves of recorded precedent, and the billable hour, is now moving toward low-click usability and project management dashboards.

Admittedly, the “legal-friendliness” of these tools is sometimes still more marketing than reality–note that the legal use case of “Initial Case Assessment” is often different from the Early Case Assessment software marketed to them in the last few years–but things are going in the right direction.

On that note, I realize the hypocrisy of blogging about this without any pictures, and will try to be less of a “texthead” myself in the future.  Change is good!!!!!!

Records managers have really found their seats at the table in the last year, specifically the e-discovery and information management RFP committee table. Records managers struggle with the digital deluge, too, including more varied data types, greater volumes of information, and a stricter regulatory environment–not to mention the impact of e-discovery investigations, legal holds that preserve potential evidence, and broader efforts to defensibly decommission data in order to minimize future potential evidentiary productions.

The ARMA conference last year in particular seems to have put the fear of God into the rest of the world.  It’s rewarding to see as someone from an information science background (yes that means a library degree to the rest of you).  In the last few months, I’ve been in several product management discussions with vendors on whether to integrate with records management tools or incorporate some native capabilities around RM in e-discovery and information management products.

• How important is RM functionality to e-discovery, should it be integrated, and how?
• Should policy and retention be globally automated or at the user’s discretion?
• Does affordable, scalable indexing and culling of large data volumes in processing tools solve the data volume problem?
• Or will better information management be the in-house e-discovery “killer”?

The answer to a lot of these questions is “it depends”–much of it rests more on process and customers’ individual business requirements than on specific technologies. But what’s clear is that customers and vendors have heeded the call, and are becoming more serious about getting their houses in order.

Defensible deletion–and a future beyond paper-based retention

Another big topic that ties into this discussion is defensible deletion. PSS Systems has been carrying this banner for some time. Several other vendors are tackling customers’ legacy data through classification and data management tools, identifying the contents, applying policy, and enabling them to defensibly delete unneeded data en masse. The “Data Overload” issue even got some more mainstream coverage in the New York Times this weekend through an Op-Ed by Shelley Podolny of H5.

These are all good housekeeping practices for the mess in which we currently find ourselves. But in some ways, we’re once again treating a symptom rather than an illness. The interesting question to me is one that came up at the IQPC conference I wrote about a few weeks ago: when (and how) will we move to information management systems that aren’t based on paper-oriented office processes which pre-date computers?

Social media: less retention or just more difficult retention?

The growth of collaboration software could encourage this for some eventually as users move away from collaborating primarily with traditional e-mail and  retention policies adapt. Interest is there among buyers: collaboration software was one of the top predicted sellers in ESG’s recent 2011 IT Spending Trends survey. And as social media evolves, we may find that many of our messages don’t need to be collected like e-mail is, but can self-destruct after a certain number of days or never be archived at all like many of us use synchronous communication media such as IM, chat, or phone calls.

It’s not a short-term or universal prescription. Businesses in highly-regulated industries won’t have this luxury, of course – finance and other verticals already must archive data of all kinds related to their business and monitor social media usage. And things may be getting worse before they get better as people learn to use these tools more effectively. Collaboration tools, like so many others, often enter the market with less attention paid to retrieving data from them and more to just getting people to use them and quickly put more data into them to boost stickiness. The implications of having to produce any data  for litigation don’t occur to users and buyers immediately–sometimes not until they’re going to spend considerable time and money to do so.  Social media’s new formats can make the collection struggle even harder: the complexities in collecting from SharePoint for e-discovery are a good example. Facebook, on the other hand, recently gave users the option to download their information directly, vastly simplifying the process of producing it.

The nuances of the problem are probably why there are so many “social media” consultancies around and why customers have such wildly divergent policies on using it from “never ever” to “the same as a phone call–just another means of communication.”

Changing behavior

In the long term, for those of us not in highly-regulated industries, do we really need to hoard all of the “stuff” that we do? We all know that changing behavior–particularly at an organizational level–is much more difficult than simply changing technology. But things are going to change one way or another, now that the smoking gun is in e-mail, IM, chats, mobile phones, and every other format and data source. It may be that we say less in e-mail or start linguistically coding our correspondence to avoid “high impact emotional content” which can now be detected through sentiment analysis tools like Lymbix (whose “ToneCheck” technology is soon to be integrated into Lotus Notes, according to Mashable). Or could there be a retro backlash in favor of less-monitored face-to-face or phone contact? Or will we just keep pushing the borders to go “off the grid” by other means through other outlets we hope can’t be monitored yet?

It may seem like a pipe dream at this point, but change will come. For many, it already has.  The broader public Internet used to be a wild west of anonymous venom-spitters (OK, it still functions that way to some extent) and now people carefully “build an identity” for themselves online, crafting what they say so they don’t irreversibly damage their reputations. Business correspondence, like the Internet, is now “written in ink” (and massive, costly data volumes), and we need to think of different ways to use it.

Overview

In early 2009, ESG surveyed nearly 1,200 organizations to better understand their plans to leverage Microsoft SharePoint. Two major themes became apparent: first, most respondents were just beginning to understand the potential of the fastest growing application in Microsoft’s history. Second, nearly everyone knew it was going to have a major impact—over 80% of respondents that had started or were planning on using SharePoint said that implementations would be a top ten IT initiative over the next two years. Those 24 months passed very quickly and SharePoint has lived up to its billing: for many, it is an extension of Microsoft Office deployments, a real-time content management and workflow automation system and a web platform (intranet, extranet, portal, etc.) that includes social media capabilities. Without question, SharePoint is a powerful platform that can be used to help address a wide range of collaboration and information sharing needs.

No matter how it is used, SharePoint is frequently the source, and ultimate repository, of critical information. This, along with ongoing new implementations and growth of existing deployments, makes the underlying storage environments extremely important. Storage cannot and should not be a basic IT infrastructure element supporting SharePoint—it needs to help IT departments cost effectively enable and optimize this powerful platform. Organizations need a storage solution that can add the most value to their SharePoint environments.

In a 2009 white paper, ESG discussed some of the key considerations organizations need to be aware of when making a SharePoint storage decision and presented the case as to why NetApp solutions should be part of the evaluation process.[1] The latter was supported by NetApp’s ability to scale performance and provision in real-time as SharePoint farms grow in capacity and  integrated data protection offerings that enable IT to create a rapid, consistent point-in-time snapshot of the entire SharePoint infrastructure (content databases, web front-end servers, etc.) to facilitate fast backup and restore processes.

While SharePoint still continues to grow and must be adequately protected, its use has rapidly evolved—as have NetApp’s capabilities to enable it. Current market trends are creating the need for even more functional storage to optimize SharePoint environments; NetApp has a variety of offerings that make potential improvements a reality.

What’s New – SharePoint Market Trends

Capacity Growth

ESG research suggests that the average SharePoint user was experiencing annual growth of 25%, with 40% of organizations reporting greater than 1 TB of data in their farms. Keeping in mind that a majority of these deployments were still relatively new (less than three years in production), these statistics were merely a baseline poised to increase as more users began taking advantage of SharePoint’s capabilities.

As a forewarning to expected usage increase and subsequent capacity growth, Microsoft issued best practices for sizing a SharePoint 2007 content database. The recommendation was to keep this SQL Server component to 100 GB or less to prevent performance degradation and ensure backups could be successfully completed. Microsoft then introduced External Blob Storage (EBS) to help users store data outside of SQL server without compromising access or performance. In such a deployment, a document saved within SharePoint is stored in a file system while the document’s metadata resides in the content database. Alternatively, IT could continue deploying new content databases when approaching the 100 GB threshold to avoid the inevitable performance and data protection issues resulting from rapid growth. [2]

In summary, less than 24 months ago, SharePoint was new to many organizations, but most, including Microsoft, knew that it was going to be a cornerstone of productivity. As a result, it was going to create the same IT operational challenges—specifically rising storage costs, poor performance, and elongated backup and recovery times—that any fast growing application does. And Microsoft provided forewarning and configuration suggestions to address the issues.

SharePoint 2010 Adoption

The biggest driver of capacity growth was the extension of SharePoint to more users who leverage it for more functions.  This is being accelerated by the adoption of SharePoint 2010; the latest release more tightly integrates with Microsoft Office 2010, making it is easier to publish content directly to a site. It also includes user interface, collaboration, workflow engine, and social media enhancements that make it much easier to design sites for internal and external use as well leverage the platform for more critical business process automation functions.

Once again, Microsoft is predicting (and ESG strongly believes that this will prove to be accurate) that the feature/function richness of SharePoint will create a massive usage increase and thus a(nother) capacity explosion. To combat this, Microsoft made some architectural changes to improve SharePoint’s scalability. However, Microsoft is also recommending that SharePoint 2010 users limit their content database size to 200 GB for performance and data protection reasons.[3]

The good news is that Microsoft also made it easier to store documents or binary large objects (BLOBs) outside of the content database by introducing Remote Blob Storage (RBS). New to SharePoint 2010 and SQL 2008 R2, RBS differs from EBS in that RBS can utilize FILESTREAM, a mechanism that will automatically store the BLOBs outside of SQL Server in a predetermined, local file system that is still part of the SharePoint environment. RBS also offers integration points so that an external solution (storage system, content management system, etc.) can execute BLOB storage using its own file system rather than FILESTREAM—in this implementation, BLOBs are stored and managed outside of the SharePoint environment.

When selecting an RBS deployment option, SharePoint users have to consider how easy the implementation is to manage and the potential benefits an external (non-SharePoint) system brings to the overall implementation. Whether a customer uses FILESTREAM or an external vendor’s RBS integration, the BLOBs are still accessible from a SharePoint interface and the SQL Server content database remains small, allowing it to deliver the necessary performance and still be quickly backed up. RBS deployments using an external solution as opposed to FILESTREAM are likely to boost performance because the BLOBs are actually saved outside of the SharePoint environment.

The introduction of SharePoint 2010 did not really solve many of the capacity challenges users face, it merely shifted them. IT departments should expect to store data outside of the content database—potentially outside of the SharePoint infrastructure—or have several large content databases to manage across their farms. Both options introduce storage management and data protection concerns.

Maximizing the Investment

As organizations begin to standardize on SharePoint 2010, they have an interesting information management decision to make regarding legacy data. There are terabytes upon terabytes of file share data sitting inside of organizations; this information, or a subset of it, would likely better serve the company if it were migrated into a specific SharePoint site where it can be managed alongside new content. Another likely scenario involves the desire to move from an existing content management system to SharePoint. This type of migration becomes even more difficult as the documents in a legacy repository are likely stored with security permissions that limit information access.

ESG has spoken with several organizations that are grappling with two big SharePoint migration challenges. The first is identifying what data to move and the second is how to get large quantities of data with select metadata (access controls, etc.) from the source to the appropriate SharePoint site. Keep in mind that once these challenges are addressed, SharePoint implementations quickly become much larger, introducing the capacity concerns discussed previously.

NetApp Addresses SharePoint Challenges

A Strong Foundation

Although it is frequently discussed as a single application, SharePoint is composed of several different tiers of components—each of which can be optimized when deployed on the right storage solution. Block-based storage protocols, including iSCSI, are ideal for SharePoint’s SQL Server content database while Common Internet File System (CIFS), a file-based protocol, can be used to store BLOBs outside of the content database as well as the web front-end and index servers. iSCSI is optimal for databases because it is a block-based protocol that provides predictable latency and response times for frequent read/write transactions. CIFS (also known as Server Message Block or SMB) is more tuned to handle file-based data such as BLOBs and index files that need reasonable performance—the predictability of that performance is far less critical. Both iSCSI and CIFS utilized standard IP networking, allowing IT to leverage an existing infrastructure to optimally configure a SharePoint storage infrastructure.

Many IT departments will forgo deploying different storage system configurations because they do not want to manage disparate solutions for their SharePoint environments. With NetApp, this is not a concern because all of their FAS and V-Series systems support all protocols, allowing customers to consolidate multiple SharePoint components on a single device or run each independently. Regardless of what a customer chooses to do, with NetApp underpinning SharePoint, they will be running a storage solution(s) that has the same operating system and file system and supports all of the same solution provider’s data management and protection software.

Aside from NetApp’s unified architecture and integrated software offerings, the storage systems have many other capabilities such as thin provisioning, deduplication, compression, and space-efficient writeable clones that will improve capacity utilization, facilitate storage workflows such as creating test and development environments, and help customers control SharePoint related data growth. Users also have several hardware configuration options including the use of different disk drive types and flash technologies to maximize performance at the lowest possible cost. These choices provide the flexibility to balance storage costs with SharePoint-specific performance service levels.

All of the aforementioned capabilities can be used across any application environment, with some having more of an impact on SharePoint specifically. However, they should be weighted heavily when customers are selecting which RBS implementation option to use. They may be enough to encourage SharePoint users to use NetApp for BLOB storage.  But, for those that still want to compare NetApp’s BLOB storage capabilities with FILESTREAM, this comparison is likely to still lead to using a more functional storage system because even though it is part of the core SharePoint environment, it has some limitations. Many of its shortcomings are centered on configuration and the ability to manage BLOBs once they are stored. As an example, FILESTREAM does not have its own dedicated administrative user interface to configure the RBS deployment whereas NetApp does. NetApp’s interface also facilitates garbage collection and configuration modifications from its management user interface. And, NetApp offers several storage efficiency benefits such as tiered deployments, compression, and policy-based archival—none of which are available in FILESTREAM.

The Basics are Covered

One benefit of using NetApp’s SnapManager for SharePoint (SMSP) is the ability to instantly create a consistent point-in-time snapshot of the entire SharePoint infrastructure. Speed is critical so the snapshot process does not impact the availability of the application and the “consistent” feature means that recoveries are going to be usable because all of the SharePoint infrastructure components (content database, BLOB data, search servers, web front-end servers, etc.) are in synch at that specific point in time. It is important to note that SMSP 5.0 now supports SharePoint 2007 EBS and that SMSP 6.0, the most recent version of the solution, carries this capability forward and adds support for SharePoint 2010 RBS. No matter how or where BLOBS are stored, NetApp SMSP 6.0 can protect and manage them along with the content database to ensure fast recoveries.

Data Protection and Recovery Options

One of the unique capabilities of NetApp’s SMSP 6.0 is that it expedites backups and restores. The former gets the most attention as IT departments just want to get their backups done within an allotted window, but when something does go wrong, it becomes obvious that restoring an entire SharePoint farm can take a long time; this will not improve as the farm components (web front-end servers, index servers, and content databases) grow. SMSP 6.0 maintains a complete index of all SharePoint objects and individual items that have been backed up via NetApp’s SnapShot (or replicated using NetApp’s Snap Mirror or Snap Vault) capabilities. IT can navigate all of the backup sets within SMSP and select the particular item they want to restore and complete the process from one console. This level of recovery can be performed against sites, sub-sites, site collections, lists, document libraries, and other items. If an entire farm needs to be restored, this can also be accomplished via SMSP 6.0.

Item-level recovery enables SharePoint users to get their data back without having to wait for an entire farm to be restored. When required, though, NetApp eases the pain for farm-level recoveries because its snapshots are much smaller than full copies and, as mentioned, the process is centrally managed and executed, ensuring that the snapshot is “consistent” across all farm components. SMSP 6.0 can now also protect SharePoint web parts, which are connections to other applications or data sources.

With many organizations managing multiple SharePoint sites—some of which are test sites that will replace current ones in production because new capabilities have been added—consolidation efforts are necessary. IT can actually leverage SMSP 6.0 to drive that consolidation during the backup process. The first step is to snapshot all of the sites from various farms (test, development, pre-production) and then restore the individual items (sites, collections, list, etc.) to a different (new or pre-existing) farm. The target farm essentially takes the items from the other sites, adding them to existing information to create the desired production SharePoint implementation. This entire process is made possible and streamlined because NetApp facilitates the instant snapshots and the item-level restores to different (than the original source) SharePoint farms.

Running SharePoint in Virtualized Server Environments

SharePoint 2010 is an ideal application to run in virtualized server environments because of its multi-component architecture, and it is likely that many IT departments will consider Microsoft’s Hyper-V solution since it is part of the Windows Server 2008 R2 operating system. Virtualization using Hyper-V offers great benefits, but some of the capabilities, including Live Migration (which moves virtual machines and the applications running on them between physical servers), only work with networked (SAN, NAS, iSCSI) storage deployments. This presents an obvious opportunity to leverage NetApp’s storage solutions. Users should also keep in mind that NetApp’s SnapManager family includes offerings that can protect the underlying hypervisor without impacting performance of access when this operation is executed.

Overall, NetApp’s unified, networked storage and SnapManager solution family make it easy to protect SharePoint implementations, giving IT the option to optimize application deployments across a combination of physical and virtualized server environments without complicating data protection operations.

Optimizing Performance

To help customers manage and control the size of SharePoint content databases, NetApp has extended the Snap Manager for SharePoint 6.0 solution to include:

• The Archiver Module. In this configuration, a SharePoint administrator defines content archival policies for documents, lists, libraries, sites, etc. and a schedule to apply the archival policy (i.e., when it is run) within the SMSP console. The administrator also designates a CIFS/SMB share location on a NetApp storage system where the BLOBs will be stored after the archive process is run. At the scheduled time, the SMSP system scans a SharePoint farm, seeking items that trigger the archive policy. If an item does, the archiver agent extracts the content from the content database and stores it on the NetApp CIFS/SMB share, leaving the necessary metadata in the content database to ensure seamless access.
• The Extender Module. This implementation extends the archive process described above and applies it to any new content uploaded to SharePoint. Each time a user uploads content into SharePoint, it is checked against the pre-defined extender policy. If the policy is triggered, the upload process is split with the metadata being stored in the content database and the BLOB saved on the NetApp CIFS/SMB share.

These modules help control the size of a SharePoint content database without compromising access. EBS and RBS integration ensures that the content can still be accessed from SharePoint and all of the metadata within the content database is updated as users interact with and modify files. The net result is a much leaner, more scalable SharePoint farm without having to manage a large number of small content databases.

Getting Data into SharePoint

SMSP 6.0 includes several migration agents that enable SharePoint administrators to identify and import unstructured data, stored on existing file servers or legacy content management systems (Exchange Public Folders, EMC Documentum eRoom,  Lotus Notes, and other websites), into SharePoint farms. This process can be scheduled and centrally managed during off-peak hours and leverages the same backup and restore process as the SharePoint site consolidation method described above. First, the existing file shares and content management systems from which data has to be migrated must be on NetApp storage or a non-NetApp system front-ended by a NetApp V-Series (preserving legacy storage investments). Second, NetApp takes a snapshot of these environments, capturing existing metadata including security permissions. Lastly, the snapshot is restored to a new or existing SharePoint farm.

The Bigger Truth

It is easy to understand how NetApp addresses many emerging challenges—migration, availability, and performance optimization—that SharePoint creates while enabling users to actually use the application for greater benefit. What is not so obvious is that NetApp does this with a single product, SnapManager 6.0 for SharePoint, which runs across any NetApp networked storage system regardless of the configuration (disk drives, protocol, etc.) the devices ship with.  Users can also take advantage of SMSP (as well as any other NetApp software capabilities) with non-NetApp storage systems by deploying a NetApp V-Series in front of those heterogeneous devices.

What’s more, this writing did not cover some of the disaster recovery and other data protection configuration options customers have when using NetApp (or third-party via V-Series) storage with SnapManager for SharePoint. With its Web-based GUI management console, SnapManager enables administrators to configure and control SharePoint application archiving, data protection policies, and migrations from a single location. Yes, customers have to deploy SharePoint on NetApp storage or a non-NetApp system front-ended by a NetApp V-Series to achieve these benefits; however, the investment is well worth it because customers can actually maintain application performance, complete backup and restores (when needed), and seamlessly move data in SharePoint.

[2] http://technet.microsoft.com/en-us/library/cc298801(office.12).aspx

[3] http://technet.microsoft.com/en-us/library/cc262787.aspx

Much has been made of the benefits to bringing e-discovery in-house for corporate counsel–especially the cost benefits for those facing sticker shock from law firms or service providers.  Finance, like other highly-regulated and sometimes litigious industries, has typically been ahead of the curve in tackling the problem.

But recognizing the issue still doesn’t make it easy to solve, particularly for large organizations trying to do it themselves with an entirely new cross-functional business process. For one, it’s a learning process.  Even if you want more self-sufficiency, it’s not easy to “teach yourself to fish.”  Maybe you can’t pay for extravagant sushi dinners at a restaurant anymore–that doesn’t mean you’ll learn to roll your own sushi at home overnight or start farming fish in the bathtub.  In the meantime, you might just order a pizza and toss a salad.

Another issue is the process of gaining buy-in at the top and consensus among departmental stakeholders from  legal, IT, HR, compliance, and, increasingly, even procurement. One panelist compared the challenge of changing her organization to “trying to turn around an aircraft carrier.”

These are the kinds of choices and challenges many companies are facing as they “teach themselves to fish” with e-discovery, creating an entirely new business process that affects the whole organization.  What parts of the process should be outsourced?  What is most efficient for us to perform ourselves?  And what happens when we have to explain the whole thing in front of a judge?

Gaining Consensus

Panelists at IQPC said they were most likely to keep outsourcing episodic activities like attorney review and production. But internally, they still need technology to respond to subpoenas such as an e-mail archive with an e-discovery tool on top, like Enterprise Vault with Discovery Accelerator.  The problem some found was that IT didn’t know or care about requirements for litigation response (specifically: compatibility, reliability, defensibility, and usability by paralegals) and didn’t know about products available with suitable functionality. Security policy was another roadblock, preventing outside vendors from connecting to the companies’ internal systems. Panelists recommended having all departments involved and engaging IT early on, before purchase with a proof of concept.

Budgets

Budgets were another obstacle. Some e-discovery purchases promise quick ROI (such as a processing and early case assessment tool used on a single case).  But these kind of investments may be made at the departmental level by business users who aren’t interested in a more comprehensive solution beyond their own needs. More extensive CAPEX investments can be stymied for a number of reasons. The CEO or CIO may not realize the extent of the problem until it hits them or their own staff directly.  It may be that, internally, the company does not have the resources for a DIY approach or finding and training staff might be harder than just continuing to outsource. Other buyers are delaying adoption for the “next generation” of technology from the evolving vendor market (regardless of the money they could be saving themselves now), citing past systems migration disasters. The decision involves durability, utility, the likelihood that an investment will be “legacy” in 12 months, and broader potential applications. Ultimately, it can be easier to just push e-discovery expenses through OPEX with a consultant rather than attempting to get CAPEX investment and a full-time employee, particularly since legal is often considered a cost center within the business, not a revenue driver.

Monitoring Outsourcing

Another issue was monitoring and managing those tasks which were still outsourced, like review and production. Panelists had established KPIs around cost and deadlines for their outsourcers and outside counsel, as well as gauging the responsiveness rate on review. They warned against scope and budget creep from outsourcing, advising attendees to communicate case requirements from an early stage and monitor every material change in scope in terms of its cost and defensibility. Third party outsourcing can also raise costs; panelists said they expected their service providers to negotiate suitable rates with third parties like hosting providers. Panelists also reported engaging procurement teams in vendor selection to manage costs and consulting with “National Discovery Counsel” at outside law firms on their strategy.

“Control what you create, and what you keep”

Panelists noted that good proactive information management may be the best way to minimize e-discovery costs in the long-term by controlling what you create and what you keep, reducing the data set available to be worked on if you don’t need it.  Establishing comprehensive retention policies has been an asset to enterprises in battling the digital data deluge, although data disposition has been a more difficult proposition on a large scale. Panelists opined that the future might bring new approaches to the problem. As the workforce evolves, one noted, we can potentially develop more appropriate content management styles that don’t pre-date computers.

Overall, panelists advised having patience in dealing with the process of organizational learning and change, and perseverance in finding the right solution. “If we find something that works,” one noted, “we keep using it.”