Research firm MarketsandMarkets forecasts that the software-defined storage (SDS) market will be worth $22.56 billion by 2021, representing a compound annual growth rate (CAGR) of 36.7 percent. This surge in adoption is being fueled by the growth of cloud-based infrastructure and virtualization technologies and today’s massive increase in data volume.

SDS is attractive to organizations that need to accommodate this growth and the consequent huge increases in storage capacity at an affordable price. However, as adoption of SDS grows, it is important for organizations to be knowledgeable about current options.

Multiple hazards await in today’s SDS market, including compatibility problems, inconsistency issues and lack of features. Any one of these, let alone a combination of them, can create problems. Below are recommendations for what to avoid and what to look for to deploy a SDS strategy that best serves your organization’s needs.

Why Unified Storage?

A unified SDS system that brings all the “flavors” of storage together will help you get the most out of your solution. For instance, if your current file-based storage system offers support for object store as well, it can save the hassle of managing and balancing many different complementary storage systems. First, this unified approach is easier to manage, and second, it makes better and more efficient use of resources in relation to performance and capacity. It’s similar to virtualization, where you cut back on hardware resources that are idling. By using a unified approach, you are using your resources more intelligently.

That seems logical, right? However, such an approach is harder to find than you might think. Some software-defined storage companies claim to offer flexibility and the ability to meet enterprise needs with object, block and file storage, to be both hyper-converged and hyperscale, and claim to support flash storage. However, many lack the features to back up those claims.

Instead, many SDS options are narrow in scope. They often focus on one use case, such as, object storage, archiving, hybrid cloud, scale-out file systems, hyperconvergence or SAN.

A focused niche SDS strategy does have an attractive price tag, typically about one-third the price of more comprehensive solutions. But you get what you pay for; they also have one-third of the features. In addition, they are not focused on general-purpose NAS.

Consistency and Why It Matters

A general-purpose NAS that scales well is what most organizations need. But just as with SDS, not all NAS solutions are created equal.

Many enterprises do not understand that consistency is critical in scale-out NAS. Some storage environments are only eventually consistent. This means files written to one node are not immediately accessible from other nodes. Even when the other nodes have been updated to record the change made to the original node, a delay of just fractions of a second can cause problems with accessing applications or users. This can be caused by not having a proper implementation of the protocols, or not tight enough integration with the virtual file system.

There is another way to do it, though: strict consistency. Being strictly consistent means files are accessible from all nodes at the same time. The view of the file system through each node is strictly consistent, so that any modification on one node is instantly available from any other node. Make sure that your solution can be consistent between protocols as well. That means if you write something in SMB, for example, it should be immediately visible over NFS as well.

Elements of a Comprehensive SDS Approach

In addition to the need for a strictly consistent NAS and a unified solution, this is what a comprehensive SDS option looks like:

  • Hardware-agnostic: By bypassing lock-in to a specific vendor and/or technology, you can use standard commodity storage hardware and servers. You can add additional hardware of your choice as needed to scale performance and capacity over time.
  • Hyperconverged: Architecture that is software-based integrates compute, storage, networking and virtualization resources and other technologies on a commodity server.
  • Scalable and flexible: With SDS, it’s possible to start small and later rapidly add multiple virtual machines to the same cluster, eliminating the cost and hassle of building new clusters in order to accommodate scale-out. If a storage cluster is built on a symmetric architecture, linear scaling up to hundreds of petabytes and billions of files is possible, simply by adding more storage nodes to the cluster. Adding storage nodes and increasing capacity can be carried out during runtime and does not interrupt any ongoing operations in the cluster.
  • File systems: You need file storage to manage unstructured data. Make sure your SDS setup includes crucial file features such as tiering, quota, snapshot, encryption, antivirus, WORM and retention. It should also be able to integrate into Microsoft Active Directory, have support for multiple authentication providers and enforce authorization checks. If your company is a large one, ensure that the solution has support for multi-tenancy, where you can create multiple file systems in the same environment.
  • Hybrid cloud: If you have a local presence and a presence in the cloud, part of your data needs to live in and be accessed from tAhe cloud as well. For example, part of your local storage system will be exposed to virtual machines running in a public cloud such as Amazon. That means your SDS file system needs to cover both environments, so you can easily pass files between them.
  • Disaster recovery: You can protect each of your applications with a unique disaster recovery policy and remain highly available if you choose an SDS solution with a storage cluster that is important to back up.

Find What Works

Enterprise storage capacity needs to scale – and quickly – as data volume continues to balloon. SDS is an approach that offers rapid scalability for a reasonable price, but buyer beware. Many SDS providers promise the world, but actually only offer limited feature sets within narrow use cases. Enterprise storage needs are well served by an SDS solution that uses general-purpose NAS and offers compatibility and consistency. Vet providers thoroughly to make sure they meet the criteria listed above so that your organization doesn’t experience disappointment after the purchase.

About the Author

Stefan Bernbo is the founder and CEO of Compuverde. For 20 years, Stefan has designed and built numerous enterprise-scale data storage solutions designed to be cost effective for storing huge data sets. From 2004 to 2010, Stefan worked within this field for Storegate, the wide-reaching Internet based storage solution for consumer and business markets, with the highest possible availability and scalability requirements. Previously, Stefan has worked with system and software architecture on several projects with Swedish giant Ericsson, the world-leading provider of telecommunications equipment and services to mobile and fixed network operators.