– Stefan Bernbo, founder and CEO of Compuverde, says:

The rise of cloud computing has led many service providers to rethink their Big Data storage strategies with a renewed focus on reducing costs and improving performance. This shift is out of necessity.  If the status quo is maintained, it will become increasingly difficult for people to buy cheap cloud services.  There will be storage restrictions for everyone on the network, higher energy consumption and higher prices overall.
High Costs Squeezing Providers

Many service providers are beginning to use centralized data centers that make information accessible online to its users from anywhere. By consolidating the equipment, service providers have found one approach to reducing costs. Additionally, centralized data centers provide improved Internet connections and better performance and reliability. However, though performance improves under a centralized data center methodology, scalability has become much more difficult and expensive. More high-performance, specialized equipment is required, leading to inevitably higher costs and energy consumption that are challenging to control at scale.
Performance in the Cloud

One big concern for cloud providers, who must manage more users and larger performance demands than enterprises, is solving performance problems likes data bottlenecks. While the enterprise network’s average user demands high performance, their systems are typically serving fewer users who need to easily access their files directly through the network. Additionally, enterprise system users are normally sending, accessing or saving low-volume files such as spreadsheets or documents, consuming less storage capacity and lessening performance load.
It is a different story, however, for a cloud user outside the enterprise. When the telecommunications service provider (TSP) system is being used simultaneously by a high volume of users, it can cause a performance bottleneck. TSP systems must not only have to scale to each additional user, but must also hold the same level of performance for the entire network. All of this must happen while the average cloud user is accessing and storing much larger data such as music, photo and video files.
Scaling Costs and Storage

For cloud providers, the business ramifications of these storage demands are significant. Service providers need to be able to scale rapidly to accommodate the proliferating demand for more data storage.  Users are accustomed to free online services, and are not shy about discarding providers that put up paywalls. To be cost-effective, service providers need low cost storage that scales easily and delivers high performance to boot.
Scalability: Best Practices

Three best practices for service providers seeking a model combination of scalability, performance and cost-effectiveness are as follows:
  1. Avoid a single point of entry to the data system. If the server has only one point of entry, it has just one possible point of failure. This presents an issue, since the demands of cloud computing on Big Data storage are very high and often creates a data bottleneck. Addressing this issue and alleviating load at the point of entry adds cost and complexity to a system very quickly.  On the other hand, a horizontally-scalable system that distributes data among all nodes makes it possible to choose cheaper, lower-energy hardware.
  2. Choose commodity components where possible. Commodity hardware can make good business sense.  These servers not only cost less, but also use far less energy. This significantly reduces both setup and operating costs in one move.
  3. Opt for distributed storage. Even though the data center trend has been moving toward centralization, distributed storage presents the best way to build at scale.  This is because there are now ways to improve performance at the software level that neutralize the performance advantage of a centralized data storage approach.
Conclusion
Big Data storage architecture today is primarily comprised of high-performance, vertically-scaled storage systems. Because most of these architectures can only scale to a single petabyte and are expensive, they will continue to be more costly to sustain in the long haul. However, a horizontally-scaled data storage model that distributes data evenly onto low-energy hardware has the ability to increase performance while reducing costs in the Cloud.  By employing these best practices, cloud service providers can take steps to improve the efficiency, scalability and performance of their data storage centers. 
About the Author:
Stefan Bernbo is the founder and CEO of Compuverde. For 20 years, Stefan has designed and built numerous enterprise scale data storage solutions designed to be cost effective for storing huge data sets. From 2004 to 2010 Stefan worked within this field for Storegate, the wide-reaching Internet based storage solution for consumer and business markets, with the highest possible availability and scalability requirements. Previously, Stefan has worked with system and software architecture on several projects with Swedish giant Ericsson, the world-leading provider of telecommunications equipment and services to mobile and fixed network operators.