By Abhishek Rathi, Vice President, Opera Solutions
Buying Big Data software and services is a Big Deal — regardless of the size of your organization. The category is ambiguous: There are too many providers offering too many flavors of Big Data software and services, and implementing the wrong or inadequate tools and technologies can derail your efforts. In addition, technology and business needs are evolving quickly, so companies need a solution that is agile and doesn’t become obsolete in five years. In the same vein, once a certain path is chosen and a significant investment is made, companies may be stuck with that decision for years to come. And finally, while your organization likely knows it needs a Big Data solution, those who make the purchasing decisions may not have a full understanding of all the disciplines involved in deploying one to fully exploit its benefit and transform your business.
With all of these of these factors in mind, choosing the right software or services is an intimidating process, and for some, it can even be paralyzing. So here are five tips to guide your decision-making process and other factors to consider before committing to a Big Data software solution.
- Don’t confuse a Big Data solution with a point solution
One of the biggest mistakes that companies make is buying Big Data tools to solve one specific problem or buying them piecemeal without considering the needs of the entire organization. Before buying anything, consider all stakeholders, including those in marketing, sales, operations, R&D, IT, and analytics, to understand what they need to drive the business forward and how information can flow seamlessly across the organization. Buying point solutions or one-off tools or infrastructure that address only one aspect of the solution limits your organization’s ability to extract intelligence from Big Data and convert it to value at scale. And implementing multiple Big Data tools increases IT complexity because most point solutions do not interoperate.
Whoever does make the purchasing decisions in your organization needs to bring experts from data analytics, IT infrastructure, software, and business users to successfully deliver benefits from the data throughout the entire organization. To have knowledge of only one or two aspects could lead you to purchase the wrong solution and/or increase your costs considerably.
For instance, if the decision maker purchases an analytics package to address certain business needs but does not have the underlying software platform to quickly put those models into production, then the models quickly become useless without a significant IT investment. Similarly, if you buy Big Data solutions that offer easy integration but do not address the range of possible business use cases, your organization will not be able to extract the maximum value from its data or its Big Data investment.
- Accessibility is key
Big Data can be complex. Many Big Data solutions are designed for only those with specialized skills to use, such as data administrators, analysts, or software teams. These individuals are expected to run queries and deliver insights for the entire organization. This structure is far from ideal. To achieve the maximum value from your Big Data investment in a fast-changing environment with ever-increasing business demands, the insights from the data must be democratized and made accessible. To this end, it is imperative that the fundamental design of the Big Data solution you choose enables accessibility and usability of the insights gleaned from the data throughout the entire organization — and especially to the end users who make critical, real-time decisions that have a direct impact on revenue. These users ultimately determine the value of the insights available in the organization.
In addition to ensuring usability among all levels and types of employees, it’s also critical to ensure usability among silos within an organization. Often, information used by one part of an organization could greatly benefit other parts of the organization. Breaking down these silos and sharing the insights garnered from the single source of enterprise-wide data allows the entire organization to operate with a single source of truth, which helps align departments and propels the company forward as a whole unit — as opposed to how most companies operate, which is disjointed and results in a “one-step-forward, two-steps-back” motion.
- Avoid repeated trips to the well (or data lake)
Another mistake that many companies make with Big Data is that they try to solve new-age problems using new-age tools but with an old approach. That is, implementing software or tools that address a single business challenge or use case. Delivering insights from a Big Data solution can be a labor-intensive endeavor. Make sure that the software solution you buy delivers insights that have a high reusability factor — meaning that it extracts and operationalizes valuable patterns and insights, or Signals, so you don’t need to go back to the source data every time you need to solve a business problem.
For example, if a marketing manager wishes to run a customized, data-driven ad campaign, the traditional approach would be to build a one-time model for that particular use case. Unfortunately, in order to run a similar campaign later or in another region with different parameters, the model would have to be rebuilt from scratch, which could take several months, as IT would have to code the model again and integrate it into your IT infrastructure. More advanced software solutions create a searchable layer of Signals independent of use cases, which are proven to be relevant for multiple use cases. This layer sits above the data pool and enables users to develop more use cases — and solve more business problems — in less time.
- Forget Big Data: Think small
The biggest challenge with Big Data is converting it to small data so that it can be more easily managed. Big Data requires a lot of infrastructure, and typical data management software and databases are not designed to manage Big Data projects. Plus, despite advances in data collection, unstructured data contains a lot of noise, gaps, and discrepancies. In order to make sense of the data, the Big Data solution you choose needs to be able to separate the wheat from the chaff, and make inferences to fill in any potential data gaps. For instance, if you only need to capture 200 customer attributes to understand what incents them, then there is no need to capture and analyze 2,000,000 attributes. Look for software that can synthesize Big Data and parse your data into more manageable pieces focused on various problems that you are trying to solve.
- Buy bundled solutions to solve for adaptability and ease IT integration
Changes in business environment and technology evolution could limit the return on your Big Data investments over time. While you can buy best-in-class solutions today, you need to ensure the solutions you deploy adapt or evolve with the changing technology landscape. This could pose a significant challenge if you have multiple components in your architecture. You can avoid this challenge by implementing a platform that has a built-in technology stack that incorporates best-in-class components. This will significantly reduce the complexity of the infrastructure, decrease the number of integrations your IT department has to manage, and ensure evolution of your environment with changing technology.
And finally, a few closing tips: Given that there are many new and upcoming Big Data solutions providers making huge claims, it is particularly important to look beneath the surface of a provider to determine the kind of value it has created for its customers and how it has delivered that value. In addition, look at the types of customers it’s serving; you’ll be able to see how innovative a given provider is by whether its clients are innovative. Resist the urge to follow the herd or the biggest companies; instead follow the lead of the most innovative companies. As with any long-term relationship, implementing a Big Data solution requires a lot of effort and attention to detail, but if you start with a solid foundation and establish the right processes from the get-go, you will set up your organization for many years of success.
