Cross-Regional Data Integration

– Frank Huerta, CEO, TransLattice, says:

Like any industry, the database sector sometimes sets and sometimes follows the newest trends. These trends appear due to technology innovations either within the database sector or in a related field, creating disruptions that spur more innovation. At times, the hype surrounding a trend is unwarranted and soon fades. At other times, the hype is legitimate and presages a real transformation. Such is the case with Big Data.

When we look at the trend towards all things Big Data, the volume of data sets worldwide is growing dramatically. As more and more companies use data analytics to influence important business decisions, the ability to store and access that information is becoming more of a concern. As such, data integration is an integral component of being able to do both of these things effectively. Not only is the task of managing mountains of data challenging, but when you add multiple data center locations around the world into the mix, you can have a mess on your hands rather quickly.

Take, for example, an online video streaming service. With a huge subscription database to manage, and end users signing in from around the world, ensuring high availability and excellent performance is critical for success. Customers have high expectations when it comes to video streaming services, including 24/7 access to their account from multiple devices.

Data sets that include profile information, user preferences, session data and subscription information must be stored, managed and delivered across multiple platforms and locations at all times. Personally Identifiable Information (PII) that include critical data such as credit card information, names and addresses can be especially challenging to handle. These complex and oftentimes contradictory requirements can be quite difficult to manage. Failing to provide a flawless user experience can lead to a loss of revenue, increased business risk and customers attrition.

This example illustrates only a few of the intricacies of data integration that need to be carefully considered in order to effectively manage data in a global environment.

Living in an Always-On Environment

Historically important business data was primarily kept in a single, on-premise location, making it easy to access and control.  It’s unlikely that this is the case today, given the geographical expansion of enterprises. The business world has become an “always-on” environment with customers, partners and employees located around the world at any given time.  Ideally an organization would be able to store, access and manage data in each active region, but this is something that traditional methods of a single centralized data store are unable to manage efficiently.

Numerous challenges have arisen for organizations thanks to globalization. For instance, many foreign governments are becoming increasingly strict with regards to data privacy and security that originates in-country.  Not only are the policies themselves becoming more stringent, but regulations vary from country to country, making compliance even more difficult to maintain. Because of this, policies must be created and maintained that ensure data is stored with compliance regulations in mind – a task that’s easier said than done when a company is operating cross-regionally.

The unfortunate reality is that companies are forced to choose the lesser of two evils – either store data based on convenience factors, thereby risking non-compliance, or establish separate data stores by region. Each method possesses its own set of challenges:

  • Storing data without factoring in location – Storing data out of compliance with local regulation can carry serious and regulatory implications.
  • Individual, regional data stores – Storing data in separate geographic regions, in keeping with compliance, forces organization to continuously consolidate and synchronize data. Whether the organization is synchronizing this data several times a day, weekly or monthly, immediate access in real time is unachievable.

Clearly neither of these options is ideal when running a business.

Having Your Data and Using it Too

Luckily, there are new approaches to data management that allow organizations to maintain existing infrastructure, while enabling location compliance.  One such approach is using an integrated policy-driven management system that automatically synchronizes data in real time and provides a 360-degree view of the information at all times.

Using new data integration solutions that address data location compliance challenges significantly reduces costs and administrative time typically needed for the task. This type of approach uses a “scale-out” architecture where capabilities are increased by adding identical data management “nodes.” By simply adding multiple nodes, scaling can be easily achieved either within one data center or across multiple locations worldwide. An integrated policy management solution eliminates the manual labor typically involved with scaling a similar system and provides a more streamlined, automated process.

Using existing database infrastructure and adding data management nodes when and where they are needed allows organizations to run an integration solution alongside current data stores. As transactions are completed, most of the data is stored per the usual policy, with region-specific data stored in an accompanying node that keeps the information in-country. As more nodes are placed around the world, they form a geographically distributed fabric that provides data visibility in real time.