Data integration critical to business strategy

by CXOtoday News Desk    May 21, 2013

Data Integration

Enterprises will need to start looking for the right approach and technology for data integration. This is because data will continue to rise in volume and it’s important to figure out the scalability factors and solution to this problem.

“Every 30 to 40 percent increase in data volume usually forces an organisation to re-look at infrastructure,” said Venkat Lakshminarasimha, Global Big Data Integration Specialist, Informatica.

“From a velocity perspective, some organisations have 50 GB of real-time data streaming in per second at peak times—this means that you need to look at scalability of infrastructure, and Big Data solutions,” said Lakshminarasimha.

The rise of Big Data is taking the growth of data to a larger scale. However, it needs to be figured out how to transfer data from point A to point B in a scalable and resilient manner. The core issue being that the amount of data required to move between data stores will saturate quickly.

The approach is requirements-driven. Those charged with managing data and data integration should have a complete understanding of where the growth in data will exist. These requirements need to be aligned with data storage and data integration architecture.

The real challenge is to create the right approach to data integration. The changing requirements around the use of Big Data include both structured and unstructured data, leveraging distributed processing and the ability to scale to an ever-increasing load.

However, that’s only part of the story. A data integration solution that can provide the core integration services, such as transformation, translation, interface mediation, security, governance, etc, also needs to be selected. The toughest part is to select and deploy technology that can provide the required scalability.

The truth is while data is expected to grow it is critical to understand  how critical data integration is to the strategy.