The Road To Big Data Is Still Rocky. Here's Why.

by CXOtoday News Desk    May 21, 2015

big data

While companies are racing toward implementing Big Data initiatives, a recent Xerox study says there are significant issues slowing the capture of value. Business executives said there are a wide range of challenges in implementing Big Data strategies, including data security, privacy and data quality.

“Executives see the potential of data-driven intelligence taking root, but the soil is still quite rocky in spots,” said the study conducted by Forrester Consulting which found that the lack of user training and inadequate change management are hindering the business transformation that big data solutions offer.

“Executives see the potential of data-driven intelligence taking root, but the soil is still quite rocky in spots,” says Craig Saunders, director, Analytics Resource Center, Xerox Consulting and Analytics Services. “The ecosystem is full of challenges.”

info

The study, based on a survey of 330 top-level executives describes in detail how factors such as “big data maturity” and poor data quality impact the enterprise.

 It cites three key trends underlining the state of big data in the enterprise today:

Big data is key decision-driver for 2015: Three fifths (61%) of organizations said decisions made during the next year are likely to be based more on data-driven intelligence than factors such as gut feeling, opinion or experience.

Inaccurate data proving costly: But 70% of organizations are still encountering inaccurate data in their systems and 46% believe it’s impacting negatively on their business, requiring re-calculation or totally unusable data sets.

Data security and privacy: 37% of respondents rated data security and privacy as one of their biggest challenges when implementing big data strategies.

“Despite the challenges, the large majority of companies are moving forward with big data technology across a wide range of different use cases,” states Saunders. “But there’s also a wide range of issues that keep executives up at night.”

The study found only 20% of respondents show high competence in dealing with big data, which the study defined as ‘Datarati’, while 31% are shown to be clearly lagging behind in their approaches (‘Data-laggards’). Most, (49%), were categorized in between these two groups, and defined as ‘Data-explorers.’

There is a marked difference between the Datarati and Data-laggards in terms of data quality. Nearly two-fifths (38%) of Datarati say that they never or rarely find misleading or inaccurate information within their data sets, while only about a fifth (19%) of Data-laggards said this.

A third (33%) of the Datarati have complete trust in big data analysis when making executive decisions, compared with only 17% of Data-laggards.

 Partnerships Remove Roadblocks

Company aspirations around big data remain high despite the presence of inaccurate data. Overall, adoption of big data solutions are expected to transform businesses through providing closer engagement with customers (55%), better engagement within internal teams (54%) and supporting greater employee productivity (54%).

“The majority of executives anticipate ROI soon, but realize that many organizational silos need to be broken in order to achieve that vision.” says Saunders. 

Over half (55%) of respondents in the study declare that they lack strong enough processes to ensure true data quality. To this end, 33% of respondents plan to hire more data engineers over the next 12–24 months, and 30% will also be looking to hire data governance developers and data scientists. 

Partnering with external experts is one way executives hope to make progress. The study showed 30% of respondents plan to partner with external providers to bring big data projects up to speed in the next 12 months. Three-fifths (59%) of respondents would choose to contract with two suppliers working in partnership, where one brings deep industry knowledge and the other is a specialist analytics firm.

“To make the most of the opportunities that big data and data analytics presents, organizations need to make the right investments in their big data ecosystem - people, culture, systems and processes, as well as good partnering choices,” concludes Saunders.