Technology, data complexity hinder big data adoption

by CXOtoday News Desk    Feb 22, 2013

Big data
Delivering the future of business analytics, Pentaho Corporation announced results of its survey that found nearly 50 percent of its respondents are either in production with a big data project or planning to deploy one in the next 18 months. The poll also revealed three key trends with big data deployments, according to Pentaho:

• Big data talent crisis is exacerbated by overly complex Hadoop technologies combined with the expectations of data analysts for access to and self-service for big data sources.
• Data diversity prevails and new types and volumes of data present considerable data management problems.
• Revenue and customer big data applications demonstrate immense value to companies on the front lines with big data who are solving real problems and uncovering new business opportunities.

The other findings include:
Big Data Talent Crisis: The primary consumers of big data are either business analysts (55 per cent) or data analysts (42 per cent) with some serving both roles, which demands an understanding of analytics and a deeper technical acumen. Only 20 per cent of respondents identify themselves as data scientists. The survey also pinpointed the top three challenges in deploying big data as: Resource availability (45 per cent), skills (43 per cent), and complexity of big data (38 per cent). Clearly, respondents are struggling with programming-intensive data analytics technologies and complex data sources. Continued improvement in the simplification, automation, visualization, interactivity and self-service functionality of big data solutions are also needed.

First Big Data Winners: Revenue and Customer Applications – Emerging big data applications and use cases can reveal highly vital insights. According to the survey, customer transaction analysis (43 per cent) is the most popular application in use today, followed by application log analysis (37 per cent) and e-commerce transactions analysis (20 per cent).

Among the early market adopters, half are visualizing their big data via reports/dashboards and interactive analysis. Finally, regardless of application type, the majority of the respondents said they are deploying on-premise, with only one-third using the cloud.

Data Diversity Rules: With multiple data sources the norm, over 50% of survey respondents cited both managing data and manipulating data as big bottlenecks. Furthermore, more than 60 per cent of respondents report that they are managing three or more data sources (32 per cent with five or more). Almost half (47 per cent) say their organizations are using standard relational databases as well as Hadoop or NoSQL.