News & Analysis

ChatGPT is An Intern in Analytics

AI-led chatbots could at best become first level analysts as enterprises try to automate data analytics across their companies

If one were to believe the CEO of an AI-powered data analytics company, then natural language processors such as ChatGPT have a long way to go when it comes to enabling automation of the entire data analytics process. In other words, these chatbots can serve as interns in an ecosystem that would continue to hire high-level analysts. 

A report published in SdXCentral quotes Alon Goren, CEO and founder of AnswerRocket, a provider of AI-powered data analytics to suggest that though ChatGPT could be considered a tool in the data analytics automation activities, the need for human intervention in the process could remain critical for its success for some more time. 

Goren likens data analytics as something akin to seeking information on Google, which simply retrieves answers from websites that contain the information. Similarly, with data queries, the answer resides somewhere within a mound of spreadsheets and only analysts can actually find and turn these data points into actionable business intelligence. 

Data from multiple sources, which is the right one?

The challenge is that the average enterprise has hundreds of data dashboards or spreadsheets they need to search, Goren says, adding that this makes it tougher and time-consuming to respond to open-ended queries and could be considered to have to do with more than a single data point. So, the challenge is about finding the data, laying it out and then deriving something that is of value for enterprises in the form of insights that can drive action. 

Analytical flows typically need humans to run multiple sets of queries across a multitude of data sources to generate a handful of data points that prompt a follow-up. Sometimes, this could take weeks or even months, which means by the time data is collected and insights gathered, the momentum could be lost. 

Treat ChatGPT as a creator of narratives

With the arrival of ChatGPT, analysts at his company are automating this process. Goren says it is used as an analytical tool in use cases where basic data doesn’t suffice and requires assessment and presentation that drives insights and specific action. Newer language models cannot do this transformation exercise but can start writing a narrative for users. 

ChatGPT is among models that use underlying data analytics to write narratives that provide the businesses the ability to take decisions immediately based on instant follow-ups. What has actually changed is that in the past a typing error could impair the processor’s ability to respond while now this is not so and every iteration of a query enhances the result. 

This is what the large language models have changed as they don’t really care what language one is conversing in or how good one is at spellings. However, this also means that the chatbot can actually hallucinate information assuming that the user query means something that makes sense for it. In some ways it combines fiction and non-fiction, Goren says.

Which source is the right source? Does it matter?

Moreover, they can easily replicate incorrect information, both while getting it from external resources such as websites, or internal ones such as company presentations or notes. Today we are riding the wave of access to more information but there is an inherent risk that ChatGPT will access false information because of this very reason. 

So, what’s the road ahead? Goren feels that in spite of the technology being in its infancy, several risks of such AI-led chatbots can be reduced if users actually figure how they need to interact with the natural language processors and the limitations of such models. Users must focus on how to format queries to get the best data sets. 

He calls it the ‘retrieval augmented approach’ where the user is providing the context to the technology to help it fetch the right data sets as a response to a query. So, users need to confirm that the information presented is coming from the right place. For e.g. a simple check is to respond with an OK to ChatGPT’s answer and then ask what is the source of this information. 

In business analytics, an enterprise would not execute a business plan just because one is being presented. One has to provide details around the quality of information gathered, and its veracity based on comparisons with other sources. So, it is not just about did I get the facts right. It’s more about do the facts even matter? 

So, it all boils down to how users would interact with any analyst. You ask a question but do not expect to get numbers or even a chart back. To make the most of such models, users need to keep the conversation going with an innate ability to respond to follow-up questions. These make conversational AI the tool of the future in terms of business analytics, he concludes.

 

Leave a Response