Observability is emerging as a critical consideration for today’s DevSecOps teams, who are tasked with adapting to the radical transformation of IT environments. At a high level, observability refers to the degree to which the internal state of a system can be inferred based on externally available outputs. Therefore, the more observable a system is, the more it will enable teams to understand, manage, and enhance its performance. And with IT teams struggling to cope with today’s complex and dynamic IT environment, experts believe having a robust observability practice means fewer service disruptions, better customer experiences and more successful digital transformations. And all of this, leading to an improved bottomline.
Ben Goodman is the Senior Vice President & General Manager, Asia Pacific & Japan at New Relic, a cloud-based observability platform built to help companies create more perfect software. Some of New Relic’s regional customers are Capillary Technologies, Unacademy, Practo and Bangkok Bank, among others. A veteran with over two decades of experience in the IT industry, Goodman is focused on helping enterprises across Asia define what observability means within the context of their application landscape – as they move to the cloud and multi-cloud environments.
In a recent discussion with CXOToday, Goodman explains how CIOs can use observability to gain new levels of visibility and drive digital transformation. He also talks about the company’s robust plans for the India market. Excerpts.
What should a CIO understand about Observability and what kind of business challenges can it solve?
One of the biggest challenges for CIOs is the constant justification of investment. Technology is a large instigator of investment requests, whether it’s around innovation or sustained engineering, so big challenge CIOs face is knowing whether an investment is actually performing and having the desired impact. It’s also one of the reasons many organizations have wasted budget over the years. A major benefit of observability is that it helps to answer these questions by giving organizations access to quantifiable insights such as customer sentiment. It gives them opportunities to optimize performance and it brings it all together in a way that the CIO can confidently say, I’ve invested X, and the net effect of my business has been Y; whether it’s customer experience, whether it’s operational efficiency, or whether it’s money calibration. That’s what observability really does at a macro level. It helps CIOs answer the question, “How do I monetize tech investments that the organization makes?” For example, in India we have many clients who are in the food delivery and grocery space. They are really interested in understanding how investment in their technology stacks can positively impact the end customer experience. We’re helping these organizations to create those linkages and understand how those investments are actually performing.
What is the significance of Observability in enhancing CX and customer loyalty?
When we look at customer loyalty, we often look at the event itself: a customer comes to a platform and either makes a decision to purchase or not. It’s a very binary sentiment – either an outcome is achieved, or it isn’t. What we tend to lack is what was their experience while making that decision, or what are the things that we can do to positively impact their digital experience? What observability data does is help to understand why they did or didn’t transact and help identify if there’s an opportunity to streamline the experience. Observability essentially turns a binary event into an event that’s got a lot more actionable data points. That’s why we see observability, CX and loyalty being intrinsically linked.
How can organizations achieve intelligent Observability in their operations?
Intelligent observability comes through a few different areas. The first is about adopting observability as code and saying, if I have something that generates data, I need to find a way to instrument it and bring that data in to build out my models. Observability is an AI AI/ML/DL model. The more data that we have, the more trends that we can pick up, and the more we can apply automation.
I think the biggest paradigm shift was a couple of years ago. Organizations would only instrument the apps or the infrastructure that they needed to keep the lights on; they wouldn’t instrument things that weren’t necessarily required for day-to-day uptime. What that meant is that they could only ever get a model of production. Now what we’re seeing is organizations instrumenting a lot more of the processes: a lot more of the customer journeys, a lot more of the gateway, and a lot more of the infrastructure and cloud operations with the intent that once they have the data, they’ll be able to find meaningful trends that can positively impact the outcome. So we went from monitoring (is it up, or down, or working?) to observability (is it working as efficiently as effectively as it can, and are there ways for me to real time optimize). This is the shift from monitoring to intelligent observability.
So how can organizations achieve it? If you’re an organization that has a monitoring culture, i.e you’re using tools to find out whether something is working or not, it’s unlikely you’ll ever achieve it. If you have a culture that’s built around the precedent that anything you launch to the market needs to have analytics capability built into it by design, that it needs to have the ability to understand, review, optimize and consistently tweak, then you’re on your way to achieving it.
If you look at the organizations that are getting the most out of intelligent observability, they are the ones who literally have a Head of Observability. That’s a peer to the Head of Security, Head of Networking, Head of Infrastructure etc, because that person is ultimately tasked with providing insight into all of the other pillars to help them make better decisions. If you can’t make that cultural shift, if you don’t have a senior leader who owns the senior technology organization and also is communicating with business leaders to help them get better business insights, you won’t successfully make that cultural shift.
How is New Relic making a difference in this area?
Two key things. Number one: as an organization, we made the shift to a consumption-based model last year. This wasn’t just a change in how we report to the street, but a fundamental change in terms of how we treat our customers and look after them, and a cultural decision made by our CEO, Bill Staples. The shift to consumption means that every New Relic staff member’s entire focus is on customer excellence and adoption to drive maturity and help customers better understand and leverage the platform.
This cultural shift has driven us to create comprehensive assets and methodologies to make it easy for customers to adopt intelligent observability. As an example, we’ve developed a framework called OMA (Observability Maturity Architecture). OMA comprises learnings from thousands of customers across the globe on how they get benefit out of the platform. Things like how to instrument faster and more efficiently, how to work out commercial models etc. We’ve built a really comprehensive model that takes them from low maturity, to growing into a high maturity state by following logical steps that we’ve learned from other organizations. At the same time, we are staffing roles around technical account management and are increasing our technical teams across the APJ region, specifically in India, to help support this.
In the cloud-first and digital era, how can Observability help solve complex organization problems?
The cloud era is an interesting one because there’s three elements where we see observability having a big impact. The first is as an organization moves to cloud, they lift and shift their traditional on-premises or hosted infrastructure platform to the cloud as is. We know that drives a huge amount of operational inefficiency and cost. How observability can help here is to understand: a) What does my infrastructure look like today? b) What could it look like in the cloud operating model? And c) where do I need to replatform versus lift and shift? At New Relic, we actually have a team focused entirely on this element because we believe the cost savings and operational efficiency for organizations can be in the range of 25% to 30%. We see huge operational benefits in taking a more measured and structured approach around observability in cloud migrations.
The second piece is about consistent optimization. We see cloud providers like AWS introduce new services that are focused around ML and AI – these are great services but there needs to be a constant audit process around how organizations are using their cloud resources. Are organizations over spending because they’re provisioning for peak loads 100% of the year but only using 50% of them? Or are they under-provisioning and therefore compromising the customer experience? Many organizations are dependent on manual tools to give them this insight. Observability can give organizations real-time analysis of what’s happening in their core services to make sure they provision cloud environments to match the load at the time, but also have the capacity to scale up and down with new demand and/or new services being bought on.
Then the final piece is around integrating the end user customer journey with cloud tooling. For large e-commerce customers, their back-end demand for cloud services is typically a function of the front end demand from their customers, which can peak astronomically on sale days. A lot of organizations use traditional historical models to do that provision, and we see a lot of wasted costs, or over-provisioning of services for peaks that are just not necessarily there. Observability gives the ability to real-time forecast these peaks, and only provision exactly what is needed on those critical days, enabling organizations to easily scale up or back when needed.
These three areas combined is how we see observability solving complex organizational problems, and a positive byproduct of this is that a lot of our customers can actually fund innovation agendas and future projects purely out of the optimization savings.
How is the company supporting organizations in India through their digital transformation journey as their technology ecosystem grows in complexity?
I think it’s important to look at India across different markets. First and foremost, India has one of the largest educational sectors in the world, producing some of the best developer and site reliability talent across the globe. New Relic recently announced the New Relic Student Edition which gives students access to our platform, and allows them to develop the skills for observability at university to make sure they are job ready from day one. We see this as a great investment, not just for India, but also for a lot of Indian talent who are operating globally and contributing to global businesses. The New Relic Student Edition together with our more philanthropic investments around initiatives like AaroygaSetu to support the Indian COVID response, are initiatives that we believe are not just good for New Relic, but are great for the industry as a whole in raising digital competency. And the reason I call that out first is that’s really the foundation of everything that we do.
The other area of India that we support is within the more traditional firms who are going through their own transformation. We’re seeing traditional insurers radically transform from traditional paper-based, to mobile app based policies, and develop plans to create super apps around all the insurance offerings for the Indian market. That’s a really big change in terms of physical process to digital process, security, and customer journeys – and these companies have hundreds of millions of customers, so the implications are huge. For them, they are looking to leverage the experience of other digital natives that we’ve worked with across the region, both in India and other countries, and understand how we support them.
It’s important to note that we also appreciate that security requirements and governance requirements in different organizations and sectors vary. We’ve got different approaches to handle different organizations and recognize that India itself isn’t a market as a whole. It is a lot of different markets with lots of different subsections, and different organizations growing in different directions. We see India as a big growth market for New Relic, and over the next couple of years, we’ll have more and more expertise on the ground, not just to service our customers, but to better understand the growing demands of the Indian market.
Which industry verticals is the company working with? Can you briefly share 1-2 cases on Indian enterprises?
Capillary Tech is a great example. This is a customer experience company looking to radically transform e-commerce loyalty programs for other regions, and they are leveraging New Relic to better understand their customer experience. Capillary serves customers across the globe out of India, and one of their biggest challenges is that their customer base leverages their platform differently. How consumers leverage loyalty programs in Thailand is different to how they leverage them in the US, which is different to how they’re leveraging them in Europe. We’re working with them to give them the insight to understand how those customer experiences vary, and to globalize their product so they can customize it to local requirements, whether it be mobile, digital, or print, etc. On the other side, we’ve got India-based edtech unicorn Unacademy who are striving to build the world’s largest online knowledge repository for multilingual education.
Before implementing New Relic, Unacdemy’s engineers were facing issues with tool sprawl and data silos resulting in blind spots. Now by leveraging our platform, Unacademy can easily visualize, analyse, and troubleshoot its entire software stack in one connected experience whilst delivering continuous uninterrupted service to a network of more than 50 million active learners and 1000-plus top educators. Those are just two of many examples we see in India, of how the technology is being used outside of just traditional application and infrastructure monitoring.
What are the company’s plans for the India market in the next 1-2 years?
We put our first team member on the ground in India last year, and whilst we can’t disclose the numbers, that team has grown significantly over the last nine months and will continue to grow over the next couple of years. We recognize that market demands in India are different. The scale of the number of clients that our customers support is different with many supporting upwards of 200 million monthly active users. That drives a different demand in terms of our support of their platform in relation to global infrastructure requirements and technical expertise, and we’re investing to meet that demand. There are other plans in the works for India that will see increased investment in both people and other services, however we’re not at a point where we can disclose that right now. Suffice to say that our investment in India will grow significantly over the next 12 months, and that will be both to service local demand, but also the global demand that India services through many of the software companies and system integrators that have a global impact.