Industry Insights on AIOps: An Interview with Rich Lane, Senior Analyst, Forrester Research
Murali Nemani: Hi everyone. My name is Murali Nemani. I’m the CMO of Science Logic. Welcome to our thought leadership series featuring industry analysts, visionaries, and leaders who are bringing industry insights that are valuable to our audiences. Today we have Rich Lane, who’s the senior analyst at Forrester. Rich, thank you for joining us.
Rich Lane: Well, thank you for having me.
Murali Nemani: Rich, you’ve just completed a new wave, as well as a study that is in a new dimension and a completely new area that is breaking some boundaries in terms of what the established way of looking at the industry for IT operations has been and where you think the market is going. Maybe you can just give us a little bit of background on yourself as well as the new analysis that you’ve just conducted.
Rich Lane: I was an IT operations practitioner for 20 plus years. I worked a lot of different systems. I worked on monitoring platforms. I worked in database platform. I have a lot of experience with the tools that are out there in the market. When I came to Forrester and they said this is your topic of discussion, your area of coverage, how do you want to frame it. I said, well the market has moved on from traditional siloed monitoring tools. We used to have the infrastructure stayed in their lane, network was over here, application performance monitoring was over here.
Rich Lane: What we found though is as we built very distributed resilient systems that that paradigm no longer works. We need tools that talk to each other. We need them to be able to give us insights end to end from an application stack. The reason why this is so important is that we measure success now as a business in what services we’re providing to our users, our customers coming, interacting with us in a digital experience. If you have all of your data spread out over many, many disparate systems, when something happens, you’re getting calls, or you’re getting emails saying there’s performance issues, or a product isn’t available, or people are getting errors, well how do you find out what’s wrong. If all your data is spread out across the enterprise, you can’t do that. I said the modern day applications need to be able to take data from disparate sources, bring it all in under one umbrella, and do that sort of smart and a correlation of things, you know, find out where that needle in the haystack is. I said well maybe I can look at the market, look at the tools are out there, and look at the vendors and find out which ones are kind of forward thinking in going along this path. I evaluated a large amount of vendors. I distilled them down to what I thought were the major capabilities they should have to play in this space. Then I ended up with 13 that I thought were the leaders. Then I came up with a set of criteria and what I thought were as important to enterprise users, you know, people making buying decisions. They’re going to spend a lot of money on next generation system, what do they need to know. Then through the research I think we came up with some really good answers.
Murali Nemani: One of the core areas that you spent some time on is on the adoption of AI and ML technologies, and the role that that has in future of these modern tools, right, for operations. There are those who come with this mindset of AI and ML as being really rich algorithms that ultimately solve some very complicated, sophisticated sort of problems. Then you’ve also spoken about the integrity and the importance of data as part of the getting the right outcomes. Can you talk about your approach around AI ML and what your report would suggest?
Rich Lane: To get a true, utilize a true ML solution or AI solution, we throw these terms around a lot, what do they really mean. These algorithms have been around for decades. They’re just mathematical, statistical analysis formulas that smart people did a long time ago, but there was no computers to apply them to. Now we’re at the point where compute and encoding and modern development techniques have allowed us to apply them. But in order to apply them, we have to have a couple of things up front.
Rich Lane: You have to bring all the data into one spot because we have to look end to end at what’s going on. We have to normalize the data because if all the data sets are disparate, we just get a bunch of junk in there and you have a data swamp if you will. Once we do that and get the nice data lake that we all dream about, then we can apply the smart algorithms on top of that to find business insights that give us better ideas and to show us what’s going on in our environments, in the things that we don’t have visibility in today.
Murali Nemani: To me, do you have a way that you recommend to your clients, or the relationship of how they should think about the role of algorithms versus the role of data and how those two, sort of how you approach those two?
Rich Lane: I think for enterprise customers, the data part is the most important, where are they getting the data from, how good is the data. If you think about … Take a data source, IT ticket data, we’re all going to put the data in a different way. That’s very difficult to normalize. Fortunately, machine data is pretty standard. We can figure it out. There’s patterns. There’s headers. There’s things we can read. But we still have to have somebody, and as an enterprise operations person I’m probably not going to be the one to do that, write the code that takes it and correlates it across the end to end through across all devices. Then and only then can the machine learning the algorithms really be of any use. If you don’t have good clean data, you’re in a bind and no algorithm is going to give you the right answer or probably any answer.
Murali Nemani: When it comes to applications, today we have this emphasis on the quality that these mission critical applications are running, they’re the lifeblood of many businesses. Yet, classic APM tools have maybe penetrated 5%, 10% of that application environment. You’ve talked about that there’s a big evolution in the APM market itself, and based on also the needs of the enterprise, and also those needs really being that they want to get broader coverage. They want to be able to give, let’s say, a certain set of capabilities to determine the health of a particular application. Can you talk about how you see that evolution happening and also maybe some recommendations for clients?
Rich Lane: Sure. I mean APM was originally formed as a diagnostic tool. We modernize. We’d say Java is a legacy language now, but at the time when it was the newest language, we had no way of seeing inside of the JVM. That was a problem for support people and operations people, and even developers, quite frankly, APM came along and it was super helpful and did its job and told and told them how to optimize their code, where the problems lay, and what have you. Even going fast forward to today to more modern architectures, heavy APM, what I call it, isn’t as relevant there. But the APM that we’re talking about now is distributed tracing and open tracing type things, using the tools that that leverage these technologies is more of a lighter, but just as important evolution of APM as you will. I think it’s almost a must have for modern architectures.
Rich Lane: It’s really important that all monitoring platforms have these capabilities to find out as we have these traces, transactions spread out all over the place, very distributed, we have to be very aware of what’s going on there and it’s very difficult to look at. Having a system that can then take a day in the life of a digital user, follow it through all the pieces of the very complicated infrastructure, and tell us where the performance problems lie is a must have.
Murali Nemani: One of the outcomes that everyone’s looking for is automations. Automations has been around for a long period of time, we’ve talked about it, but it’s even more prevalent today given the speed of which with the businesses changing. How do you think about the relationship of let’s say evolving, let’s say this level of sophistication of automations that an operations environment or a team puts in place and the kinds of things that we would need in order to drive better automations?
Rich Lane: Automation is, it’s talked about a lot right now. People are trying to get their head around what does it mean. Because you’re right, we have been talking about automation for 30 years easily or longer. Right now we’re at a good place though where with the use of machine learning, with looking at all the historical data, looking at the real time data we can make a really smart ideas on what to automate. Really good use cases are sort of around capacity planning. It’s always a struggle for our IT organizations because we over-allocate, under allocate, we don’t pay attention. But we can say through automations, hey this is going to happen in the future. You’re going to run out of capacity, auto allocate more resources to it. It’s a very powerful thing. Seems simple, but for operations people fighting fires every day to not have to worry about that, it’s a very powerful thing to be. It gets them working on higher level activities.
Rich Lane: The future of automation has to be more around the self healing you hear about. You know, I’m going to look through all your historical data around application outages, problems, what have you, be it ticket data, be it machine data, I’m going to make smart inferences based on that because I’ve seen this seven times before. Here’s the resolution. Do you want me to automate it? If we can get to that point, which I believe we will in a short time, that unburdens the operations teams so much more and gets them back to doing their day jobs, advancing things, new technologies, building out better infrastructures.
Murali Nemani: The evolution of modern tools is a big part of your research and the kind of tools that operators need to be successful. These are the lifeblood of the business, the digital platforms. To make it resilient you need, and keeping up with some of the new technologies like microservices and containers in a multi cloud world, often begs this question is do I have the right tools. Can you talk about what your research has revealed, the kind of insights that you’re sharing with your enterprise customers?
Rich Lane: I think it’s two or three things that leap right out to me and that I found through research. One was, if you’re going to look for a modern tool set, and enterprise customers ask this, what should I look for. Say, well, first thing that you need to look for is a time to value. There’s no longer do I feel comfortable buying a tool set that takes a year to implement. People want hours, days, at the most. The biggest thing also with scalability, can you grow with my business, or do you hit a wall somewhere. I want to know that five years now I’m going to be super successful. I’m going to grow three or four fold. They’re still there with me as a partner.
Rich Lane: I think that the last thing is how well are you developing a roadmap and sticking to it, how forward thinking are you as a vendor, as a tool set. Because sometimes you know they get caught in catch up mode or they discount some new technologies, and I want to know if I’m always going to be cutting edge, if I’m going to always push my development teams to be forward thinking, I want to know that my operations teams are armed to move along with them and not lag behind.
Murali Nemani: Rich, for those who have read your report and are currently living in your old shoes, how do they think about evolving their organizations and their operations environments towards this new world order to this new reality that you have outlined?
Rich Lane: Yeah, I think it starts with IT getting back to being the thought leaders of providing services through data, data management, and data insights back into the business. We’ve struggled with that on and off. As technologists we’re never good at promoting the things that we’re good at. We’re just trying to keep the lights on sometimes. But we have to become thought leaders again and going back to the business saying, hey, we have all this data, we now are at the point where we’ve moved way beyond the limitations of data warehouses of ages gone by, and now we can give you realtime business insights.
Murali Nemani: That’s one of the things that we hear quite a bit, which is this question of this AI ops, AI ML adoption, maybe we’re not ready for it. Maybe it’s something we do a couple of years from now. I think your position is actually to get started, right, and to engage. Can you talk a little bit about what that means?
Rich Lane: Yeah. I mean, the first thing you should do is say what are my biggest pain points today, what do I struggle with. It’s almost the same answers across the board. It’s visibility. I don’t have any understanding whether the right applications are up or down at a given time, and that keeps me awake at night.
Rich Lane: Having that premise, then go out into the marketplace and look for certain capabilities. How big are you? Do you need large capacity scalability? I think that’s … Can you grow with my business? I plan on expanding a lot. What kind of smart predictive things can you tell me about my environment? Do I have to set manual dials and switches myself to control threshold levels, or can you do that for me in an automated and intelligent fashion? Can you predict things that are going to happen capacity wise or otherwise down the future? Can you integrate with my other systems of record like ITSM, and Slack, and things like that that my people are using to be productive? Can you take the burden off them of having to open tickets, having to follow a ticket queue all day long, and having that be an impediment to their daily work environments?
Rich Lane: I think asking yourself what are your pain points today, are there solutions that can alleviate them and take them away from me, and can a system grow with me as I develop and go to more modern architectures and solutions.
Murali Nemani: Well, thank you Rich for coming and being part of our thought leadership series. We really appreciate having you.
Rich Lane: Well, thank you. I look forward to continuing the conversation.
Murali Nemani: Thank you for joining us on our thought leadership series. We look forward to seeing you next time.