Sanjog Aul [00:00:07]:
Hello and welcome to the CTN Think Tank. The topic for today is Deployment of Generative AI, Facing Challenges Head On. So when we are referring to generative AI, what all are we facing? What are those challenges? We talk about people having issues with planning, they have problems with figuring out how to implement it. There are some ethical concerns. There is a job displacement issue. There are focuses on governance and compliance related issues, and then they have got ethics and responsibility that we have to take, or rather responsible AI. How to make that all happen.

Sanjog Aul [00:00:48]:
Now, it is not the easiest for anyone to tackle, especially when Gen AI is moving and or morphing at a weekly cycle, which means every week we see something new coming out. And when you’re trying to do this adoption in an enterprise where we are used to changing things in every three to six months or longer, God bless us all. It is pretty difficult, pretty daunting. So that’s exactly why we wanted to have this think tank discussion, where we’re going to talk about the top challenges each of the leaders are facing, and then we talk about what all they have tried to do, and then thereafter, if they are stuck, what help can they get from other people in the group? And we will all be talking about something which is actionable, and it is useful not only for the group here, but also for our audience. So, with that said, I would like to introduce our guests for today. We have Aref Matin, he’s the Executive Vice President and Chief Technology Officer with Wiley. We have Juana Fernandez Silva, Director of Data and AI Partner Solutions with Microsoft. We have Nataraju Muniraju, who is the Vice President for Data Sciences at Santander Bank.

Sanjog Aul [00:01:51]:
We have Nithin Ramachandran, Vice President of Data, AI and Transformation with Kohler. We have Sami Huovilainen, who is the Managing Director, Head of Next Generation Analytics with US Personal Banking for Citi. And we have Shailesh Katti, who is the Vice President and Head BD, which is business development, AI, ML and Automation Ecosystem with Tech Mahindra. So, welcome everyone. What I would like to do is to start with you, Juana. The first question for you will be, what is the top challenge which is holding enterprises back from getting the most out of generative AI? And I have another question, which each of you needs to answer. What if Gen AI was a wild bird, which you wanted to tame and then make it your pet? Which wild bird would that be and why? So, starting with you, Juana.

Juana Fernandez Silva [00:02:54]:
Okay, thank you. So probably there are multiple challenges, but probably the main challenge is in terms of adoption, to have the people prepared to adopt that technology. We are talking about a new, different technology that is different, the usage is different. So the capabilities of the people that is using that is totally different. We are moving from tools that we have to click in order to use it and press the button, and that now is prompting. So we need to interact with the tools in a different way. So I think we need new capabilities about how to express ourselves in order to ask what we need to obtain for that technology.

Juana Fernandez Silva [00:03:39]:
So I think probably this is one of the main challenges. And the second question, about the whiteboarding, can you elaborate a little bit more?

Sanjog Aul [00:03:39]:
So this is like a wild bird that you find, the bird that flies, and there’s a wild bird, you want to tame it and make it your pet. That’s what I would say. If you look at Gen AI, that is a wild bird. You’re trying to figure out how on earth do I tame it and then make use of it and where you feel in control. So which bird would you compare it to?

Juana Fernandez Silva [00:03:48]:
Wow. I don’t know. Maybe an eagle. Could be the eagle.

Sanjog Aul [00:03:53]:
So let’s go with that. Let’s take eagle. And then, so over to you, Aref. Now, this is you next. Tell us about your top challenge and your wild bird.

Sanjog Aul [00:03:59]:
Give us some background about yourself and, of course, the challenge and the basis of why you’re saying this is your top challenge.

Aref Matin [00:04:05]:
Yeah, sure. So I’m CTO at Wiley, and as a CTO, most people know we have to keep the trains running. And keeping the trains running translates into many things. Systems need to be functional, customers need to be kept happy. You really need to be up and running, high availability, performance, worry about security, code that is meticulous across the board. So it translates to many things that need to be put in place and take place on a daily basis. So keeping the trains running is a little bit at odds with making sure that you’re continuously receiving this barrage of AI news at the same time.

Aref Matin [00:04:59]:
And trying to keep up with this pace of arrival of new things that are coming in the AI, generative AI field at a breakneck speed is an interesting challenge that enterprises face with, and we need to be establishing some positions. What do we want to do with Gen AI? What do we need to worry about with Gen AI? And what do we want to avoid with Gen AI? These are now routinely daily questions that are coming our way, and we really need to be at top of our game to ensure that you’re answering these questions here. I was also going to say, for the bird, I was going to say an eagle. It looks quite majestic when it’s flying, it’s quite vicious when it’s on the attack. And we need to make sure that we tame the bird to ensure that it is behaving correctly.

Sanjog Aul [00:06:18]:
Nataraju. I’ll go by Nata. Go ahead, share your top challenge and, of course, the wild bird.

Nataraju Muniraju [00:06:24]:
Sure. Thank you, Sanjog. I’m more of a player of the AI, so I try to use AI in my day to day to improve our customer experience or simplify our employees experience. So my top challenges are like keeping up with the AI technology. We build something and it’s working fine. Tomorrow morning we get a new version and we need to update that. As you guys know, development cycle for AI in financial institutes is long, like anywhere from three to six months. So within six months you’ll have ten different packages.

Nataraju Muniraju [00:06:59]:
So how to keep up with that? Upgrades and enhancements to be compliant. That’s one of the major challenges. The second challenge is educating cross functional teams on AI, because right now AI is considered as one of the, like, savior. At the same time, monster. The people who like AI, they all call it as a savior. It’s going to make things great. At the same time, compliance, CDO office and all, it’s probably seeing that as a monster because there is no control on it.

Nataraju Muniraju [00:07:30]:
So how to bring those two teams together and apply in a way it works for everybody. So those are two challenges. Now coming to bird. Every time I think about AI, I always think it as a crow. The reason is crows is one of the smartest bird. Or second smartest bird, I forgot. And it’s everywhere. You never know what it does, how it does. You really need to have an act to even control the flock of birds.

Nataraju Muniraju [00:07:58]:
So that’s how I always think it’s crow, smartest bird. But it’s too many out there. It’s difficult to choose if you want to tame anyone, like which one you’re going to choose.

Sanjog Aul [00:08:08]:
That’s a great parallel that you’ve drawn. Awesome. Great. Nithin.

Nithin Ramachandran [00:08:14]:
Thanks, Sanjog. Hi, everybody. So I serve as the VP of Data, AI and Transformation at Kohler. We’re a global company, spread across almost 30 countries right now. So as you oversee the global portfolio of data, analytic and AI solutions, of course, Gen AI is a big topic that keeps coming over. Everybody’s excited across the globe, irrespective of which country.

Nithin Ramachandran [00:08:39]:
The people that you talk to, they’re very excited about applying generative AI. And for me, the biggest challenge, as I think through, is you’re leading teams across different cultures, across different businesses and business models. And when you contrast that, the change management and the rollout plans, Nata talked about, how do you educate the workforce? Juana talked about how you are looking at prompt engineering and educating them. But when you put all of those change management considerations in place, really the biggest challenge is you have a change management consideration, but you have to contrast that with just the speed at which technology is pouring out these days. Ten years back, you had time to put a change management plan. Today, before you put a change management plan together, OpenAI has a new release in place. So how do you really keep pace with that, especially when you think about an enterprise as large as Kohler? That’s really the biggest challenge that I look at in my role. As far as the question, the wild bird, Sanjog, great question, by the way, you got me to think a little bit.

Nithin Ramachandran [00:09:45]:
To me, it’s going to be a parakeet or a parrot. Great, really smart bird. It repeats anything that you’re doing. It’s influenced by what you’re saying, but depending upon how you train it, it can talk back to you in an amazing way, engage in an amazing conversation, or what you get back is utter nonsense and hallucination. So to me, that’s the greatest parallel I can find in a wild bird to generative AI. Back to you.

Sanjog Aul [00:10:12]:
I’m so fascinated with the number of different flavors of birds that we have been able to tell to compare generative AI. So, over to you, Sami.

Sami Huovilainen [00:10:20]:
Yes. So, Sami Huovilainen, I run a data science team at Citi. I’ve been here for 20 years, so I’ve had the privilege to see the innovation from logistic regressions to GBMs to neural nets to BERT, and now generative AI picking up a huge amount of pace. So a very exciting industry to be in. So now, in terms of key challenges, I would say within people, processes, technology and governance. From people perspective, definitely learning. And processes: when you re engineer and automate processes, you still need to keep running the old process for some time. So how do you have bandwidth to basically even sunset some of the existing processes? In terms of technology, to find basically enough compute to develop and similar compute to implement all these solutions with a proper latency and accuracy.

Sami Huovilainen [00:11:20]:
So the problems lie in those areas, generally speaking. Governance I think we can talk about later. But in terms of people, I guess my challenge is to say basically within one to two, three, hopefully not too many more key priorities within the firm, those are the ones that are deemed to have by far the largest impact. They are mission critical. So how do you, on top of everyone’s day to day priorities, how do you carve out cross functional teams whose number one mission is to get this use case implemented while they still have day to day jobs too? So, looking forward to hearing advice from my fellow panelists.

Sanjog Aul [00:12:02]:
And did you talk about the bird?

Sami Huovilainen [00:12:04]:
Oh, I see. My bird would be owl, because in here you need to have vision into forward, but be able to quickly look back as well to see where are we coming from and what can we learn from our mistakes. So owl has the ability to toggle between forward and backward.

Sanjog Aul [00:12:26]:
Amazing. Shailesh.

Shailesh Katti [00:12:29]:
Awesome. Thank you. Great to be part of the panel. I lead, like you mentioned, the Business Development function for America. Most importantly, I’ve been AI evangelist for almost eight plus years now. And like other panelists said, we’ve seen the journey take off from the point of the regular, we nowadays call traditional AI, now to generative AI. We do happen to see all of the work that has contributed in AI adoption from traditional world continues to be one of the areas where focus is.

Shailesh Katti [00:13:10]:
And while the generative AI, which is another tool in the toolkit or the technique, we do see, while there is a lot of interest, there is a level of control and regulations that are associated with it needs to be implemented. And I think as an evangelist, as a practitioner, we see this happening quite every day with different set of customers. So the problems or challenges, what you talked about, absolutely, they are in place, and we are trying to articulate some of the solutions, part of our overall approach, as we call it, as enterprise AI. And that’s where we are shaping the fact of how do we bring in or how do we put the controls for many of these moving parts in a structured manner, and how do we make sure that the production is the way we track any other technical implementation, with the visibility available, with responsible AI. Quite separate from all the noise or all the work that’s happening on the advancement of AI, I would be happy to share some of my experiences working with customers. You’re right, the question was great about the bird. It kept me, I thought about many of them, and what sticks out is the dragon for me. Being in the industry for long enough to see the amount of transformation with generative AI that’s coming up, it is quite appealing to see it can take you to new heights very quickly. But at the same time, it needs to be something that you know how to ride on to make sure that you are taking the best use of it for enterprise AI.

Shailesh Katti [00:15:23]:
And we believe that unlike any other places, enterprise is much more feasible to control the dragon and make use of it for taking you all to new heights, I would say.

Sanjog Aul [00:15:40]:
That’s great. So with so many flavors of birds and so many different challenges, I think we are well positioned to talk about solutions. So now with that said, Juana, I’m going to come back to you because you were the first one who shared your challenge. If you don’t mind, just again reiterate your specific top challenge that you would like to have the team here weigh in on and give you insights.

Juana Fernandez Silva [00:16:03]:
So I think that probably the top challenge is, we have been talking a lot about adoption and control and usage, and I would like to reflect on a couple of things. One is that sometimes when a new technology came, especially people in the technology side, they want to test it and they want to produce pilots, or okay, let’s use it for this or let’s use it for that, but starting from the technology. So I think that we need to start not from the technology, we need to start from the business, the business problem that we need to solve, what we want. So we need to start from the end, not from the technology. So I think this is important related to adoption.

Juana Fernandez Silva [00:16:47]:
And also, we have touched different points that also will give us the idea that Gen AI is not only a technology that came, it’s something that we also can develop internally in the organizations, because we can customize, we can create new solutions using Gen AI. So I think that we will find easily, and we are finding now, people that with just a simple tool with low code, they can produce their internal, maybe chatbots, for example, for a specific area. So it’s not only technology that is deployed and used, it’s also technology that is everywhere across the company. So it’s really important to set up the scene to adopt, how we are going to adopt it and also how we are going to train all of these people that has that in their hands. And they have it just now. So I think that resonates on me again, connecting with the people, connecting with the usage and also how we can manage that usage, and for sure always moving from early stages and pilots into scale. I think we mentioned, all of us, how to scale, and this is something that scales so fast, and timing.

Juana Fernandez Silva [00:18:09]:
Okay, the other point probably is timing. I remember, I think a couple of months ago, a CTO asked me for a list of functionalities. I said, I’m not going to give you list of functionalities on Gen AI that doesn’t exist anymore. So I think we need to also change our mentality when we are talking about Gen AI.

Sanjog Aul [00:18:31]:
All right, great. So Sami, you have comment on what Juana is looking for?

Sami Huovilainen [00:18:36]:
Yes, I would like to, Juana, maybe discuss a little bit of the business priority. What is the business problem? I felt like all of us, maybe a year ago, many of us were looking for some use for the shiny object, which was generative AI. Everybody was saying, well, since it’s available, might as well use it. So where can I use it? What are the most suitable use cases? Now I feel like we’ve come up somewhat of a full circle, at least half circle, back and say, what are the business problems that we are facing? And then find the right tool. It doesn’t even need to be Gen AI, if you came to realize that this is my big problem and I can solve it with something simpler. But it’s important to basically identify the business problems, rank order them with usually something like a time, value, uncertainty. Like how long time does it take to solve this problem? What is the value if you solve the problem? But how certainly can you solve the problem? So I’d rather take 100 million problem with 50% probability of solving than 400 with 10%. And of course the time is money, so the sooner you think you can get to the finish line, obviously the better.

Sami Huovilainen [00:19:51]:
So you can get some quick wins and then organization excited to solve the next problem.

Sanjog Aul [00:19:56]:
Nithin?

Nithin Ramachandran [00:19:58]:
Yeah, for sure. Juana, I think it’s a great point, and every time I have a discussion with a peer in the industry, we’re always talking about the same problem. Especially with generative AI, are we getting into a situation where it’s technology in search of a problem? And I think it’s a little bit of an and discussion. There’s a traditional framework of how companies worked, where you had business dictating priorities and then projects coming to IT and IT fulfilling this. I think it’s changed quite a bit.

Nithin Ramachandran [00:20:26]:
Right now, I think it’s more of an and equation. So the way, for example, we are kind of addressing that at Kohler is we’ve taken a traditional approach, was we had a platform team and a products team. And the idea around it is you have technology, technology drives capabilities, capabilities drives tools and experiences, and they in turn finally achieve a business outcome. So it’s really thinking through those hierarchies of like what is the role of technology? What is the ultimate goal?

Nithin Ramachandran [00:20:52]:
Yeah, so the idea around it, as I was saying, was to have technology teams oriented at adopting these fast changing technologies to release capabilities to the business users, and then working backwards from business problems to capabilities, and we meet at capability. So by having a platform and a product organization and also accelerators as part of your organizational structure, you really can do both. Work backwards from a problem and forwards from technology to capabilities. That’s how we structure it at Kohler.

Sanjog Aul [00:21:22]:
Great. So, Aref?

Aref Matin [00:21:23]:
Yeah. I actually have a twofold comment and then my challenge that I’d like to collect opinions from the panel about. If you look at any business in its purest form, any business is a collection of processes. Successful businesses have figured out how to really run those processes very efficiently and manage through that. Unsuccessful businesses struggle with those processes and things break down. And so a collection of processes, any business that you look at has hundreds of such processes behind the scenes.

Aref Matin [00:22:11]:
How do you know which one of these processes is the one that you really need to go automate and make sure that you leverage AI or Gen AI to advance the cause of the business is the most interesting question of the day. And I agree with the other comments that were provided. But at the end of the day, you have to really figure out the priorities of the processes. So that’s kind of my comment on the previous comments. But the challenge that is the elephant in the room in all enterprises is that neither Gen AI nor AI are free. And there is this concept of a budget in any enterprise. The more we learn about AI, the more we are seeing that there is a variable to the cost element of this, and it is not fixed.

Aref Matin [00:23:18]:
And the more AI you apply, the higher the cost will go. There’s this perception that has been created by the supplier community that if you start adopting AI, your costs will go down. This is completely inaccurate. It’s just like any other aspect of any technology. First couple years is going to have to be serious investment before you can actually harvest. And it’s a big conflict with the way the businesses are thinking these days. We are trying to bring efficiency in and drive the cost down significantly, while AI is really continuously increasing the costs, and also in a somewhat of a variable form, which means that it’s not predictable. So I’d like to understand, how do the rest of the panelists here deal with this type of a situation on a daily basis?

Sanjog Aul [00:24:23]:
All right, we’ll go quickly. We have got quite a few. Aref, your challenge looks very interesting because a lot of people want to comment on it. So starting with you, Juana, please.

Juana Fernandez Silva [00:24:32]:
It’s a small comment. When I had a look, I think last week, about what are the top uses that we are seeing in Gen AI, one is reinvent business processes. So now listening to Aref, that resonates to me. Because I think also we need to start to think if our processes are the right ones. So that is just throwing this idea about reinvent your business processes. And definitely, we have been talking about business. I think probably the answer, or the easy answer, is to have our projects aligned with what are our business priorities.

Juana Fernandez Silva [00:25:10]:
And one comment related to cost. Totally agree with you that initially our cost has to be up, but definitely if our use cases are aligned with business priorities or business objectives, also we are going to increase the revenue that compensate that cost. And always return of investment is the common conversation that I have every day with partners, in this case of customers, how we measure that return on investment. So just giving those two points quickly.

Sanjog Aul [00:25:44]:
Very good, Sami.

Sami Huovilainen [00:25:46]:
So I would say that look at processes that are very repeatable, clearly, and which ones have data available for the automation itself. Things like voice transcripts and images could be good. The scale clearly, like how sizable, because how many humans are involved now versus how much could the computer sort of do it on its own? And in terms of sizing too, how many audit or regulatory issues do the current processes expose ourselves to today? So that could be a big benefit to say that, well, all these near mishaps or deficiencies will not take place if you automate it. So I would say things like where the control processes are partial as well. Think of millions of, let’s say, payments matched by check, but humans only monitor a portion. Now with the automated solution you can actually monitor 100% and then make sure that the customers are treated 100% fairly. So those dimensions help us at least prioritize some of the process automation initiatives.

Shailesh Katti [00:27:02]:
Yeah, I think from the automation or the process aspect, I wanted to contribute a different thought process that we’re seeing are encouraged, and it’s coming now in many forums, is to look at AI, which has always been looked at as an agent or infusion that’s helping you enhance either productivity or change certain level of your processes. Traditionally that has been the case. Like robotic process automation came in, it infused certain level of automation to improve the productivity. I think what’s important to look at is with the amount of capability it has, there is a disruption angle which is being looked at now. Instead of infusing your existing processes, AI, which is looking at the dimensions from data, where all the systems that we built in the enterprises data is a byproduct. So if you are looking at AI from the dimension of can I use it starting with data and not as a byproduct, you are actually looking at every process in the organization in a very different way. And I would give an example for a finance, for a CFO. We are engaged today to look at various reports that are enabling the function to track the health of the organization, and is the most important function when it comes to quarterly results. A lot of that can actually be conversational, can be leveraged through loads of information available within the organization, as well as with your competition on the Internet, and make use of that at an operational as well as strategic level.

Shailesh Katti [00:28:59]:
So that’s an example of how this will completely change underneath processes available for the reporting of your CFO function, from a point of not leveraging any of those processes now for the use. And with that the disruption is imminent. Now I’m switching to the point of the challenge that you mentioned, and Aref, very spot on. We realized that for a large oil and gas company with 60 legal assistants, we were running this function which helps the legal team in addressing all the regulatory queries, and it was all fine until the time we said, now this has to be modeled for 6,000, and the cost was almost 30x. So what we looked at then is, I think one, the ability to create an API led available, intelligent led model as an approach most of the companies have taken, the optimization limitations are huge. So in an architectural world, we have seen it can be a hybrid approach with open source, with capabilities that are available that can be obviously leveraged to optimize.

Shailesh Katti [00:30:31]:
We do see all the features like the PTUs introduced by Microsoft and others, because of this problem, will keep enhancing. The challenge that you are raising, Aref, very uniquely, is that it needs to be accessible, it needs to be easily deployable from a cost point of view. As with the current cost, obviously the ROI is very difficult to meet, but I can clearly see that there is light at the end of the tunnel, with both the technology companies pushing for it, making it more optimal, as well as architectural ways and means you could implement to optimize this cost in your current construct that you are taking to production.

Sanjog Aul [00:31:19]:
Great. So thank you so much. Now we’ll take some quick inputs from Nithin, starting with you.

Nithin Ramachandran [00:31:25]:
Yeah, thank you, Sanjog. So, Aref, the point that you brought there around how do we leverage technologies like generative AI and machine learning and process automation. The experience that we have had at Kohler, as we’ve started looking at business processes and automation, has been, we don’t have to go after the entire process. I’ll give you an example of what we had done with customer service. Essentially, we did a lot of design thinking exercises, a lot of observations, and what we came out with is a singular thing. We were receiving thousands of phone calls where customers were just trying to find out, is my faucet a Kohler faucet? That was it. They were just trying to find it, because everything in that customer service event led from there. And we found out that that was really the bottleneck of the process. We didn’t have to go address the entire process with any AI solutions.

Nithin Ramachandran [00:32:17]:
We built a deep learning based machine vision model that could essentially identify the product. So what used to take about a five, seven minute exercise on a call was now reduced down to seconds. But once that bottleneck was cleared, the entire process issue was resolved. And that’s just one example. Every process that we have taken, whether that be credit approvals, whether that be cash to cash, order to cash, no matter what part of the process we take, generally what I found throughout, not just at Kohler, but even in prior careers, has been every process has a bottleneck that’s usually less than 10% of the process, but it essentially solves 99% of the problems. It’s not even 80, 20.

Nithin Ramachandran [00:33:03]:
And for us, the time savings of just resolving that one issue in that process, the time savings, the customer engagement, the revenue through repurchases that we received was more than enough to compensate for any spend on generative AI or AI or any of the technology we would have put in place. So I think that’s the key unlock when we look at process. I don’t think there’s any CFO that says if you give me a 4x return on investment, I’m not willing to invest. We will find the money. And I think that it’s on us to really go in and understand that one unlock in the process that can get us there.

Sanjog Aul [00:33:40]:
Great. Very quickly. Nataraju, love to get your comments.

Nataraju Muniraju [00:33:45]:
Sure. Thanks, Sanjog. I’ll keep it quick. So when it comes to cost, the approach we take is kind of a twofold. One is we always look for solution driven AI, then AI driven solution. What I mean by that is if solution requires AI, then we will use AI, if that’s the only solution. If there are in the organization, like big organization, you’ll see, hey, this is a new cool AI, we want to use it. We don’t entertain those, just because, as I said, there are a lot of challenges.

Nataraju Muniraju [00:34:16]:
But if solution requires it, that’s where we go. So that’s the first approach we take. The second thing is addressable opportunities. When I say what is addressable opportunity, is basically what is the size of the problem you’re solving, what’s the cost you’re going to get out of that, and what is the other benefits like ROI or customer satisfaction? We always have that. Every problem, we put that into the addressable opportunity bucket, and if cost x, and if my addressable opportunity is y, we can simply do the math and yep, that makes sense. Then we go ahead. So I’m trying to keep it very short.

Nataraju Muniraju [00:34:50]:
Those are like two approaches we take. Like solution driven AI and addressable opportunities permits AI, then we’ll go with it. One comment I will add, it’s not necessarily less solution, but if your data is a byproduct of your AI, then you will definitely going to incur a lot of cost. As long as your data is clean, readily available for any model and report or anything, then your costs substantially reduce. So I would say before investing on AI, invest on your data and make sure that your data is completely governed, controlled, everything is inbuilt.

Nataraju Muniraju [00:35:28]:
Once you have that, the cost of AI is comparatively lower. If you’re building everything as part of the AI, then it’s substantially higher. Thanks, Sanjog.

Sanjog Aul [00:35:40]:
Great point. So yeah, Sami, over to you. Let’s have you share your challenge again and things that the team here can help you with.

Sami Huovilainen [00:35:51]:
Yes. So across people, processes, technology and governance, I’ll focus my main problem within people. We and probably others have, let’s say, one to three high priority use cases which are driving substantial impact to the business, and these use cases have been prioritized by the top senior management. Now these use cases are being worked on by people who identified probably the opportunities. But these folks, they also have their day job, so to say. So you have to create SWAT teams, very cross functional, more so than ever, who will basically above their day to day duties work seamlessly together, find the time and resources and bandwidth to basically deliver these capabilities, usually within, I’d say within one to two years, and then move on to the next one. I think organizations get faster and better in implementing and developing these capabilities, but you always have to start from somewhere.

Sami Huovilainen [00:37:01]:
So how do you create a culture of collaboration and carve out bandwidth between the key people who will be delivering, from analytics to tech, finance, product, marketing? It is, like I said, more cross functional in nature than anything I’ve seen before.

Sanjog Aul [00:37:24]:
All right, Nithin?

Nithin Ramachandran [00:37:25]:
Yeah, Sami, I’d say that you’re not alone in that challenge. I think that that is a challenge for every organization that is embracing innovation today, and innovation is complex. Aside from the cross functional piece, there’s also an incremental piece to it, which is this huge strategic objective you have, but you also need to bring people together but allow them to make incremental progress. Teams may not be able to digest huge strategic objectives, so it’s also breaking it down. We’ve had that journey at Kohler, and I’ve had the pleasure of leading our agile transformation practice for the last couple of years.

Nithin Ramachandran [00:38:04]:
So we kind of worked in the traditional fashion, like you said, which is like the person who’s coming up with the idea works on it, very business versus IT kind of work culture. But we brought in agile with a specific objective that if digital transformation had to happen, cross functional ideation and input had to happen, and we also had to move towards giving the power to the associates of the company to come up with ideas to a problem. So the first thing that we did around that was organizational structure, getting incentives and organizational structure right, allowing people, so we set up teams. These teams were 100% dedicated.

Nithin Ramachandran [00:38:41]:
You couldn’t do this as part of a side job. If the problem was really important and you needed a marketing person in there, that person was 100% on the team. The engineer was 100%. So first of all, 100% dedication to the problem at hand, creating an organizational structure around it. The second part is incentives, allowing for these teams to work across together, keeping them a charter, putting a governance structure around it where there were incentives to work together. The other part of this piece is, from a leadership and culture standpoint, that’s a big change to happen, to bring these cross functional teams together. The first part that we really focused on was celebrating cross functional input and cross functional work. We did a lot of work around that to celebrate that.

Nithin Ramachandran [00:39:26]:
The other part was leaders leading the change. It was not about this is my org and my structure. It was more about all leaders celebrating an outcome focused way of working. So brought the teams together, 100% dedicated, set up a charter, set up a quarterly review investment structure, allow the teams to work incrementally, and of course, finally, leaders leading the change. We did that. We went from less than five agile teams to today we’re like 40 plus teams in under 18 months.

Sanjog Aul [00:40:00]:
Great. So, Aref, love to get your thoughts, and let’s quickly move through because we have three more folks who have got their own challenges to share and get some solutions.

Aref Matin [00:40:08]:
Yeah, so I think part of this was already said. In our case, we had luckily organized for the proper organization that could be the foundation for jumping into AI. 100% of the technology team in this corporation reports into my organization, and therefore cross collaboration has always been a requirement. When we decided to really pursue this mission, we just, like many other organizations, tried to look outside to see what’s available and get started with some subject matter experts. Quickly, make a long story short, came to realize that that’s the wrong approach. Our best experts are the experts that are in house and are actually familiar with our processes and our products and our customers, and also have quite a bit of AI expertise in the background. So we assembled our own internal team, kind of an AI services lab. And in this lab, essentially, there’s some requirements that was, I think, mentioned by Nithin.

Aref Matin [00:41:31]:
We need people that are 100% dedicated, both on the product management side as well as on the engineering side, to focus on this activity. And a portion of this AI services lab is going to be staffed by teams that exist in other parts of the organization that are doing some cool innovation work, or AI work for specific products or processes. And those people will rotate in and out of the lab and will make sure that they spread the love. As they come in, they bring in foundational technology and capability into the AI services lab, and as they move out, they take out knowledge and spread that out throughout the entire organization. And this way we grow our entire AI knowledge base inside the organization, and also advance our short term, immediate high priority activities through this AI services lab.

Sanjog Aul [00:42:37]:
Great. Shailesh. Are you muted, Shailesh.

Shailesh Katti [00:42:41]:
Sorry. First of all, great points already being made. I just wanted to contribute from the fact that when you’re looking at these rollouts and cross functional teams, I think first time, at least in the history, where every department is contributing. Earlier it used to be solutions led through IT. Now every solution, the policy, the change aspect, the legal, information security and all of it is coming together to have that blessing as it moves forward. So from the team’s point of view, we have realized that when you start scaling, having them early on, on the mission that we want to achieve for the company, from these support groups, is equally contributed and factored into your plans of execution. As a lot of times then, at the time when you’re ready to go to production, maybe a security clearance or a legal clearance can hold you back. That was one input that I thought, bringing in other support functions to be part of this. And most importantly, we are seeing when you’re looking at specific prioritization, a lot of times very simple problems that are obvious to solve are the one actually building and getting you the most amount of ROI.

Shailesh Katti [00:44:16]:
It’s an example, a simple claims process. We could achieve literally 50x saves in terms of how much easier that process has become, to a point of current process of adding more and more steps on approvals and many other aspects that are available in the organization. So that is my realization, that the most simple process to understand is the one that gets you the most amount of return on investment in the execution. Then all the experiments on creating question answers and cognitive search capabilities in the organization.

Sanjog Aul [00:45:02]:
Great. Juana.

Juana Fernandez Silva [00:45:02]:
Yeah, all great points. A couple of points, very simple, that I want to share, in my experience that we do in Microsoft. One thing is testing. Having as an employee you can test products before that they go out into global availability. This is totally volunteer program. I have been in that program since already ten years ago. So I’ve been testing functionalities and it’s part of my day job, but it’s also evaluated every year in the conversation that I have with my manager. So it’s something that I do volunteering, but it’s part of the contribution to other success.

Juana Fernandez Silva [00:45:42]:
That is one of the circles that they evaluate me every year in my performance. And the second one was that the tool, it’s available. For example, the last one, in the case of Copilot, that Microsoft Copilot, we have Champions program. And I think it was Shailesh, the one that mentioned across every department, and we have champions on each and every department of the company, champions of the product, that share best practice, make their own videos that they publish, or they send emails or message to others. Hey, I found that this is a great tool to do x and y. So I think those are two things.

Juana Fernandez Silva [00:46:26]:
One is before you deploy, having that internal usage testing, having these feedback loops where you get improvements, and once it’s deployed, having these champions, on each and every, from the top down and horizontally across every department. That is something really useful that we are seeing.

Sanjog Aul [00:46:49]:
Thank you. Nataraju.

Nataraju Muniraju [00:46:53]:
Yep. I think a lot of points are already discussed. I’m just going to give a couple of bullets, which we followed, might be helpful throughout my career. One, when you’re forming a SWAT team, there is no way SWAT teams are going to be like just doing one task. The reason for SWAT team is they’re going to do multiple tasks. So whenever SWAT teams are formed, as a leader, I will always give some room in timelines. If it takes three months, I’m not going to promise two months.

Nataraju Muniraju [00:47:23]:
I’m always going to promise four months, even though it takes three months. So that way it gives us an opportunity for learning and sharing within a team. The second thing is always under promise over deliver, instead of over promise under deliver. So if you can build a rocket ship, just promise to build the thrusters. Then if you deliver rocket ship, that’s great. But I always break up the deliverables into smaller chunks so that the team, or the SWAT team, like finance team, CEO teams, they have time to basically think through the process, learn through the process and deliver. And as they build gradually, they get better at it. The last point, I would say when you’re forming a SWAT team, focusing on these kind of things is always mix up veterans versus newbies.

Nataraju Muniraju [00:48:06]:
We always think that, okay, these are the best people. Just put them together. Yes, they are. They are the only people who learn. But I try to mix it up in a way that, okay, I have the couple of best people, let me put some newbies, because they will learn as part of the process. And I will make veterans that as a success goal for them, saying that, hey, you deliver these two projects, that is your goals for entire 2024. So in that way that becomes their responsibility, or they take it upon them to basically teach others and make that project successful. So there are like three things I would say. Take time, give some room yourself, so that we can learn and share.

Nataraju Muniraju [00:48:45]:
Start, make your problem into smaller pieces, solve one at a time instead of solving the entire world. The last one is always mix your team, don’t keep the same team. It doesn’t help your organization, it helps them, but not the organization.

Sanjog Aul [00:48:59]:
Great points. All right, Nithin, now your top challenge, and let’s have the team help you.

Nithin Ramachandran [00:49:06]:
For sure. And I’m guessing that this is a challenge for most of the data leaders and AI leaders in the space today, which is at Kohler, we’re a global organization spread around the world, multiple teams, multiple countries, multiple business models, multiple legalities. As we all start looking at Gen AI and all of this emerging technology, I think all of us pause for a minute around the security risks, the IP risks and all of those pieces, in addition, of course, to change management. And you contrast that with, yeah, there’s a release of a new version of ChatGPT or any other generative AI solution before you can blink your eye. So the biggest challenge is this, that I’m looking at input of how everybody’s dealing with this, is how do you prepare an organization to move at the pace at which new capabilities are being introduced, and trying to do it in a responsible fashion.

Sanjog Aul [00:50:03]:
All right, what’s the playbook, folks?

Sami Huovilainen [00:50:08]:
All right, so let me date myself and go back at 20 years. So, day of logistic regression, I had to, or had the pleasure of, partnering with Fair Lending, Legal, Privacy and Model Risk Governance to make sure that that tool is fair, statistically sound and doing what it is intending to do. Now we then went to GBMs and XGBoost and neural nets and then to BERT, some deep learning methods, and with the same rails, basically partnering with Fair Lending, Legal, Privacy and Model Risk Governance. Whenever new technologies or new algorithms were introduced, we sometimes had to spend some time together and often engage with external counsels and outside reviews to make sure that we are doing the best possible in order to ensure the models are in control, safe and fair and all that. Now I’m leveraging right now the same rails, on the other hand, amplified by the fact that Citi in 2019 introduced ethical guidelines corporate wide, and 2021 we established our AI COE to make sure that everybody in the company uses consistent methods for ensuring that the governance and privacy, all the other concerns are in check. Now, I would say that for these selected use cases, let’s say one to three, it is a time of co creation at much bigger scale than in the past. Like, let’s say when we went to neural nets, because it is all new to all of us, to everybody, and it is moving so fast.

Sami Huovilainen [00:51:56]:
But I feel like the key is to have really close partnership with all the governance and other cross functional leaders and teams, and be ready. I think people everywhere in the organization by now are rather aware that this is moving fast. So once we get to roll out the first use case, like I said, I feel like we build muscle to be able to do second time around faster and third time faster. But it’s back to partnership with other teams and co creating and staying abreast of the market.

Sanjog Aul [00:52:34]:
Awesome. Aref.

Aref Matin [00:52:37]:
Yeah, it’s just like any other new technology, except this one is becoming more fast and furious than others. There’s a natural human behavior that anytime there’s a new technology, there are risks and you have to worry about things and go and set up a governance committee and make sure that people are following the rules. And in reality, most of the rules of the game are unknown, even to this date. And so the soonest you organize about the risks associated with AI is the point of time that you realize that you’ve been wrong. So I would not really recommend hassling too much over do I have my governance all built in to make sure that I meet all the corner cases that exist there. People in the technology field are pretty smart and don’t need a whole lot of education.

Aref Matin [00:53:54]:
They do need some training, that you need to basically keep an eye out for the intellectual property that can be trained. People can learn that. They do need some training around the security guardrails as to how to deal with this. But we are all aware of companies that kind of went overboard at the early stages, by banning the usage of generative AI at work, as if it’s going to stop everything. And it didn’t. And many companies eventually had to give in and start using it, but some have put in heavier governance structure around it than may be necessary. That’s one aspect of this. But also another aspect of this is that this goes back to the question, Nithin, that you brought up regarding how do you encourage people to do this.

Aref Matin [00:54:52]:
I think it has to do with the culture of the company and the culture of the tech organization, product organization, inside the company. And leaders of these organizations really need to create a psychological safety net for their colleagues who are trying to experiment with these things and, yes, fail fast. And unless we fail in this area, we are not going to be able to learn all the issues in this game. And it’s important to have that psychological safety created inside an organization so that people know that experimentation is part of the game, and as long as they don’t end up repeating the same mistake multiple times.

Aref Matin [00:55:45]:
So I would recommend those two approaches.

Sanjog Aul [00:55:51]:
Great. Shailesh.

Shailesh Katti [00:55:53]:
Yeah, I think it brings to a point where I was talking to a CISO, and one of the comment he made is with the new world of generative AI, the amount of possibilities on penetrations has gone 10x in terms of what you could protect as an organization with all the shields and frameworks that our enterprises have built. Now with this new tech stack, there is ability to come from areas which are not seen by those layers. And I think with that, what we have realized is while you can have frameworks, policies and gate points, other sort of players are bringing in as indemnity and other clauses that can help you control the losses. But the most important thing is having your existing pipeline built with the awareness of security is crucial. What we have done is that having the layer which Juana touched upon, prompting, which is a new technique, SQL injection is another area that you could actually inject unwanted things. It is important to create the layer on top of your existing pipeline which is continuously monitoring these threats, and it is a scalable layer, because those threats that you are looking at from today’s point of view around your use cases may not be as sufficient as you start adding more functionalities to it. So example, ethics related responsible AI, there are certain techniques, the ingestion related AI, there are certain techniques.

Shailesh Katti [00:57:42]:
So it’s a combination of techniques that the responsible AI has introduced, which sits as a layer on top of your existing security framework and catches issues while you’re running through the pipeline in your different cycles until it reaches production. So you’re catching it early on instead of realizing it towards then. So, Nithin, my recommendation would be to look at creating the agile, I shouldn’t say completely flexible, but extendable architecture that brings in these newer checks and balances that are available either through hyperscalers or through techniques that are being introduced by a lot of open sources available, as part of your pipeline, which is doing this AI for AI. So it’s a checker AI that is monitoring your pipeline on a continuous basis for these areas that are security related concerns.

Sanjog Aul [00:58:51]:
All right, quickly, Juana, before we move on.

Juana Fernandez Silva [00:58:53]:
Very quickly, and sorry if I’m being a little bit naive to close the conversation, but I think that more than in every other aspect, on the security aspect, I think you do not need to be alone and you need to ask for help and participate in forums for your industry, rely on vendors. So I think this is one area that we need to collaborate more and be more together and listen to others, because definitely I think you already have a lot of points, but I think here more than ever, it’s important to share between companies, as I said, in a specific industry, with the technology providers. So I encourage you all also to join multiple security forums and bring your questions, your concerns there. Because definitely I think this is a thing. I’m sorry if I’m being a little bit naive, to close, Sanjog.

Sanjog Aul [00:59:53]:
No, this is perfect. Good point that you made. So, Nataraju, this is your time to tell or ask your issue and get help on the issue that you are seeing.

Nataraju Muniraju [01:00:03]:
Yeah, thanks, Sanjog. I think one of the major issues, similar to what Nithin asked, like how to keep up with updating AI packages and how to go through the governance. I was taking a lot of notes on the comments coming out from this discussion, because it’s same thing applicable for me. So I’ll go to the second most important one. What is your company culture in terms of creating the AI literacy across other teams? The technology teams which is involved with AI, they are very well versed of AI, and data analytics team or data science team who’s building AI, they are also very well versed. How you cross pollinate that knowledge to other teams, like compliance, regulators, InfoSec? How do you do that? What is your approach?

Sanjog Aul [01:00:49]:
All right, Nithin.

Nithin Ramachandran [01:00:51]:
Yeah, I’ll take that up, because it was only like two weeks. I’ve been working on the learning framework. Two weeks back, officially took over the digital university, which is essentially responsible for learning. But I’ll kind of break this down. So, Nata, your question, into kind of just a couple of areas. This first part about, there’s knowledge and learning around the work that is being done, especially if you’re in larger organization, like privacy or compliance may not know there’s an experiment running. How do you make sure that there’s constant communication on what’s coming up on the roadmap, and how do you involve right people in the discussion? So this is really where we start looking at, and I think Sami raised this point earlier as well, which is like fewer, clearer objectives that we can go after.

Nithin Ramachandran [01:01:32]:
Rather than trying to take the paint spray approach, like, let’s go after a few things, because the amount of people that need to be brought together for a challenge like this is quite huge. The second part is general learning around helping people adopt it. So we’ve kind of done two things. One is we call it the Kohler capability accelerator. We partnered with Ivy League universities to create some custom curriculum, and industry partners as well, to create some custom curriculum for us. That’s more of a surgical knife approach, where we’re looking at what are the key roles, like product owners, product managers, analysts in the business, taking those key roles and taking them to a targeted development program where we have a say on what the curriculum is, taking them through that. So that has been one part.

Nithin Ramachandran [01:02:16]:
The second part is kind of what we are setting up right now, which is the digital university, which is meant to be more of a, this is general education for all of our associates to take on, and then working with learning and development in HR to include that in their development plans. But I think by taking a combination of these three approaches, fewer objectives to go after so that we can all come together on it, second, surgical knife approach around critical roles to drive the learning, and the third part is more like foundational learning for everyone, but most importantly, working and ensuring that they have an incentive to complete it, driving that through their development plans. I think that’s worked really well for us, at least so far.

Sanjog Aul [01:02:56]:
Aref.

Aref Matin [01:02:57]:
Yeah. So just like any other topic, carrots and sticks. So I think training is the best way to work with your colleagues to ensure that, from all disciplines, to ensure that they understand what business objective are they trying to solve and which processes they’re trying to prioritize for the use of AI, and understand what are the implications of use of AI in those processes, and get them involved early on so that you can influence any sort of false opinions that they may have had prior to this, to educate them through the process and make sure that they stay engaged and they’re participating in this. I once joked about the sticks, that if we are sensing that there are roadblocks inside our organizations regarding these topics, they will be moving into the AI services lab and they’re going to be spending time there for six months before they go back to their existing functions. So that was a joke, but this is a bit of a carrot and stick approach, that you have to provide the opportunity for people that are interested to learn so that they can actually come forward and learn. And you have to allow people, like I said earlier, to experiment and fail and learn through the process. There’s not an easy way to go about this otherwise.

Sanjog Aul [01:04:42]:
Shailesh.

Shailesh Katti [01:04:45]:
Yeah, I think one technique that we have seen at least three of our customers leverage, and is effectively working. In fact, some of it started with our learnings when we started using it internally for our own teams, for our own developers, testers, right from the time when the legal blocked us for not using, to the time now they embraced us. It has been a journey where we realize that having a governance of all the stakeholders, which are responsible for their own areas, rightfully so, in the existing world, are coming together. These forums are typically conducted to create awareness, and are also the change agents in their respective areas to embrace this technology, as there is frankly a lot of noise around what’s happening around with the negativity for this, and to make sure that that negativity is not getting into the enterprise world, which is shielded already with so many processes and techniques. So it’s important to have this change agent led governance forum established. And second is the recognition of either failures or successes in these forums broadcasted, because that one or two stories is creating one interest and building the culture of leveraging it. As most of them have their own personal accounts, they are using ChatGPT in their personal world. They are now looking at how am I going to contribute to the success of the company, and hence enabling that through the points of recognizing them in those forums, both successes and failures, has also been very helpful for our customers.

Sanjog Aul [01:06:49]:
All right, great. So Shailesh, now this is where we would come back to you about your own challenge. And you mentioned like method to the madness, right? That’s what you mentioned earlier, is what you want to see the rest of the team here give you as input. So is that what it is? Would you like to say it in your own words? The pet peeve?

Shailesh Katti [01:07:08]:
Yeah, I think we’ve always seen that there is structure taken for every new change, and it evolves. We all have seen it from the traditional waterfall to agile and so on. So for the enterprises to adopt in the traditional world, we have always seen the stacked approach of maturity. So having the approach towards tracking your progress in a manner where you are actually maturing your implementations, and embracing this to be not a use case driven but an enterprise led initiative, is something that I want to see. How in your world you’re driving as AI leaders.

Sanjog Aul [01:08:03]:
All right, who’s going to help Shailesh and the rest of the world? Okay, Nithin?

Nithin Ramachandran [01:08:12]:
Yeah, I’ll jump in on that one. It’s a hard problem, Shailesh, but I’ll kind of tell you, this may or may not be a clear answer to your problem, but to me, a lot of what I focus on is around the culture change. One of the key things is just language influences mindset, mindset influences behaviors. And especially as we were hearing the language in the organization, the language was we want to improve delivery and we want to make it on time, on budget, or we want to improve delivery. And one of the biggest things that I had to do with my organization is saying, I don’t want to hear the term delivery, I want to hear the term, we’re solving a business problem. We’re enabling a customer experience. Technology is in service of that mission.

Nithin Ramachandran [01:08:58]:
And we will do whatever it requires to get that done. But the idea is not that we’re going to deliver something and that delivery is going to be utilized somewhere to result in something. It is, no, we are an active part of delivering an experience in a business. Because to sustain the pace of innovation, at least, and I know this was a pain point that I raised, but at least my solution to it has been, to sustain the pace of innovation, the only result that I’ve seen is bringing the teams as close to the problem as possible. When we start looking at it from a maturity framework, a lot of what we have looked at is saying, okay, we’re solving the business problem. We have to take an experimental approach, proof of concept, pilot, in production, feedback loop. That is essentially the approach that we have taken in terms of driving this, because with Gen AI and AI and trying to drive that real fast, direct feedback loop, we just can’t find any other way.

Nithin Ramachandran [01:09:54]:
The traditional, as you mentioned, the maturity curves and all of those, none of those pieces work, at least in our view and our experience. So I think it’s like that. Starting with that mindset change and then taking the experiment, proof of concept, pilot, production, feedback loop approach.

Sanjog Aul [01:10:13]:
Great. Aref.

Aref Matin [01:10:14]:
Yeah, similar. As the Gen AI buzz started coming in through the enterprise world, we were also coming across people that were very enthusiastic as well as people that were not so big on the idea. And we spent a lot of time talking about the typical issues of governance, security policies, things that everybody talks about here. The most important thing that we did, which I think started really panning out for us nicely, was that we basically set out our goals and objectives regarding what we want AI to do for us. And as you spend time on this, I highly recommend that you do take the time to define the goals and objectives in some significant detail. It may turn out that you will be wrong, based on what you don’t know, but it helps bring clarity into the way you are thinking and your thought process regarding what AI has to do for us.

Aref Matin [01:11:41]:
The same thing that Nithin was mentioning here, delivery, of what, and how, and in what shape, in what color, in what form. I think the more detail you get to on the goals and objectives ahead of time, the better off you are, because it helps you shape up the vision of where you want to end with the literal use of AI, Gen AI technologies throughout your products.

Sanjog Aul [01:12:18]:
All right, one last comment. Juana.

Juana Fernandez Silva [01:12:19]:
I want to add something to that. I like the idea of, okay, where I want to use AI, but I think this question has to be always open, because you are going to find new places that you can use it that you didn’t find maybe last month. So I think that it’s always important making yourself or your organization that question every day or every week. So it’s not something that, okay, for the next year we’re going to use it for A, B and C. So I think we need to find all of us the way that have that question always open, because a new case can come out from, I don’t know, a single guy that works for HR in a far country, that she finds that, are you seeing that for that usage it will be useful for me. So I think that how to keep that question always open probably is the key thing here.

Sanjog Aul [01:13:20]:
What a great conversation today. Thank you so much. All of you who have shared your insights. Aref, Juana, Nataraju, Nithin, Sami and Shailesh, thank you so much once again for making this think tank successful. And my special thanks to Tech Mahindra and Microsoft for their generous support and sponsoring this think tank. Thank you so much once again and have a lovely day ahead.