Enterprise AI is rapidly evolving from a decision-support tool to an autonomous decision-maker, fundamentally changing how businesses operate. Just as human metabolism regulates energy for sustainable growth, AI has its own metabolism of knowledge—continuously consuming, processing, and acting on data at unprecedented speed and scale.

But without governance, this self-directed AI evolution can reinforce biases, create decision loops beyond human oversight, and introduce systemic risks—leading to operational inefficiencies, compliance failures, and reputational damage. How do enterprise leaders ensure AI remains explainable, aligned, and controllable—before it starts making critical business decisions on its own?

Contributor

• Vince Kellen, Chief Information Officer, University of California, San Diego.

Transcript

Sanjog Aul [00:00:00]:
Hello and welcome to CTN. To learn more about the show, please visit ciotalknetwork.com and the topic for today is The Metabolism Of Knowledge In Agentic AI Governing AI Before It Governs Us. So here what we are seeing already is it is no longer AI being seen as an assistive intelligence, or it’s not just assisting us, it is almost trying to, or it is taking a driver’s seat. When you look at our own body, we have got metabolism happening. And there is a way, there is a science, there is a system that our body uses to keep things in balance. But is that truly happening in this world of AI, where there is some data being cracked, some insights being created, some learning is happening, some analysis and relearning is happening, or metabolism is actually kicking in, but is that truly in that balanced state as Mother Nature has given us? And, and if not, what kind of guardrails, what kind of balancing, what kind of approaches we have to take so that whatever comes out of it, it is truly unbiased, it is really meaningful and it is not going to take us in a direction where AI, besides controlling us or trying to control us doesn’t cause any other harm. Interesting topic for that. I have with me Vince Kellen, who’s the Chief Information Officer of the University of California, San Diego.

Sanjog Aul [00:01:30]:
Hey Vince, how are you?

Vince Kellen [00:01:32]:
Good, real good. Glad to be here.

Sanjog Aul [00:01:34]:
Glad to have you again now. So, Vince, this topic is rather an interesting twist. There is an interesting twist and we did discuss about it. Why don’t we start with you giving a context of what’s the genesis of something like this? And really very creative way of looking at human metabolism and trying to draw a parallel to what is happening with AI. Give us some background.

Vince Kellen [00:02:04]:
The genesis of this started with our own work with our own AI platform in which we’re using open source tools and Nvidia hardware on premise and between me and the development team, we’re starting to understand the complexity of knowledge and how knowledge flows through an organization, how knowledge flows into a system and out of a system, and with AI, how knowledge that gets represented by the A to us is actually a collection of little fragments of words, sentences and phrases from myriad places. It dawned on me that this is very similar problem to metabolism in the human body, where your body produces molecules, but they’re coming from all sorts of other molecules and ends from elsewhere in your body. And we’re coming to the realization that the complexity of what AI is trying to accomplish, which is take all of this body knowledge of the world that it knows of into it as well as new knowledge and then represent it back to us in novel forms is an extremely complicated problem. In fact, I think everybody’s underestimating the complexity of it. So everybody keeps talking about AGI and super intelligence and all these things are going to naturally happen. I’m not certainly convinced that that’s say complete at all because of the complexity of the knowledge out there. So what that means is there are a number of problems that will are beginning to surface with AI. Obviously, you know, one bias and control.

Vince Kellen [00:03:38]:
But we started to think, okay, we need control at many different levels, right? And so the origin of this was the metabolic framework. It’s probably not a bad one because we want to keep the AI working in a certain narrow of quality or performance or outcome or safety. What the human body wants to do. So that’s the origin of the story, is that we need a more sophisticated metaphor in order to make progress with AI.

Sanjog Aul [00:04:10]:
Would you say there is a fear of the unknown or paranoia that whatever way these organizations who came up with these different models and the way they are processing, the kind of interfaces are given and the kind of outputs that are coming out, are they not looking at it? Is it, is it something that we are trying to build our own layer of safety and that’s what’s triggering this thought process of us looking at metabolism as almost like a safety net or a safety check on top of what these top level companies who are putting a lot of money into it are not doing?

Vince Kellen [00:04:43]:
Yes and no. It’s a realization that the GPT, the general transformer model, you know, pre trained transformer model, that model is locked, fixed, it’s sort of monolithic. Even though there’s some modularity in it, it’s a kind of a monolithic model. So it’s like a big lawnmower that’s gone over all of the information in the world that in and of itself has insufficient capability to provide all the regulation we need. So when I say regulation of the AI within a homeostatic band of performance, there’s different levels of this. So at the top level we have societal, industry norms or organizational norms. These are just norms that we would like the AI to perform in. Where a lot of the debate is right now on the regulation of AI, we have a second level on federal and state law, typically privacy and data in those matters.

Vince Kellen [00:05:36]:
And we’ve seen law come out that’s looking at the safety of AI. When you get inside the organization, the organization has its own policies, procedures and processes that The AI will need to respect and work within. And then of course, within the technology itself, you have both the inputs and the outputs which can be evaluated for safety, quality, et cetera. And then inside the AI and especially the agentic framework, we have a lot of components now growing quickly, especially agents, typically Python code running in the environment. Each one of those needs a little bit of a checking as they’re running in order to stay within a bound of safety and start to design that into the tools. And on top of this sort of five layer cake of regulation, we now have a really big form of a type of regulation. The regulation you’ve been applying so far has been crisp logic, first order logic. You know, if this happens, that you have to do this, or if we see this, this has to happen.

Vince Kellen [00:06:36]:
That’s been our worldly law in regulation, policies, procedures, et cetera. With the LLM now and AI, the LLM itself can be a participant in its own self regulation, but it will be a probabilistic approach which will never be a hundred percent in a way that kind of mirrors metabolism. Metabolism isn’t a completely perfect regulation system, so type 2 diabetes and metabolic syndrome are perfect example. There’s no regulation on the upside of calorie consumption and food consumption. So the probabilistic approaches of metabolism begin to break down in that particular disease and same thing here. So we’re now having to contend with this new animal of probabilistic, so to speak, and how the AI is performing. Interesting, complex, a lot of tools to address these levels of regulation, but there’s also a lot more.

Sanjog Aul [00:07:31]:
Now I have a two part question. Number one, if we had to tackle at the human metabolic or whatever that you are referring to as a parallel that you’re drawing that human metabolism, and if this was to be tackled, would it be better tackled with a central set of bodies who bring this together? Or you’re better off federating it and saying, okay, this is your body, so you take care of it. So if ucsd, which is your organization, is trying to do something with these LLMs, then you’re better off given control to tackle your own metabolic approaches to AI inputs and outputs versus getting motherships, if not one, multiple motherships who work together to give you much better outcome. Because at the end of the day that guardrail that you’re creating is also a lot more sophisticated, which will require resources and time and energy. And would we really be equipped at individual human level or individual entity level who’s trying to do this? You know, the kind of Activity that you’re talking about, metabolic control would vo be so equipped to do it individually, or you’re better off pushing it upwards and more evangelize it. So powers take care of it.

Vince Kellen [00:08:53]:
Sort of flip the question a little bit. Can human sociology accept one master AI to rule them all? No. And I’ll give you an example. China released Deep Sea Perplexity, put it into their platform, and the first thing they did is they kicked off the guardrails that Deep Sea had on to respect the Chinese national societal norms they wish to impose on the AI. There’s a case of once the open model is released, an entity perplexity comes and say, okay, this is a cool model, but let’s shake off some of these regulatory frameworks around it, because it’s not what we agree with. So I think disagreement between humans around the regulation, especially around societal norms, is going to be an interesting one. So I don’t think there will be any form of a single mothership that will drive that. Maybe at a national level, that can happen with national law.

Vince Kellen [00:09:51]:
And even if in the presence of a national law, there’s the tailoring of it to hit the norms and the policy procedures and roles within an organization, organizations will have differences in how they operate. And that’s my point. AI is going to be a derivative of the complexity of human sociology. So therefore the regulatory frameworks around it will also have to be as complicated as the human sociology. So no, it’ll be impossible to have sort of one mothership directing all of this. And I think a good part of the world is very much afraid of that.

Sanjog Aul [00:10:30]:
Now, on that, the second part of my question. Thanks so much for the response to the first one. The second part is that if we try to become too customized for each entity, and of course, hopefully not each human being, but each entity, when we are talking about metabolism of AI, then each organization might in its own way think, okay, I have created the right guardrails, but then that would create way too many flavors and there would not be any adherence to any standards. Or if any new evolution that happens because even the AI and magentic AI is evolving, then we will always keep playing catch up. And some people will unknowingly be not in breach, but in conflict with the very intent that they started with, the guardrails will keep breaking because it’s a moving target.

Vince Kellen [00:11:17]:
True, it’s a moving target, but each organization competes, each company competes with its other competitors, and it competes based on its knowledge, how it wishes to render its knowledge. It competes with how it issues its policies and procedures internally, its business processes. So AI will be suitably tailored to meet the competitive and strategic advantage of organizations, whatever the that may be. And if it is truly strategic, which I believe it will be, then there will be heavy tailoring to match the the organization and compete upon it. I think that’s a central fight now between the AI providers. Is there enough distinction in inference alone for them to compete? So I think yeah, that will always be a risk. But we have that risk today in our own policies and procedures, independent of AI moving targets, especially anything related to technology. I’m not seeing a ton of variability or distinguishing qualities of AI that there’s a few big ones, no question.

Vince Kellen [00:12:29]:
By and large, the current regulatory framework is not necessarily ill equipped to handle it. We just got to learn how to use current regulatory framework, especially law and national policies to do that.

Sanjog Aul [00:12:42]:
So now that we have spoken like a skeptic, what should happen and whether it is really a good way for an enterprise to start? What’s your approach to even tackling something like this, where you’re coming in not truly blind about what’s going on internally, but to quite an extent. Most of the models that have been created overall, as they become more complex, if the outputs become more complex, the internal working becomes more opaque to us. So are we trying to eat the cake and have it too? Which means we want explainability, but at the same time we also want the guardrails and we also want the bias checks and everything else. Is that realistically possible? Is that what you’re trying to tackle and trying to accomplish?

Vince Kellen [00:13:27]:
I think it’s very realistically possible. I think we’re seeing evidence of it in the cost of understand. Inference is a locked model. Take a long time to build a model per se and then block it. Doesn’t change. And so all the complexity now around this is going to be in the agents being developed and marketed. And there’s a gazillion of them right now occurring. And we’re building our own agents.

Vince Kellen [00:13:50]:
The barriers to entry for building those agents have dropped. This is kind of like how HTML lowered the barriers to entry for application development on the Internet writ large. So anybody could create HTML and create websites and do E commerce real quickly. We’re seeing the same thing here in AI. As those barriers to entry drop, you get a lot of agents being built. And so think of the big focus of the regulation now is going to be in those agents. Now the good news for organizations is if you buy agents from a vendor, you can look at the contract and you can see if they will reveal as much of the earts for you to understand how that company is controlling those agents and whether they got the right controls around it. In our own work, since we develop a lot of our own agents, it’s what’s the software coding technique? What’s the technique between the agents and the tools underneath them that retrieve data from other systems? How do they work? So the good news is it starts with kind of mundane old school regulation.

Vince Kellen [00:14:53]:
Is this particular agent performing per the output we want? Well, how do we do that? We bring our users in to help evaluate and judge the output, which now looks like conventional testing, software testing. We get our users to rate thumbs up and thumbs down on the content as well as qualitative feedback. All of that becomes knowledge that’s fed back into the AI that helps to improve its performance. Unlike software development where you release just good old fashioned software, users say wait a minute, this isn’t right, they give you feedback, next version that comes out tries to correct it. So I think the good news is the bulk of our work is going to be in that form of regulation in the agentic world. For the LLM we could talk a little bit about how we can look at how the LLM itself is producing output and we’ve got some approaches we’re using there. It’s particularly worried and yes, I think organizations can easily do this, especially since the barriers to entry of draft and the number of models has increased dramatically. The cost of tokens for the inference have dropped dramatically.

Vince Kellen [00:15:52]:
It’s becoming commoditized almost soon to be it’s being offered by some to some organization, some of the providers to organizations as free. So in that environment it’s going to look a lot like Internet one revolution.

Sanjog Aul [00:16:06]:
So, Vince, you, you’ve been CIO for many decades and before this AI.

Vince Kellen [00:16:14]:
Just a couple, So don’t do panic attack.

Sanjog Aul [00:16:21]:
So the reason I’m saying this is because see when the IT portfolio was being built, right, and the focus was not to do tech for the sake of tech and you are the most pragmatic leader, one of the most pragmatic leaders I’ve seen when it comes to managing it. I know you always evangelize the concept of build, buy, rent, lease, whatever is needed and so that the perfection doesn’t get in the way of good. Because you at the end of the day are doing this for business outcome versus a cool geek project. So, and then now you at the stage where we are with respect to AI adoption, we are seeing some vendors and when in it, when it was cloud vendors or anyone else or any other SaaS vendors we always spoke about we are not able to see under the hood and that’s what created the skepticism. But we still went ahead with it, at least as part of the portfolio because we wanted to get the work done. So what has been your approach as you’re dabbling into it? Would you allow good to prevail or would you seek perfection because you feel the risk is way too much for you to allow good, which is not the perfect? You can’t see under the hood for everything. And yes, your guardrails may be there, but they may not be perfect.

Vince Kellen [00:17:36]:
Our real world experience has been that good is good enough to make progress. You don’t need a highly sophisticated model or set of toolkits to make really good progress. The large language model works out really well on mundane questions and most people have mundane questions or more mundane tasks where the big providers have been focusing on things like reasoning and deep research and what I call predicate logic type reasoning within the LLM. And those are going to be useful and helpful. But 85 to 90% of what goes on in our organization tends to be a little bit more mundane. So we focused on speed that up, make that go faster, and you can control the temperature of these LLMs and control their hallucination rate and you can certainly evaluate their safety on a regular basis and say, hey, that looks like the LM’s producing an adverse response here. So no, there’s still enough open models, hay models and closed models out there that you can make progress. Our own trend GPT platform largely uses llama models, but we also go out to and models for a couple of tasks.

Vince Kellen [00:18:48]:
So we’re going to keep this flexible approach to choose models. For certain cases people say, well that’s complicated. Well, I don’t have a large team doing this and I have actually some students working on it. This isn’t requiring rocket scientists to do the work. I think some people don’t understand is that I look at this as the 21st century application development environment for it shops. It will overwhelm all other forms of application development over time. Here people will be coming and using AI to write code, using code to put into an AI system and there’s enough to make great progress. Right now we’re already seeing it in the market.

Vince Kellen [00:19:29]:
I think there’s enough practicality here to make progress. But our approach is keep it flexible, keep it low cost Democratize it and be ready to shift as the market shifts.

Sanjog Aul [00:19:40]:
Now, based on what you’ve tried so far, right, And I’m sure you’ve done some sandboxing, maybe you would have tried to put that in a limited, controlled environment in a live setting. What has been experienced so far with this whole metabolism approach to balancing what AI produces so far in your lab or in your setting?

Vince Kellen [00:20:03]:
Well, in our setting we actually have a number of assistants we’ve created and some of them are mundane, like our job description helper. We got a UC San Diego assistant which is tapped into all of our UC San Diego content and it can give answers to questions like hey, where’s good dining on campus? To what are our policies around expense reimbursements overseas? We have a contract reviewer, one which the legal teams or the teams looking at non disclosure agreements and soon other agreements, they just email it to an email box. It gets picked up by the platform and then it gets redlined with our templated language and shoots them back a copy of the document, saving them a whole bunch of time on initial document review and redlining and highlighting. We also have a data Agent Factor Retrieval Assistant in which it can fetch accounting data for you. How much money do I have left in my budget? How much money do I have left in my grant? Thus bypassing the BI layer. So we’ve got a number of practical things that are out there. Where the metabolism comes in is under the hood in our neck of the woods. It caused us to think about how we develop these applications in a more sophisticated way.

Vince Kellen [00:21:18]:
And I’ll give you an example. I was talking to the teams and saying, hey, as you’re writing the Python code to create an agent, that’s a form of knowledge, your knowledge could be used in the AI to help automate how you create those modules. So think of it as AI creating AI. Now you’re thinking of Skynet and all that other stuff still under tight human regulation, but saving the developer time. Or here’s another example. When users interact with an AI platform, they themselves are asking questions. The AI platform is getting answers, in some cases recommending questions. Do we we give our users some recommended questions they can ask? That improves the quality of their questions.

Vince Kellen [00:22:03]:
It also brings an answer that gets rated up or down. All of that knowledge is knowledge inside the organization that now is inside the AI. That can help improve the quality of the AI, even though the transformer model is fixed and not changing. So those are examples where we’re thinking about everything as a knowledge fragments, whatever it is Wherever it is. And all of that can be put into the soup of this AI. And then as it creates new inputs from users and outputs from the LLM, that’s forms knowledge fragments. All of those can go into this soup and be available for larger knowledge bundles or knowledge molecules, we’ll call them text output. Coming out, coming out.

Vince Kellen [00:22:46]:
So the metabolism framework really here is just to help us guide our thinking on internal development. It also leads to a really interesting thing in which as you do agentic development, you can now think of the LLM as not just a generator of text, but a judge on the quality or the appropriateness of the text. LLM is judge. So in this regard the LLM behaves kind of like the kidney in the human body. It throws everything and then selectively includes things that the body needs. So it’s very exquisite filtering or judging. And so that caused us to rethink how we internally as our development team think about the LLM and the agent in our process.

Sanjog Aul [00:23:31]:
So let me take an example of a sausage factory. Basically I’m just going to use this to make a point. If you got sausage factory and you have to make sure that there are no germs and there is a certain, you know, approach to how it should be produced as a quality and there is a definition of quality. And as the thing flows through it, you look at every level, but you also look at the input and the output to make sure that the whole process is producing what is supposed to be produced. And if there is any anomaly in any new type of input that you gave, or there was a variation, then it is caught in between or if there is a the system within doesn’t have, or the guardrails that you put in there, the quality checks that you put in in the system itself, if it is not working, it triggers it. So somebody has to watch it or the system itself has to trigger. Now that was a sausage factory which is still got with limited number of inputs coming in. And it’s far more practical.

Sanjog Aul [00:24:32]:
But now we are talking about that taken to a total level, different level because the type of inputs would change. And now AI is creating its own AI. Like you said about, like the agentic AI could be working. So who is watching.

Vince Kellen [00:24:47]:
In our case we are we what we call an ensemble of experts approach to safety. What that means is we have a background process that runs every day and evaluates the inputs and outputs to the system and looks at it from a variety of aspects for safety. And those aspects include things like are there information Hazards in here? Are there malicious uses? Is it being exclusive? Does it have some level of toxicity to it? Looking for offensive statements, misinformation, harmful interactions between people, criminal planning techniques, you know, regulatory control substances which might be problematic, self harm, illegal weapons. These are all rubrics and frameworks that the research community has been using to evaluate safety of elements. We’ve taken those, we’ve built it into this. We use four other, actually five other models, their models to evaluate the safety of those inputs and outputs. So we are using different models under the concept that we can’t trust any one model to evaluate what our AI produces. So we use five or six of them, run, it’s always automated and then things are flagged and the teams run down what’s flagged and look in deeper to see if there’s an issue.

Vince Kellen [00:26:11]:
That’s one form of what we call out of mind safety checking. I think we’re going to, many models, especially Lama already have in band some safety checking in it. But I can see down the road we’re going to get more interesting forms of in band meaning in the middle of the interaction, additional safety checking going on. You’re right, you have to mind the store. No question on this. If you got a probabilistic engine that’s getting more sophisticated, you got a make sure your homeostasis processes are as explicit as the implication going in and out is.

Sanjog Aul [00:26:47]:
So, if I were to compare to something like an accounting, where you got the checks and balances built in into the very accounting system which has got the guardrails, but then on top of it you do internal audit and then on top of it you do external audit.

Vince Kellen [00:27:00]:
So two sets of eyes on each transaction.

Sanjog Aul [00:27:03]:
Exactly. So if you were to look at AI, what you just said, that you’re going to watch it, is that the system already that you put in place, you are claiming that you put in the checks and balances and you guys are the internal auditors and then on top of it you’re going to bring external pair of eyes to see whether you whatever the checks and balances are and what your internal audit is it truly bringing the output which you thought it is supposed to bring?

Vince Kellen [00:27:28]:
Yes, and we are absolutely doing those checking today and we’re absolutely able to deal with external audience who want to come in at logs and all this stuff and evalu way. And I think external auditors and internal auditors are going to get better at this as the years come by. In fact, if I were an enterprising auditor I’d want to really specialize in AI auditing. AI system auditing. So it could be very interesting, especially when AI can control very critical processes in certain industries. That’s going to be an interesting point.

Sanjog Aul [00:28:03]:
So governance is traditionally seen as an overhead, you know, in most cases because, and I’m not saying it’s not a necessary evil, it is and everybody should pursue, but still it’s seen it. And a lot of times when you have to make a case for it, whether internally in an organization or as you’re evangelizing this for the rest of the world to adopt, there has to be an ROI discussion on this. So what would all this bias and what all has this bought you besides a good sleep at night?

Vince Kellen [00:28:34]:
Well, the governance there’s many levels of governance and I remind folks that when the homeostasis of the technical system is in an extremely narrow band of output and extremely high quality band of output, you tend not to have governance. So we don’t have governance over the plain old telephone system. That governance has disappeared long ago, if there ever was. So as technology matures and it becomes extremely reliable, the governance needs begin to drop off. So we’re in this early state of AI that’s going to occur over the next couple decades.

Vince Kellen [00:29:13]:
So governance will be necessary. We’ll need to be evaluating that. And again, take this multi level approach to governance. There’s organizational governance. Are we doing the right things? Are we prioritizing the right work? Is the work that’s going on of suitable quality and safety? And that is necessary? You need to do that. And then when you get into the technical construction environment on how to technically build these things, you need all your normal software engineering checking that goes on. You can call it governance. It kind of is.

Vince Kellen [00:29:42]:
And so you need all those pieces. So yes, we need the governance and it’s not necessarily heavy. Governance doesn’t have to be heavy. I think most people are, and rightfully so, very concerned about the new technology. So new technology is always held to a higher level of scrutiny than the old. This exact same thing happened when the Internet hit. Everybody’s looking skeptically at the Internet and thinking about, okay, what governance should we put in place? And then immediately higher barriers or higher quality levels, creating barriers to entry for organizations to adopt the technology.

Vince Kellen [00:30:19]:
In the marketing world, when the Internet hit, everybody was concerned about, well, can you really measure this on the Internet? Is it really effective? In the early days of the Internet, people were looking very skeptically at whether the Internet would really be successful in marketing through a Lot of barriers that companies put up to adoption. Some of those companies put up. Those barriers are no longer here. And you’re seeing that the Internet world has taken over marketing from the rest of the world. So we’re going to do a similar thing here with AI. It allows what I call organizational aversion to adoption early, but in time it will grow as the technology matures in its quality.

Sanjog Aul [00:30:58]:
So AI governance is not new right from the very get go people concerned. So they started putting things in place and a lot of effort has been put in. So if you were to bring this human metabolism parallel to the AI governance in your model, how is that playbook being tweaked and what is your recommendation for any organization, organization which is already doing AI governance? What would they do any differently than what they’re doing now?

Vince Kellen [00:31:23]:
Yeah, my recommendation would be that the first place we go is governance of the priorities and the safety and quality of the overall system. From a very human standpoint, we need to shift quickly into how are we regulating the actual software construction and how are we ensuring that the agentic framework doesn’t go off the rails on us. And that might be largely a CIO concern. It’s the same thing about how do we make sure the software engineers aren’t having this go off the rails. My concern is the barrier to entry gets lower meaning by both vendors and us to develop agents. I think quite a bit concerned about the quality of those things. The good news is in the market, if a company buys agentic framework X from small company Y within the first six months of piloting, it’s suddenly not working. They’re going to reject it.

Vince Kellen [00:32:23]:
And word gets out that this product is having trouble. So there will be some self regulation sorts in the market, but we need to think kind of at lower levels of concern and certainly in the CIO world on this. So CIOs have to get deeper into the stack and make sure they understand the componentry under the hood. It’s not unlike if you’re going to buy a car and there’s pieces manufactured from other parts of the globe. Somebody’s got to understand where those pieces are coming from and if they’re of suitable quality as they go into the car. The other piece to this is our deployment methodology is shifting to what I call operations and is a continuous delivery sort of way in which, especially with new modules or new applications that you develop, you deploy to a very limited audience and you just slowly increment, slowly you incrementally increase the size of that audience. As you do this thorough feedback with the community around quality until everybody gets comfortable that this is operating within bounds. But then you’re also prepared to reverse that spiral and reduce the audience in size.

Vince Kellen [00:33:33]:
So it’s a much more dynamic, real time testing and feedback approach versus, you know, develop for three months or develop, go sprint for a month and test for two weeks and then release. This is a little bit more, not quite daily, but it’s, but it’s a much more fluid environment that we have to operate in. So I think the development methodology has to rise to the sophistication of the technology as well.

Sanjog Aul [00:33:59]:
So if you had to, since you have been evangelizing this concept and you want to appeal to the leaders within an enterprise to go ahead and take a look at it, and what’s the risk of them not doing it?

Vince Kellen [00:34:13]:
Well, the risk of not doing it is if you apply the AI to a very critical process and you misapply it and you underregulate it, you open yourself up to legal risk. There’s no question. So you’ve got to choose your targets that you’re going to release to carefully. And this is part of an article that came up with the Jagged Frontier of AI, which is still true. It’s not quite clear for many organizations will this process or will this application work in AI? And so you got kind of make sure you carefully do that. And then at the same time you got to get with the IT teams and the manufacturers of this and start to dig into their componentry and ensure you’ve got the levels of quality in there and transparency in it to the best that those vendors will want to do that. Yes, I think the, I don’t necessarily evangelize metabolism to everyone. I actually, I just was speaking to some accounting folks about metabolism, saying, hey, your world is going to get more complicated here.

Vince Kellen [00:35:21]:
I mean, you need to think about how you’re doing your accounting work in these more complicated ways. But the practicality of this is we’re all going to have to get more sophisticated in how we think about the control of the new technology because its degrees of freedom are far, far wider. The plain phone system has very few degrees of freedom. It’s really electrons moving down a copper wire and it’s just a voice signal on top of that. There’s tremendous variability that you can do with that here with AI, there’s a tremendous variability of the output that it can generate for us.

Sanjog Aul [00:35:55]:
Is this truly a tech project or is this a change management involved as well?

Vince Kellen [00:36:00]:
I think it’s both. It’s a change management, so you’re right. In fact, I think it’s largely now a change management problem. I think the technology is quickly sorting itself out. With this class of GPT technology, we get a replacement to the GPT, which I think there are some interesting concepts here that would create a new wrinkle. And now it’s a lot of change management. And instead of using the word change management, I’m going to call it adoption management, because you’re not trending this. I mean, yes, you’re always trying to get people to change, but you’re trying to get people to become comfortable with it and grow in their use of it.

Vince Kellen [00:36:39]:
Part of the way we’re approaching the AI is there’s two ways to encourage that adoption. One way is to help with a bespoke Assistant like our JavaScript helper, our fund manager approach, and our legal contract reviewer, where to the user it’s kind of a black box and it’s easy for them to interact with it. That’s one use of it. But many people have many different knowledge tests that they do on a daily basis that might be idiosyncratic to them, special to them, that could be aided tremendously with the generative AI. And so that’s going to require deepening adoption by the individual person. And so it’s also this, how do we encourage the adoption in a way that works wisely, correctly, and actually benefits the person and the organization at the same time? So I’m calling change management. We’re calling adoption management.

Sanjog Aul [00:37:30]:
If you had to appeal to the leaders within respective organizations that they go and take a second look at what you’re proposing, right? Draw a parallel between human metabolism and try to bring that in. What would you want them to start doing and thinking and saying differently for this to gain ground and actually get you get them the outcomes that they need to get.

Vince Kellen [00:37:57]:
And it depends on the audience. If I were talking to CEO of a $6 billion organization, I’d ask the CEO, do you know how many IT systems you have? Like 200 or 300. Okay, each of those has data. Do you know how many web pages, PDF documents and other loose content you have in your organization? Anybody come up with answer and says, we got 30 million of them, be great. How much of this knowledge is peculiar to your organization? Say, well, there’s a lot of knowledge this peculiar organization I want to protect and prove. So great you now have through AI, a tool that can ingest the sum total of all that and produce really interesting portfolio and control potentially over processes. You’re going to think that’s too much for me to think about? Absolutely. You’re going to need a regulatory framework and a construction framework or a purchasing framework that matches the level of sophistication in your current IT technology.

Vince Kellen [00:39:05]:
So you just don’t simply go to an provider and see or find me handling. You want to knit this into your organization so it’s really using your knowledge in a way that is enhancing your competitive advantage. What does it mean? You have to think deeply about that knowledge. You have to think deeply about the people in your company who are creating that knowledge and how they do it and how you wish to enhance and curate that over time. So now you’re going to have people who are just simply typing transactions into a system, but their mind will be interacting with the system and enhancing their mind while also enhancing their system. So now you have a technical human system that is getting really intimately connected. You need to have good regulation around that to make sure you’re not missing targets, overshooting targets, or going off the rails.

Sanjog Aul [00:40:00]:
Once again, thank you so much, Vince. This is a rather fascinating conversation we had today, you know, driving a parallel between human metabolism and how metabolism of knowledge happens in agenting AI or should happen in AI and what kind of guardrails we should put. So thanks so much.

Vince Kellen [00:40:18]:
Thank you. It’s been a pleasure as always.

Sanjog Aul [00:40:20]:
Thanks so much again, Vince and listeners. Please connect with us on social media, subscribe to our YouTube and podcast channels. Once again, this is Sanjog Aul, your host, signing off till next week. Take care and God bless.

Download Podcast
Apple Podcast, Spotify, iHeartRadio, TuneIn, and Pandora. Find other syndication channels here or search CIO Talk Network podcast on any other app.