Truly Agentic Means Reasoning, Not Rewritten Automation
Episode Summary
Ashish Bhatia argues that a lot of what the industry now calls agentic is the previous era’s automation rewritten in new frameworks. What makes a workload genuinely agentic, he says, is reasoning built in deeply enough to handle the corner cases that used to send engineers back to patch the rules, plus enough memory that the system does not suffer session amnesia between runs. From there he lays out three phases: humans assisted by AI, humans orchestrating agents they hand cognitive work to, and agents invited onto the team as coworkers in a hybrid workforce. Phase two, he says, is very much feasible today, and of phase three he points only at sparks. The line the rest of the conversation hangs on is his rule for getting there safely: before you own your AI, you should own your eval. He then applies it to model selection, arguing you start with the largest model and come down the capability curve.
Key takeaways
His opening claim is that the agentic migration is largely relabeling. The industry is transforming a lot of its automation workloads into agentic workloads, he says, and a lot of those are yesteryear’s automation rewritten in the new agentic frameworks
What makes a workload truly agentic, in his account, is where the thinking sits. You need reasoning built in deeply enough that the cognitive overload of building the automation moves off the human and onto the AI, he says, and that is what would make it truly agentic. The old way baked the rules into the system and let it run forever
The test he uses for whether reasoning is real is what happens at the edges. There are always corner cases nobody tackled, he says, which is why teams keep going back to fix their automations, and the promise of agentic reasoning is that those corner cases and outlier scenarios get handled gracefully instead. He calls that the key piece which is missing
The second thing he wants built in is memory, and he names the failure it prevents. Longer-term, memory and context retention matter so these systems do not suffer from session amnesia, he says, and can hold long-form context on what is going on and use it for the decision at the corner case, the way a person would
Phase one of his roadmap is assistance, and the label is Microsoft’s rather than his own coinage. Humans collaborate with AI agents and get augmented responses for whatever they are trying to get done, and he takes the phase one assistance name from Microsoft, where he had just come from
Phase two is where the human stops doing the work and starts dividing it. You offload some cognitive work to an agent while remaining the orchestrator, he says, breaking a large cognitive task into pieces and handing each to the right agent. His examples are deep research and code generation, and he says this is very much feasible today in a lot of companies
Phase three is the one he frames as a change of category rather than degree. You stop being an orchestrator of tools and invite agents onto the team, he says, moving from thinking about these things as tools to thinking about them as coworkers, digital employees, in what he calls a hybrid workforce where they do end-to-end work and call their own sub-tools
He does not claim phase three has arrived, but he points at the day’s news for the sparks of it. Newer models can use tools inside the reasoning workflow itself, he says, rather than reasoning, calling a tool, waiting for it to return and only then answering. Within the reasoning workflow they can call multiple tools and come back with an opinion
His model for building trust in an agent is the one every manager already has. You get a new employee, he says, and you treat an intern differently from a professional with five years of experience. The questions are the same either way: consistency of decision-making, consistency of outcomes, and trust built the way you would build it with a human colleague
The line the episode turns on is his rule about evaluation. Before you own your AI, you should own your eval, he says, and his reason is a question: if you do not own it, what do you own in the pipeline
Asked how you actually do an eval, he says the tool is beside the point. It is the determination of whether this is the cognitive task you want to offload and how you tell good outcomes from bad across the whole spectrum, he says, naming adversarial, wrong, hallucinated and inappropriate outputs as the classes to write down
Writing the eval down is what gives you somewhere to put a result. Being able to write down what constitutes good and not good for a given cognitive task gives you a baseline, he says, so that when an outcome is generated you can say which class it falls into
He runs the eval in two loops rather than one. In the dev or inner loop you tweak and improve until the agent reaches a threshold you are comfortable with, he says, and in the production or outer loop you monitor the runtime behavior in the wild, identify where things go off the rails, and feed those scenarios back into the dev loop before the next version ships
On model routing his answer starts with the consumer, and the reason is money. Users pick a favorite model and often reach for an expensive one to get a smallish task done, he says, so the serving companies can save a ton of dollars and tokens if they do the right routing and the right classification
For the enterprise he calls routing the same thing on steroids, and then names the cost. If you are not picking the model yourself it introduces another degree of variance, he says, where a model you pick is set in stone; a model picked on the fly becomes a new factor to validate, which is not necessarily bad but is another vector to check before you push
His worked example of routing already in production is Perplexity, and it doubles as the caution. You pick a tier there rather than a model, he says, and multiple models run under the hood before the answer reaches you, but those are Perplexity’s intentional choices. Hit a model router as your API layer and the variance sits in that layer, which is why he says you would want to own the classification yourself
His build-and-buy advice runs downward from the best model, not upward from the cheapest. Start with something off the shelf and pick the largest model you can to prove the goal is achievable, he says, then come down the capability curve model by model until you reach a breaking point, which is the most efficient price-performance model to run the scenario on
He connects that back to the eval, which is what makes the descent measurable. You can always pick the biggest and baddest model and not get value for your dollars, he says, so you run your evals across the models you are interested in, look at the outcomes, and let what you are paying in tokens decide
Asked what is over-hyped, he names prompt engineering and says the attention belongs one level up. We should think deeply about context engineering rather than just prompt engineering, he says, because models do not know your business or your data. Some of it is clever prompting, but a lot of it is bringing the right context
The leadership blind spot he names is building AI onto the old shape of the work. A lot of AI is duct tape AI, as he calls it, and rethinking AI-native use cases is what AI leaders need to think deeply about again
Where he expects the next transformation is vertical AI, and the reason is asymmetry. Models are becoming more open and more pervasive while data becomes more closed, he says, so enterprises sitting on large data have a unique ability to turn it into something valuable, which he expects to create a lot of new growth
His two leadership instructions are both about first-hand use. Be curious with AI and try the tools yourself, he says, because if you are not, somebody else is telling you what to do. And the fear of experimentation is something you can only tackle by experimenting more
About Ashish Bhatia
Ashish Bhatia is an AI product leader with more than twenty years in product management, across Microsoft and Nokia. At Microsoft he spent over a decade on AI platform work, helping bring generative AI to Power Platform and AI Builder and leading platform strategy for Copilot Studio, which is where the language of assistance, orchestration and hybrid workforce in this conversation comes from. He is now director of product management at Audible, part of Amazon, working on generative-AI-powered conversational discovery. He speaks and advises on agent platforms and responsible AI adoption. At the time of this recording he was leaving Microsoft and about to join Amazon, which is how he is introduced on air.
In this episode
| 00:42 | Welcome |
| 01:36 | What the shift from automation to agentification means |
| 01:42 | Yesteryear’s automation rewritten in agentic frameworks |
| 01:59 | Reasoning built in deeply enough to offload the cognitive overload |
| 02:30 | Corner cases, and why teams keep going back to fix automations |
| 02:54 | Memory, context retention, and session amnesia |
| 03:23 | Is full reasoning with no human in the loop premature |
| 04:13 | Phase one: humans assisted, and augmented responses |
| 04:39 | Phase two: the human as orchestrator, offloading cognitive work |
| 05:19 | Phase three: agents invited onto the team as coworkers |
| 06:08 | The announcement that day: tools used inside the reasoning workflow |
| 06:54 | What breakthroughs would make full autonomy safe |
| 07:20 | Onboarding an agent the way you onboard a new employee |
| 07:35 | Consistency of decision-making, and building trust |
| 07:58 | Before you own your AI, you should own your eval |
| 08:26 | The tool is a means: deciding what good and bad look like |
| 09:03 | Writing it down gives you a baseline to classify against |
| 09:12 | The dev loop and the production loop |
| 10:01 | What invisible model routing unlocks for the enterprise |
| 10:42 | Routing by the outcome, the question and the intent |
| 11:07 | Saving dollars and tokens with the right classification |
| 11:26 | For enterprise it is the same thing, on steroids |
| 12:09 | An aside on enterprises already running more than one model |
| 12:33 | Perplexity: you pick a tier, not a model |
| 13:46 | How this changes the build and buy decision |
| 13:52 | Start with the largest model and prove the goal is reachable |
| 14:37 | Running the evals across candidate models, and what the tokens cost |
| 15:06 | The lightning round |
| 15:21 | Over-hyped: prompt engineering, versus context engineering |
| 15:42 | Bullish for 2026: document understanding and processing |
| 15:55 | The blind spot: duct tape AI, and AI-native thinking |
| 16:26 | Vertical AI, and enterprises sitting on closed data |
| 16:55 | Asked where to follow him, and the answer the transcript loses |
| 17:05 | His defining edge: absorb as much as you can |
| 17:38 | Be curious, because otherwise someone tells you what to do |
| 18:15 | Wrap-up |
In Ashish’s words
“Before you own your AI, you should own your eval”
— Ashish Bhatia (07:58)
“A lot of those are yesteryears automation rewritten in the new agentic frameworks, if you will.”
— Ashish Bhatia (01:42)
“we are moving away from thinking about these things as tools to coworkers, employees, digital employees of the world”
— Ashish Bhatia (05:19)
“these things don’t suffer from kind of session amnesia as much, and have long-form context and long context of what’s going on.”
— Ashish Bhatia (02:54)
“how do we understand consistency of decision-making, consistency of outcomes, and then start to build trust with them”
— Ashish Bhatia (07:35)
“In the end, most users just care about the outcome. They don’t care about picking the model.”
— Ashish Bhatia (11:26)
“You can always pick the biggest and the baddest model, but you might not get the value for your dollars”
— Ashish Bhatia (14:37)
“A lot of AI is duct tape AI, as I call it.”
— Ashish Bhatia (15:55)
“Be curious with AI. You have to try these tools to be able to realize their benefit, because if you’re not, somebody else is telling you what to do”
— Ashish Bhatia (17:38)
Resources
Ashish Bhatia on LinkedIn: Where he posts, and the best place to follow his work on agent platforms and responsible AI adoption
Audible: Where he is director of product management, working on generative-AI-powered conversational discovery. He was about to join at the time of this conversation
The announcement he points to
Introducing GPT-5: Announced on 7 August 2025, the day this conversation was recorded, and the release he is pointing at when he says at 06:08 that there are sparks of phase three already. It is a unified system whose reasoning model uses tools as part of its thinking rather than reasoning first and calling a tool afterwards, which is the change he describes. He does not name it on air
Ideas and terms discussed
Yesteryear’s automation rewritten: His diagnosis, and the argument the rest of the conversation rests on. Organizations are moving automation workloads onto agentic frameworks, and a lot of what results is the old thing in new clothes. The difference he insists on is where the thinking happens: rules baked in by an engineer and left to run, versus reasoning deep enough that the system handles the case nobody wrote a rule for
Corner cases: His test for whether an agentic rewrite is real. Traditional automation sends its builders back repeatedly to patch the scenarios they did not anticipate. What he wants from agentic reasoning is those outliers handled gracefully at runtime instead, and he calls that the key piece which is missing rather than a thing already delivered
Session amnesia: His name for what memory is supposed to fix. Without retained context an agent starts each run from nothing, so the long-form context that a person would carry into a judgment call is unavailable exactly where the corner case needs it
The three phases: His roadmap, and the spine of the episode. Phase one is assistance, humans working with AI and getting augmented responses. Phase two is the human as orchestrator, breaking a large cognitive task apart and handing pieces to the right agent, which he says is feasible in a lot of companies today. Phase three is the hybrid workforce, where agents are invited onto the team as coworkers doing end-to-end work and calling their own sub-tools
Own your eval: The rule everything else in the conversation is measured against, and his own formulation. The eval is the written definition of what good and not good look like for a specific cognitive task, across the whole spectrum from adversarial to hallucinated to merely inappropriate. Written down, it becomes a baseline that any output can be classified against, which is what makes trust something you can build rather than assert
The dev loop and the production loop: How he wants the eval used, in two places rather than one. In the dev or inner loop you tweak until the agent clears a threshold you are comfortable with. In the production or outer loop you monitor runtime behavior in the wild, catch what is going off the rails, and route those scenarios back into the dev loop before the next version ships
Invisible model routing: A model picked for you, by intent, instead of by you. On the consumer side he thinks it saves the serving companies real money because users pick expensive models for small tasks. In the enterprise it is the same thing on steroids, with a cost: a model you choose is set in stone, and a model chosen on the fly is a new degree of variance to validate. His conclusion is that you would want to own the routing classification yourself
Down the capability curve: His method for choosing a model. Start with the largest model off the shelf and prove the goal is achievable at all, then step down model by model until you reach a breaking point, which is the most efficient price-performance model to run the scenario on. The evals are what make each step measurable, and what the tokens cost decides the rest
Context engineering: What he thinks the industry should be attending to instead of prompt engineering, which is his pick for the most over-hyped thing in generative AI. Models do not know your business or your data, so some of the work is clever prompting and a lot of it is assembling the right context to hand them
Duct tape AI: His phrase, offered without a definition, for the AI he thinks is added to work that was designed without it. He names rethinking AI-native use cases as the leadership blind spot holding back returns, and duct tape AI is the counterpart he wants replaced, which is the same argument one level up from context engineering
Vertical AI: Where he expects the next transformation, on an asymmetry he describes: models are becoming more open and more pervasive at the same time as data becomes more closed. Enterprises sitting on large data therefore hold something that is getting scarcer rather than cheaper
Named on air
Microsoft: Where he had just come from, named at 04:13 as the source of the phase one assistance language he uses for his own roadmap. He spent over a decade there on AI platform work
Cursor and Replit: His examples at 04:58 of phase two in production for code, alongside deep research from Google as the example for research
Perplexity: His worked example at 12:33 of model routing already shipped: you pick a tier rather than a model, several models run under the hood before the answer reaches you, and the choices are the vendor’s rather than yours
ChatGPT, o3, GPT-4.1 and GPT-4.5: The models he names at 10:26 and 10:42 when describing how a consumer picks by task, reaching for a reasoning model for research and a different one for creative writing. GPT-4.5 had been withdrawn from the API a few weeks before this conversation but was still in ChatGPT, which is where he says it is still available
Frequently Asked Questions
-
A workload is only truly agentic if reasoning is built into it deeply enough to handle the cases nobody wrote a rule for. Ashish Bhatia, an AI product leader who spent over a decade on AI platforms at Microsoft, says a lot of what is being called agentic is yesteryear’s automation rewritten in new agentic frameworks. The difference he draws is about where the thinking sits: traditional automation had its rules baked in by an engineer and then ran forever, which is why teams kept going back to patch the corner cases they had not anticipated. Agentic reasoning is supposed to handle those outliers gracefully at runtime instead. He adds memory as the second requirement, so a system does not suffer session amnesia and can carry long-form context into the judgment call.
Transcript 01:36 to 03:23
-
Enterprise AI agent adoption moves through three phases: assistance, orchestration, and a hybrid workforce. Ashish Bhatia describes phase one as humans collaborating with AI and getting augmented responses for what they are trying to do. Phase two is where a person offloads cognitive work to an agent while remaining the orchestrator, breaking a large task into pieces and handing each to the right agent, which he says is very much feasible in a lot of companies today and which he illustrates with deep research and code generation. Phase three is a change of category rather than degree: agents are invited onto the team as coworkers doing end-to-end work and calling their own sub-tools. He points at newer models using tools inside their reasoning as an early spark of it rather than evidence it has arrived.
Transcript 04:13 to 06:54
-
You build trust in an AI agent the way you build it in a new employee, by watching for consistency and by writing down what good looks like first. Ashish Bhatia says an intern and a professional with five years of experience get onboarded differently, and that the questions for an agent are the same ones: consistency of decision-making, consistency of outcomes, and trust earned over time. What makes that measurable rather than impressionistic is evaluation. His rule is that before you own your AI you should own your eval, and he puts the reason as a question: if you do not own it, what do you own in the pipeline at all.
Transcript 06:54 to 08:26
-
You evaluate an AI agent by writing down what good and bad outcomes look like for one specific cognitive task, then running that definition in two loops. Ashish Bhatia says the tool is a means to an end and the real work is the determination: deciding whether this is the task you want to offload, and mapping the whole spectrum of outputs, from adversarial to wrong to hallucinated to merely inappropriate. Written down, that becomes a baseline you can classify any result against. He then runs it in a dev loop, tweaking until the agent clears a threshold he is comfortable with, and in a production loop, monitoring runtime behavior in the wild and feeding the failures back into the dev loop before the next version ships.
Transcript 08:26 to 10:01
Item description -
Invisible model routing is a system picking the model for a given request based on the intent behind it, rather than the user choosing one. Ashish Bhatia expects it to arrive on the consumer side first, because people pick a favorite model and often reach for an expensive one to get a smallish task done, so the serving companies save real money on the right classification, and because most users only care about the outcome. For enterprises he calls it the same thing on steroids and names the cost directly: a model you choose is set in stone, while a model chosen on the fly introduces another degree of variance to validate. His example of it already shipped is Perplexity, where you pick a tier rather than a model, and his conclusion is that an enterprise would want to own the routing classification itself.
Transcript 10:01 to 13:46
-
You find the cheapest model that still clears your evals by starting at the top and working down, not by starting cheap and working up. Ashish Bhatia advises picking something off the shelf and using the biggest available model first, purely to establish that the goal is achievable at all. Once the problem is provably solvable, he comes down the capability curve model by model until he reaches a breaking point, which he says is the most efficient price-performance model to run the scenario on. He is explicit that you can always pick the biggest and baddest model and still not get value for your dollars. Evals are what make the descent measurable: run them across the models you are considering, look at the outcomes, and let the token cost decide.
Transcript 13:46 to 15:06
-
Yes. Prompt engineering is the thing Ashish Bhatia names as most over-hyped in generative AI, and he argues the attention belongs on context engineering instead. His reason is that models do not know your business or your data, so the constraint is rarely the wording of the request. Some of the work is clever prompting, he says, but a lot of it is assembling and supplying the right context. He makes a related point one level up when asked about leadership blind spots: a lot of AI is what he calls duct tape AI, a phrase he offers without defining, and rethinking AI-native use cases is what he thinks leaders need to return to instead.
Transcript 15:06 to 16:26
-
[00:42] Christina Ellwood: Welcome to AI Realized, the podcast for enterprise executives leading AI deployments. From tackling security, data, and operational challenges to navigating organizational transformation, AI deployment offers a unique opportunity to redesign organizations from the inside out. I'm Christina Ellwood, your host for today's episode, and we're talking today with Ashish Bhatia, AI product manager who was most recently with Microsoft and will soon be joining Amazon. Ashish, welcome to AI Realized
[01:16] Ashish Bhatia: Thank you so much, Christina. Pleasure to be here. I've been a listener, and it's great to be on the podcast today.
[01:23] Christina Ellwood: We are really excited to have you. I'm, uh-- I recently listened to another interview that you gave, so I know that you've got some thinking to share with the executives in our audience about the shift from automation to agentification. What does that mean in real enterprise terms, and how close are we to truly autonomous agents?
[01:42] Ashish Bhatia: One of the things that I feel, ooh, the industry is grappling right now is transforming a lot of their automation workloads to agentic workloads. A lot of those are yesteryears automation rewritten in the new agentic frameworks, if you will. But for them to be truly agentic, I generally try to underscore a few things. You need to have reasoning deeply built into it so that we're offloading the cognitive overload of building those automation from human, and offloading that to AI as much as possible. That would make it truly agentic. What it does is, when we were building these automation ourselves, we were literally baking in those rules into those automation systems and then let them run forever. But we know that there are always these corner cases that we might have not tackled, and that's the reason why we go back and fix those automations. But for a new age automation, which is agentic in nature, those re-- kind of corner cases hopefully will get tackled by the agentic reasoning capability, which is most promising for me, and that's why I feel that is the key piece which is missing. And once that starts to land, we will see some of this in a more graceful way, all these corner cases, outlier scenarios, and things like that. In long term, one of the other thing that would come into play is memory context retainment, so that, again, these things don't suffer from kind of session amnesia as much, and have long-form context and long context of what's going on. And that context can be leveraged for that decision-making, that corner case, just like a human would do in a real-life situation.
[03:23] Christina Ellwood: It seems like the context setting is the tricky bit in setting up the agents. And having the agents being responsible for all of the reasoning with no human in the loop may be a little premature, especially given that the models are willing to refuse to make changes and things like that. Where do you think-- guide, if you will, the audience on how they should think about the way to use agentic today, like how much is human in the loop today, given the limitations on guardrails and security and things like that. What's the kind of middle ground where there's been some improvement or change in the risk profile for agents? And then when are we really able to go fully agentic?
[04:13] Ashish Bhatia: There are, I would say, three horizon things if we look at it. We started off with humans Collaborating with these AI agents, we call that at Microsoft phase one assistance, right? Where humans are working with sub sort or whatnot, right? And then really getting augmented, uh, augmented responses for whatever they're trying to get done. Phase two thinking is where you're able to offload some cognitive work to an agent. Deep research is always a good example that I use for researching code gen, for now code writing and things like that. Those are good examples of cognitive work that you can offload. In that case, you're still the orchestrator. That is kind of phase two way of thinking about it, that humans are the orchestrator, and they're breaking apart a large cognitive task into multiple pieces, and then offloading to the right agent. That is very much feasible today in a lot of companies. Cursors of the world, Replit of the world for code, and then deep research and other kind of deep thinking from Google. They are offloading a lot of the research work to these agents. Way-- The phase three way of kind of thinking about this is that from this orchestrator of agents, you invite agents to your team in collaboration work, right? Where you almost are using these agents as a coworker, right? So we are moving away from thinking about these things as tools to coworkers, employees, digital employees of the world, right? And from an orchestrator of tools, you then go to a place which is hybrid workforce, right? Where humans are working with digital employees to get tasks done. These folks are a- or these tools are able to do end-to-end work themselves, and they are able to use their own then sub tools to get stuff done as well. Again, there are sparks of it already today, right? If you look at some of the reasoning, even the announcement that happened today, the announcement that l- newer models are able to use tools in the reasoning workflow itself versus having to reason and then call a tool, right? Within the reasoning workflows, they're able to leverage tools, call tools, and get stuff done, and then come back with an opinion, which is different from past, right? Where it would reason about something, and then call a tool, and then wait for that tool to come back, and then offer that response back to you. It can now call multiple tool within its reasoning kind of workflow or, or chain of thought, if you will.
[06:47] Christina Ellwood: That is, that's a great description of the capabilities at the different levels. I'd like to hear a little bit more from you about what you think the technological breakthroughs need to be in order to make it safe for people to fully automate or use fully autonomous agents, and how do we deal with things like stubbornness in an LLM when in an agentic world?
[07:13] Ashish Bhatia: Yeah. That is a challenging problem, and the way we could humanize it to be, to an extent is think about it this way. You get a new employee. Again, for different levels, we can think of them differently. If you get an intern, how do you treat them or work with them, or how do you grow them within the organization versus you get a professional with five years of experience that know what they're doing and stuff like that. How do you groom them and onboard them within the organization? So think about it similarly, that as we are bringing these tools on board in a team setting or whatnot, how do we understand consistency of decision-making, consistency of outcomes, and then start to build trust with them, right? Very similar to how you would have worked with a, a human employee. The key there is, and the key always is, how do you eval what the outcomes are? And it always starts for me there, right? And I always say, "Before you own your AI, you should own your eval," because if you don't own the AI, you're not owning the eval, then what do you own in the pipeline?
[08:15] Christina Ellwood: I love that. How do you-- what-- how do you do the eval? Is there a tool? Is there a process? What are you doing on the eval to own your eval before you, you do your AI?
[08:26] Ashish Bhatia: Yeah. Tool is m- means to get somewhere, but mostly it's the determination of if this is the cognitive task that we want to offload it, how do we determine good outcomes and bad outcomes and its entire spectrum, right? Whether we are looking at adversarial outputs or wrong outputs, hallucinated outputs, inappropriate outputs, whatever those are, and there are a ton of them, right? So ability to be able to write down those evals of what constitutes good and not good, and then different aspects of good as is talked about for a given cognitive task. Like being able to write that down gives you a baseline of if some outcome is generated, which class would it fall in, right? And then based on that, and this is both inner loop, outer loop or dev loop and production loop. You could deploy this in a dev loop first, right? To see how is an AI model or an agent performing. And in the dev loop, you have the opportunity to tweak it, improve it, and bring it to a threshold which is most comfortable. But then you can also use this in a production or outer loop as well. When once deployed, you can then now monitor and observe the runtime behavior or the dynamic behavior of the agent in the wild, and then also identify the need to then do a feedback loop where you identify some of these scenarios where things are going off the rails, bring it back, going back into the dev loop, fine-tune things again before you can deploy the next incremental output of out-- or next incremental version of it.
[10:01] Christina Ellwood: Agents are able to use tools, but they're also able to use multiple models. You've spoken about a future where models are selected automatically without users needing to know what's running. What does this invisible model routing unlock for the enterprise?
[10:01] Ashish Bhatia: Interesting question. I think model routing, where we will start that first is more on the consumer side of things. But then we can talk about how it would manifest itself in the, in the enterprise side as well. So for-- from a consumer standpoint, it is you go to ChatGPT, your favorite chatbot. Today, you pick a model, whether even for Cur-Cursor and other things, right? You pick a model, which is your favorite model, and you work through it, right? And for different tasks, you pick different. When I'm reasoning, I'll pick a deep research or o3 model versus if I'm doing more creative writing kind of thing, I might even pick a 1., a 4.1 or 4.5 sometimes, I mean, it's still available, ChatGPT. So a lot of that decision-making is on us, and there is starting to-- this concept's starting to manifest, which is model routing based on the outcome, the question, the intent, the right model will be picked. And from a consumer standpoint, that makes a ton of sense because as a user, I always probably might not know what is the right model. I might be using a more expensive model to get this kind of smallish task done and whatnot. So the serving companies can save a ton of dollars, tokens, if they do the right model routing, right classification. In the end, most users just care about the outcome. They don't care about picking the model. Sometimes it's just a default model and stuff like that. For enterprise, it is the same thing, but on steroids, right? But it offers some challenges that if you're not picking your model by yourself, then it introduces another degree of variance now, which otherwise when you pick the model, you know the model is set in stone. Of course, you will graduate the model and upgrade the models, and those are cycles that you go through, and there are deep validation kind of pipelines built into it. But if your model is being picked on the fly, it offers a new variance, a new kind of factor to now go validate that is the right model being picked for the task. Which is not necessarily a bad thing, but again, another vector to validate before you push things out, and you would have to build trust and confidence in that.
[12:09] Christina Ellwood: Most enterprises, according to the pr-primary research, are using more than one model today.
[12:09] Ashish Bhatia: Yes.
[12:09] Christina Ellwood: And some of those models are behind their firewall as opposed to being used in a public model with private data. So they have the challenge today of picking between the models. They're just not doing it automatically. So it connects into the agents, doesn't it?
[12:33] Ashish Bhatia: Does-- That would be- Yeah. Yeah. You pick Perplexity as an example, right? We go to Perplexity. We don't pick a model there. We pick a tier, pro tier, what tier, right? You pick a tier, you're not picking the model. So a model is being picked on your behalf. But under the hood, it, before Perplexity serves you the response, it has... They might even have proprietary models that they are leveraging to get smallish tasks done. Go search the web Uh, rank the outcomes from the web search, right? Extract the right bits from there. So there are multiple models playing under the hood before the answer is served to you. But those are intentional choices of models that Perplexity has picked up. But as an enterprise, for a given scenario, if you're using model router as the API layer that you are hitting, now there is variance in just that layer itself, right? You don't know how to... by model which model is picked for the task, and the router is doing the classification. You would want to own that classification as well at some point of time, like which task should be routed to which model and so on and so forth.
[13:43] Christina Ellwood: Yeah. I, I definitely see that's important on the enterprise. So how does this change the build, buy decision for AI teams?
[13:52] Ashish Bhatia: Hmm. I honestly, my, and this might be a biased kind of point of view from place where I am coming from, is I always advise folks to start with, with something which is off the shelf, right? Go pick the biggest, largest model that you can, and ensure that the goal that you're trying to achieve is realized, right? You know that- Your problem can be solved by a given model, right? And then you pick the best model for now. And then come down the capability curve, right? The next, pick the next model, next best model, and so on and so forth until you reach a breaking point, and that's the most efficient price performance model, uh, that you can then run the scenario on. You can always pick the biggest and the baddest model, but you might not get the value for your dollars, right, at that point of... Those are kinda some of the decisions you would have to make. Again, an important thing in that is your evals will come into play because you will run your evals through. Here are three models that I'm interested in them or want to run through, see what the outcomes are, and then however much you're paying for those tokens will then determine which model you pick to make the right decision.
[15:06] Christina Ellwood: Great. I'm gonna go into, uh, because we're reaching the end of our time, I'm gonna go to a little rapid fire round. Okay. A little lightning round. So quick answers to these questions. What's one trend in generative AI you think is over-hyped right now?
[15:21] Ashish Bhatia: Prompt engineering. I feel we should deeply think about context engineering versus just prompt engineering, because models don't know your business, your data, right? You have to define the right context for them. Some of it is clever prompting, but a lot of it is just bringing the right context.
[15:39] Christina Ellwood: What's one use case you're bullish on for 2026?
[15:42] Ashish Bhatia: Enterprise use case. I think document understanding, document processing is something that will be definitely solved in the long term, at least next year.
[15:51] Christina Ellwood: What's one leadership blind spot that's holding back AI ROI?
[15:55] Ashish Bhatia: I would say is rethinking AI native use cases. A lot of AI is duct tape AI, as I call it. And AI native thinking is something which AI leaders need to re... again, deeply think about.
[16:13] Christina Ellwood: Where do you see AI transforming the enterprise in the coming years? Give me a really specific spot in the enterprise where you feel AI is going to unlock a transformation that has not yet happened.
[16:26] Ashish Bhatia: I think vertical AI has a lot of promise. As models are becoming more open, more pervasive, even as data is becoming more closed. Enterprises which sit on large data, they have a unique ability to transform that data into something very valuable, and I see that is something that will create a ton of new growth in enterprise.
[16:49] Christina Ellwood: Definitely. What resources do you recommend to our listeners who wanna learn more about you?
[16:55] Ashish Bhatia: Then that is my favorite kind of method of communicating externally.
[16:59] Christina Ellwood: Okay. And in the AI revolution, what's your defining edge as an AI leader?
[17:05] Ashish Bhatia: Absorbing as much as you can, and I feel that is easier stated than done. But again, read, listen. Again, contrary opinions, both sides of the aisle. AI is safe, AI is gonna challenge, there's challenges there, all of those. I think that'll help you form your thought. But again, embrace as much as you can, absorb as much as you can. I think that'll help you.
[17:29] Christina Ellwood: Do you think there are specific leadership skills that are particularly useful and germane for leading AI in the enterprise?
[17:38] Ashish Bhatia: Absolutely. I feel two or three key skills that AI leaders should have at this point of time. Be curious with AI. You have to try these tools to be able to realize their benefit, because if you're not, somebody else is telling you what to do, right? So that's one thing. Engage and embrace what's going on there. And then the fear of experimentation, I feel that's something that you can only tackle by doing more experimentation. So do as much experimentation that you can do to be able to see, hey, what's working, what's not working for you. The, again, those are things that I would highly recommend.
[18:15] Christina Ellwood: Thank you so much, and thank you, Ashish Bhatia, the AI product manager extraordinaire. I can't wait to see what you're gonna be doing next.
[18:25] Ashish Bhatia: Thank you, Christina. It was a pleasure being here.