Extend Data Governance Into Models, Then Into Agents
Episode Summary
Kevin Petrie leads the data management practice at BARC, and his argument is that AI governance is not a new discipline but an old one with two more domains. Traditional data governance, which he says few companies claim to have mastered, covers quality, privacy and intellectual property. Models add clean predictions and outputs. Agents add behavior, and he cites OpenAI’s finding that reasoning models watched too closely start hiding what they are doing. The control technology companies actually rely on is not a tool: humans in the loop, approving outputs before they go into the wild. On autonomy he is blunt. Not much at this point, and he does not think there should be; most agents, he says, are a new word for copilots. Later he turns prescriptive: a tiger team from the data, AI and developer groups, starting from business pain, and a warning that pilots stall when data that was clean for one department gets dirtier at scale.
Key takeaways
The obstacle he names second is people, and he names it twice. A recent BARC survey found that the number two obstacle to success with AI is a lack of AI skills inside the organization, and the responses he sees are retraining workers, hiring, which he says can be expensive if you want the right AI-savvy people, and showcasing use cases and best practices through centers of excellence
The argument for data governance is not new, and that is his point. Longstanding data people have been advising executives for decades that data governance needs more investment, he says, and data quality is a longstanding challenge, as are privacy risks, so all the aspects of data governance that really get to trust have not been getting enough attention
What AI changes is who is listening. Executives at the CEO and board level are starting to realize that AI can have spectacular problems if you do not double down on the data fundamentals, he says, which makes this a good time for data people to have a renaissance in redoubling and extending their governance programs
He does not claim the starting point is solid. Traditional data governance is not a discipline many companies would claim to have mastered at this point, he says, and its traditional risks are data quality, data privacy and protection of intellectual property
The extension has two more domains, in order. Organizations need to move from data into the domain of models, working out how to make sure predictions and outputs are clean and safe, and then into a third domain, agents, working out how to make sure an agent’s behavior does not go rogue or have subversive intentions
His evidence that agents need a domain of their own comes from model builders. He says OpenAI has been forthright that its reasoning models, if monitored too closely by humans, will start to try to deceive humans and hide what they are doing, and that there are other models out there that will rewrite their own code in order to avoid being shut down
The control that technology companies actually use most is not a tool. Looking at software companies whose crown jewels are built on automation and AI, he says the number one control they use for AI governance is humans, human in the loop
Who owns AI governance is a partnership, not a transfer. Those programs need to be addressed comprehensively, he says: chief data officers, who he thinks are playing catch-up, partnering with chief AI officers and chief analytics officers so that data engineers and data stewards work with data scientists and machine learning engineers, and with AI-savvy developers once the work becomes agentic
There are two routes into building agents and he is even-handed about both. Vendors that understand how daunting agentic AI is, and he names Boomi and Dataiku as BARC clients doing this, offer pre-made templates that let business-oriented rather than technically oriented people put agentic workflows in place. A separate group of specialists builds custom workflows, which he calls higher risk but potentially higher reward because it stitches agents into proprietary workflows and datasets
Asked how much autonomy companies are building in, his answer is two sentences long. Not much at this point, and he does not think they should. He adds that the strict definition of an agent is a bot making and acting on decisions independently of humans
His most deflationary claim is about the word itself. In most cases those agents are really a new word for copilots or even traditional software, which has been automating workflow for 30 years, he says, and they can and should be going back to humans for approval before making a purchase or closing a transaction
The reason he wants caution is compounding, not any single failure. One failure could have a pretty big cascading effect, he says, once multiple agents are transacting with each other autonomously, so companies can and should step cautiously into that realm and work with strict guardrails and carefully defined use cases to start
The new capabilities he wants in the data layer use AI to watch AI. Clean inputs are a good start and clean outputs help at the model level, he says, but you can also use AI to monitor what agents are doing, with sentiment indicators or natural language processing to catch potentially toxic output an agent might be putting into place
Above the monitoring he wants something deterministic, and he was writing about it at the time. Rules-based checks that will block transactions or actions deemed unsafe, he says, which is a new layer and a new domain: governance of agents. He mentions he is working on a report on that topic right then
His answer to what companies should actually build is unglamorous. The goal should be to make incremental changes to existing processes rather than trying to rip and replace, he says, because most Fortune 1000 organizations have had systems in place for a very long time, including mainframes on premises that some are still increasing
He puts a number on where the second wave of value is expected to come from. After the efficiency and productivity goal, the next big bucket is still probably half of companies, looking to improve revenue and delight customers in new ways. That second bucket is where he says you want a sandbox project with innovative people, and smart employees vetting any outputs before they go anywhere near customers
The first reason pilots stall is that the data changes underneath you. Companies struggle moving from lower-scale to higher-scale production, he says, usually because they had clean data inputs that were good for a given department or data type, and as they go broader the additional data coming in is not as clean and requires real work
The second and third reasons are people and money. The people who staff a sandbox project may be younger and carry less baggage, he says, but encouraging the rest of the organization to use AI responsibly can be a real hurdle. Separately, AI inference costs, while they are coming down sharply, could still be a significant surprise cost similar to cloud usage, where you get the surprisingly high bill at the end of the month
What drives that cost is consumption. As you go broad and a lot of humans in the organization are consuming AI, he says, token usage costs for each unit of data consumed can drive up costs in surprising ways, and organizations need to figure out how to govern that
His example of existential rather than operational risk is a named consultancy. Accenture’s stock is down, he says, because clients are on the margin saying they will ask ChatGPT for pretty rich advice and get pretty solid advice for 20 bucks a month, which defers the need to hire Accenture for some strategic consulting
He volunteers a position he expects to be unpopular. He says he is maybe a little controversial in his view that there should be some regulation of model outputs at this point, and cites Grok the week before as a spectacular governance problem, a model on a public social media platform encouraging millions of people to attack and harm an individual
Where he thinks advantage actually lives is the third stage. The first stage was knowledge workers using ChatGPT, the second was models inside commercial tools, and the third is applying language models to proprietary data and proprietary processes, which is where he says you uncover real sustainable competitive advantage rather than productivity gains anyone can achieve. The common pattern he sees is a public model with carefully architected retrieval-augmented generation underneath
His prescription for getting started is a team and a starting point, in that order. Have the CEO delegate to an executive sponsor just below, a chief data officer, chief AI officer or chief operating officer, and have them build a tiger team of leaders from the data, AI and developer groups. Then start with the business: a line of business or product group identifying what hurts today and where there is too much friction or manual work
He tells that team not to assume the answer. They should not rush to say that GenAI is always the answer, he says, because it might not be, it might be traditional machine learning, or it might be other things
His two examples of creative reporting lines both move AI out of technology. He was struck that Jamie Dimon at JPMorgan Chase took the head of AI out of tech so that she is head of data and AI, reporting to him and the president rather than to the CIO. The other is Moderna, which promoted its chief HR officer to manage AI and IT, which he reads as figuring out at the executive level how people work with robots
The leadership skill he names is not a technology skill. The goal, at the end of the day, is to increase the value that each worker delivers in a given hour, he says, which is why we are better off than our grandparents and they were better off than theirs. He traces the instinct to being an economics minor who taught macroeconomics in graduate school
His reality check is a productivity number. In a good year we boost the value of economic output per worker hour by three or four percent, he says, and it actually went down last quarter, so we are in early innings. What may be different this time is speed, because the power of models is going up while the cost of inference comes down
His closing answer puts the two obstacles in order. Human skill gaps are the number two obstacle to AI success in BARC’s research, he says, and number one is data quality. His instruction is to invest in your people and help them use AI to augment their work, and he tells young people that AI is an incredible opportunity in data science, so that organizations can double down on the advantages that do not change: human relationships, the power of brands, and the power of trust
About Kevin Petrie
Kevin Petrie is VP of Research at BARC, the research and consulting firm he describes on air as global, where he leads the data management practice across research, consulting and end-user events in Europe and the United States. He writes about AI, data integration and data governance, and BARC’s own team page credits him with roughly 30 years of technology analysis and instruction behind that: he built a data analytics services team for EMC Pivotal across the Americas and EMEA, ran field training at Attunity, now part of Qlik, and has co-authored two books on data management. On this episode he mentions that he was an economics minor and taught macroeconomics to younger students in graduate school, which is where the argument he closes on comes from. He points listeners to two places, his LinkedIn profile and the BARC site, and he spells the company out on air as B-A-R-C.
In this episode
| 00:42 | Welcome |
| 01:25 | His role: leading the data management practice at BARC |
| 01:42 | Data management as what holds the key to AI success |
| 02:12 | The business implications, given safe and effective data management |
| 02:29 | Why AI deployment asks for cultural change, not just technology |
| 03:11 | The number two obstacle to AI success: a lack of AI skills |
| 03:49 | Trust with customers, employees and the board |
| 04:07 | Fortune 500 brand loyalty at a high, and protecting it |
| 04:56 | Mastercard Agent Pay, and playing with the crown jewels |
| 05:20 | Decades of arguing that data governance needs more investment |
| 05:42 | A renaissance for data people |
| 06:29 | Traditional data governance, a discipline few would claim to have mastered |
| 06:47 | Extending from data into models, and then into agents |
| 07:10 | Reasoning models that deceive, and code rewritten to avoid shutdown |
| 07:34 | The number one AI governance control at technology companies is humans |
| 07:56 | Employees approving outputs before they go into the wild |
| 08:24 | Chief data officers partnering with chief AI officers |
| 08:57 | AI-savvy developers, once the work becomes agentic |
| 09:35 | Boomi and Dataiku, and templates for business users |
| 10:06 | Custom agentic workflows: higher risk, higher reward |
| 10:29 | Not much autonomy, and he does not think there should be |
| 10:46 | Agents as a new word for copilots |
| 11:31 | One failure, and the cascading effect |
| 11:54 | Using AI to monitor agents, and catching toxic output |
| 12:19 | Rules-based checks that block unsafe actions |
| 12:43 | The roundtable model: agents as the interface between systems |
| 13:24 | What an agent is for: orchestrating a sequence of tasks |
| 13:59 | Mortgage applications, invoice processing, and the drudgery |
| 14:22 | Two use cases from the Agents at Work Roundtable |
| 15:07 | Enterprises prefer the vendors they already trust |
| 15:31 | Incremental change, not rip and replace |
| 16:32 | Take the friction out, then the revenue bucket, and vetting before customers |
| 18:16 | Innovations take time, and the 1800s |
| 18:36 | Data that gets dirtier as you go broad |
| 19:01 | The rest of the organization, and inference cost as a surprise bill |
| 19:31 | Token costs for each unit of data consumed |
| 19:48 | Organizations do not like surprises of any type |
| 19:52 | The opposite surprise at the Economics of AI Roundtable |
| 20:43 | Unknown unknowns, and how much can really be automated |
| 20:59 | Accenture, and clients asking ChatGPT instead |
| 21:40 | The case for regulating model outputs |
| 21:58 | Grok, and a model directing harm at an individual |
| 22:19 | The sycophancy rollback, and models that still need work |
| 22:46 | Own your own intelligence, and Paul Baier at GAI Insights |
| 23:30 | The three stages of the GenAI boom |
| 24:06 | Proprietary data, and where advantage actually comes from |
| 24:40 | The public model with retrieval-augmented generation underneath |
| 25:08 | An executive sponsor below the CEO, and a tiger team |
| 25:35 | Starting with where the business hurts today |
| 25:58 | Why GenAI is not always the answer |
| 26:42 | Jamie Dimon taking the head of AI out of tech |
| 27:01 | Moderna, and HR managing AI and IT |
| 27:58 | An economics minor, and the lessons of history |
| 28:22 | Value per worker hour, and why we are better off than our grandparents |
| 28:51 | Three or four percent in a good year, and the quarter that went down |
| 29:42 | Do not get dizzy, take a pragmatic look |
| 30:00 | Train people, and data quality as the number one obstacle |
| 30:50 | Where to find him: LinkedIn, and the BARC site |
| 31:10 | Wrap-up |
In Kevin’s words
“If you have smart, vigilant employees who are safeguarding and approving outputs before they go into the wild, that’s the safest way to make sure that you’re not undermining trust with customers and the public.”
— Kevin Petrie (07:56)
“I think in most cases, those agents are really a new word for copilots or even traditional software, which has been automating workflow for 30 years.”
— Kevin Petrie (10:46)
“Not much at this point, and I don’t think they should.”
— Kevin Petrie (10:29)
“This can be a good time for data people to have sort of a renaissance in redoubling and extending their governance programs to address those risks and make sure they have clean inputs for AI models.”
— Kevin Petrie (05:42)
“Companies can and should step cautiously into that realm and really work with strict guardrails and carefully define use cases to start.”
— Kevin Petrie (11:31)
“I think there should be some regulation of these model outputs at this point.”
— Kevin Petrie (21:40)
“That’s where you could uncover real sustainable competitive advantage rather than productivity gains that anyone can achieve.”
— Kevin Petrie (24:06)
“The goal is to increase the value that each worker delivers in a given hour.”
— Kevin Petrie (28:22)
“Don’t get too dizzy about the power of what Grok or these other things are handing to you. Take a pragmatic look at your existing business and figure out how to enhance it.”
— Kevin Petrie (29:42)
“Human skill gaps are identified as the number two obstacle to AI success in our research. Number one is data quality.”
— Kevin Petrie (30:00)
Resources
Kevin Petrie
Kevin Petrie on LinkedIn: His profile, and the first thing he offers when asked where listeners should go. He says he is pretty active there promoting BARC research and engaging with their user community to unpack trends
BARC: Where BARC publishes its research. He spells it out on air as B-A-R-C and describes it as a global research and consulting firm
The agent governance work he mentions at 12:19
The Risks and Governance Requirements of Agentic AI: Part one, published 15 July 2025, days after this conversation was recorded. It sets out the same three risk domains he gives at 06:47, data then models then agents, and recommends toxicity testing, explainability, monitoring, kill switches and threshold alerts
The Tricky Discipline of Governing Agentic AI: Policies, Rules, and Standards: Part two, published 13 August 2025. It lays policies, rules and standards across the same three domains, and argues that governed well, agentic AI is an innovation opportunity, and governed poorly it damages the business
Agentic AI Governance: 4 Criteria to Evaluate Tools: Part three, published 18 August 2025, ten days before this episode went out. The four criteria are governance capabilities, productivity and ease of use, ecosystem support including agent-to-agent protocols, and performance and scalability
BARC: Modernizing Governance for the Era of Agentic AI: The BARC report the series belongs to, sponsored by Dataiku, whom he names on air as a BARC client. Its own description is extending governance from data to models to agents. It sits behind a form
Also from BARC
Lessons from the Leading Edge: Successful Delivery of AI/GenAI: BARC research published 3 December 2025, across 421 organizations, which found data quality the top obstacle for 44 percent, having been a secondary concern the year before. It is later than this conversation and it puts a figure on the ranking he gives at 30:00
Ideas and terms discussed
The three domains of AI governance: His organizing idea, and the spine of the episode. Traditional data governance covers data quality, privacy and intellectual property, and he says few companies would claim to have mastered even that. Models are the second domain: making sure predictions and outputs are clean and safe. Agents are the third and newest: making sure behavior does not go rogue or turn subversive. The prescription is extension rather than replacement, which is why it starts from the program data teams already run rather than from a blank page, and why he wants it run jointly with the AI and developer groups rather than by any one of them
Human in the loop: The control he says technology companies actually rely on most, and the answer to what you use when the model layer cannot be trusted. Looking at software companies whose crown jewels are automation and AI, he says the number one control they use for AI governance is humans. His version of it is specific: smart, vigilant employees safeguarding and approving outputs before they go into the wild, and agents going back to humans for approval before making a purchase or closing a transaction
Agents as a new word for copilots: His deflationary read on the category. The strict definition of an agent is a bot that makes and acts on decisions independently of humans, but in most cases, he says, what is being shipped is a new word for copilots or for traditional software that has been automating workflow for 30 years. He is not dismissive of the ambition, only of the labeling and of the pace
Governance of agents as a new layer: What he wants in the data layer specifically, beyond clean inputs and clean outputs. AI monitoring what agents do, sentiment indicators or natural language processing to catch toxic output, and rules-based checks that block transactions or actions deemed unsafe. Deterministic blocks and probabilistic monitoring, doing different jobs
The beachhead: His word for how to start without betting the business. Layer new incremental intelligence onto the parts of your existing data architecture that work, create a beachhead of something new, prove it at limited scale, then expand. The counterexample he names is rip and replace, which he advises against, because most Fortune 1000 companies have had their systems in place for a very long time and a lot of the old stuff, mainframes included, does not go away
The tiger team: The structure he advises. The chief executive delegates to a sponsor just below, a chief data officer, chief AI officer or chief operating officer, who builds a team from three groups: data engineering leaders, data science and AI leaders, and developers. It then works with a specific line of business, region or product group, and it starts from what hurts today rather than from the technology
The three stages of the GenAI boom: His map of where competitive advantage is and is not. Stage one is knowledge workers using ChatGPT. Stage two is models arriving inside commercial tools. Stage three is applying language models to proprietary data and proprietary processes, and it is the only stage that produces advantage another company cannot simply buy. The pattern he sees doing it safely is a public model with carefully architected retrieval-augmented generation underneath
Token cost as the next surprise bill: His third reason pilots stall, and the one framed as finance rather than technology. Inference costs are falling sharply, but consumption scales with the number of humans using AI and with each unit of data consumed, so the total can behave like a cloud bill: correct, unbudgeted and delivered at the end of the month
Value per worker hour: What he anchors executives to when the technology gets dizzying, and the reason he reaches for economics rather than technology history. The goal is to increase the value each worker delivers in a given hour, which is why each generation has been better off than the one before. Against that measure he puts three or four percent in a good year, and a decline in the quarter before the recording, as evidence of early innings
Named on air
Mastercard Agent Pay: His example of a company with a great deal of customer trust taking a bold step anyway: an offering that helps customers shop, weigh options, decide and even process transactions. He calls that playing with the crown jewels and says it will be interesting to see how it pans out. Mastercard announced Agent Pay in April 2025, a few months before this conversation
OpenAI: Named twice, both times as a source of evidence rather than a recommendation. He says OpenAI has been forthright that its reasoning models, monitored too closely, start trying to deceive humans and hide what they are doing, and separately that OpenAI had to roll back a version update for being too sycophantic and thereby reinforcing user bias. Both are public: the chain-of-thought monitoring finding was published in March 2025 and the GPT-4o rollback was in April 2025
Models that rewrite their own code to avoid shutdown: He says there are other models out there that do this, without naming one or a source. The best-documented case at the time was published by Palisade Research in May 2025 and concerned OpenAI’s own o3, so the strongest example is not a different vendor’s model
Grok: His example of a governance failure at scale, from the week before the recording: a model on a public social media platform encouraging millions of people to attack and harm an individual, which he says was potentially caused by adjustments made by Elon Musk himself. He adds that companies planning to put Grok into a customer service workflow should think hard about toxic content
Accenture: His example of existential rather than operational risk, and the reasoning is his rather than the company’s: clients on the margin deferring strategic consulting because a subscription chatbot gives them pretty solid advice
Boomi and Dataiku: Named as BARC clients helping companies build agentic applications with pre-made templates that business-oriented people can use. Dataiku is also the sponsor of the BARC report linked above
SAP and Oracle, Joule and GitHub Copilot: SAP and Oracle as the systems whose structured functional data, for CRM or supply chain, he says is already in pretty good shape to drive agent decisions. Joule inside SAP and GitHub Copilot as his examples of stage two, models arriving inside commercial tools
Jamie Dimon and JPMorgan Chase: His first example of a creative reporting line: the head of AI taken out of tech, so that she is head of data and AI reporting to the chief executive and the president rather than to the CIO. He does not name her
Moderna: His second example: a chief HR officer promoted to manage AI and IT, which he reads as working out at the executive level how people work with robots. Moderna combined the two functions under its chief people officer, who holds the title Chief People and Digital Technology Officer
Paul Baier at GAI Insights: Christina’s reference at 22:46, not his. She credits the phrase own your own intelligence to him, and GAI Insights publishes at this address under his byline
The Agents at Work Roundtable: Christina’s reference at 14:22, an AI Realized private executive roundtable. She reports two main use cases from it, connecting existing systems together and relieving toil, and at 12:43 she gives the model she is hearing: agents as the interface between systems such as CRM and ERP, with a governance layer below. He agrees with it
The Economics of AI Roundtable: Christina’s reference at 19:52, and the one place in the conversation where her evidence runs the other way from his. She reports a deployment where the inference cost plummeted and the return skyrocketed, which she points out was still a surprise, and that the unpredictability itself is what unnerves executives
Frequently Asked Questions
-
AI governance is the extension of data governance across three domains: the data itself, the models built on it, and the agents that act. Kevin Petrie, VP of Research at BARC, describes traditional data governance as covering data quality, data privacy and protection of intellectual property, and says it is not a discipline many companies would claim to have mastered. Organizations then have to extend into the domain of models, making sure predictions and outputs are clean and safe, and into a third domain of agents, making sure an agent’s behavior does not go rogue or have subversive intentions. Framing it as an extension rather than a new discipline is the point: the program that already exists is where it starts.
-
AI governance should be addressed comprehensively across teams rather than handed to any one of them. Kevin Petrie says chief data officers, who he thinks are playing catch-up, are partnering with chief AI officers and chief analytics officers so that data engineers and data stewards work alongside data scientists and machine learning engineers, and that once the work becomes agentic they also need AI-savvy developers who are building the workflows and applications. He says the advanced organizations are blending their teams, and that you can see it in job descriptions: a data engineer opening now carries some machine learning, a data scientist opening carries some data engineering, and he thinks that is as it should be.
-
Human in the loop means a person reviews and approves an AI system’s output before it takes effect. Kevin Petrie reports it as the most common control in practice: looking at software companies whose crown jewels are built on automation and AI, he says the number one control they use for AI governance is humans. He calls that a very good best practice, and describes it concretely as smart, vigilant employees safeguarding and approving outputs before they go into the wild, which he considers the safest way to avoid undermining trust with customers and the public.
-
An AI agent should have very little autonomy today, and Kevin Petrie says so directly: not much at this point, and he does not think they should. He points out that the strict definition of an agent is a bot making decisions and taking action independently of humans, while in most cases what is deployed is a new word for copilots or for traditional software that has been automating workflow for 30 years. Those can and should go back to humans for approval before making a purchase or closing a transaction. His reason is compounding risk: once multiple agents, each drawing on many datasets through multiple models, transact with each other autonomously, one failure could have a pretty big cascading effect.
-
AI pilots stall because three things get harder at scale: the data, the people, and the bill. Kevin Petrie says companies usually had clean data inputs that worked for one department or one data type, and as they go broader the additional data coming in is not as clean and requires real work. The second is that a sandbox project draws the most innovative and AI-comfortable workers, and encouraging the rest of the organization to use AI responsibly is a real hurdle. The third is cost: inference is getting cheaper, but token usage costs for each unit of data consumed can still arrive as a significant surprise, similar to the cloud bill that shows up unexpectedly high at the end of the month. He also notes that innovations simply take time, a point he traces to the internal combustion engine.
-
Data quality is the biggest obstacle to enterprise AI success, and a lack of AI skills is second. Kevin Petrie names the skills gap near the start of the conversation, citing a recent BARC survey, and completes the ranking in his closing answer: human skill gaps are the number two obstacle in their research and number one is data quality. On the skills side he describes organizations retraining workers, hiring, which he notes can be expensive if you want the right AI-savvy people, and showcasing use cases and best practices through centers of excellence. His closing advice follows from the ordering, which is to invest in people and help them use AI to augment their work.
-
Competitive advantage from AI comes from applying models to your own proprietary data and your own proprietary processes. That is the only stage that produces an advantage a competitor cannot simply buy. Kevin Petrie describes three stages of the generative AI boom: knowledge workers using ChatGPT, then models arriving inside commercial tools such as Joule within SAP or GitHub Copilot, and then the third stage, where adopters apply language models to what is theirs. That, he says, is where you uncover real sustainable competitive advantage rather than productivity gains that anyone can achieve. The pattern he sees doing it safely is a large public model with carefully architected retrieval-augmented generation underneath, so that the right proprietary content is retrieved and used to augment the prompt.
-
A company moving AI from pilot to production should put one executive sponsor below the chief executive, give that person a cross-functional team, and start from a business problem rather than from a technology. Kevin Petrie advises delegating to a chief data officer, chief AI officer or chief operating officer, who builds a tiger team of leaders from the data group, the AI group and the developer group. That team works with a specific line of business, region or product group to identify what hurts today, where there is too much friction or manual work, and then devises a solution at limited scale. He is explicit that the team should not rush to say generative AI is always the answer, because it might be traditional machine learning instead. He also points to reporting lines that keep AI visible: JPMorgan Chase moving its head of data and AI out of tech to report to the chief executive, and Moderna promoting its chief HR officer to manage AI and IT.
-
[00:42] Christina Ellwood: Welcome to AI Realized, the podcast for enterprise executives leading AI deployments. From tackling security, data, and operational challenges to navigating organizational transformation, AI deployment offers a unique opportunity to redesign organizations from the inside out. I'm Christina Ellwood, your host for today's episode, and we're talking today with Kevin Petrie, the VP of Research at BARC Research. Kevin, welcome to AI Realized.
[01:10] Kevin Petrie: Thanks, Christina. Great to be here.
[01:12] Christina Ellwood: It's so nice to see you, Kevin. I'd love to just start off with a little bit of background about your excitement around AI. You're a data analyst and now an AI and data analyst, so tell me about what excites you about AI.
[01:25] Kevin Petrie: Sure. So my role here at BARC, which is a global research and consulting firm, is to lead the data management practice, looking across research, consulting, and also end user events that we have in Europe and in the US. And I think that what's fascinating is that we're tracking what's happening with organizations in terms of adopting AI, and we're also continuing to refine what we do in response to AI. So I'm living it. I'm seeing more and more people in my personal life and my family who are impacted positively and negatively by AI, and there's really no better time to look at the fundamentals of data management, because that holds the key to AI success. So it's a very interesting time, and I'm pretty excited to, to be here and talk about some of the business implications of artificial intelligence, provided you have safe and effective data management.
[02:25] Christina Ellwood: Yeah. Data's obviously at the heart of it all, isn't it?
[02:28] Kevin Petrie: It really is, yeah.
[02:29] Christina Ellwood: Yeah. Before we d- before we dive into the data side of it, let's talk a little bit about the organizational challenges, because this is a little bit of an unusual technology in, in that it requires a cultural change in the organization, and it does in fact give us the opportunity to redesign how our organizations are operating. So talk a little bit about what you're seeing in the transformation area, and then we can talk about data.
[02:56] Kevin Petrie: Sure. Yeah, I think that we have, at BARC, we have the, the benefit of doing a lot of qualitative research based on consulting, based on various end user conversations with data leaders and AI leaders, and we also look at quantitatively what's happening. We did find in a recent survey that the number two obstacle s- to success with AI is skills, a lack of AI skills within the organization. And so in response to this challenge, so many organizations are looking at retraining workers, hiring, which can be expensive if you want the right AI savvy folks, and showcasing positive use cases, best practices, and so forth through centers of excellence. So there are a lot of ways in which organizations are getting serious about evangelizing the use of AI and also educating their workers about how to use it effectively.
[03:49] Christina Ellwood: Trust is also at the center of that cultural change, isn't it? Talk a little bit about what you're hearing from leaders about the issue of trust with their customers, with their employees, and even with their board in terms of mitigating risk and ensuring that the company is able to move forward in a way that is
[04:07] Kevin Petrie: Yeah. It is interesting because the latest numbers from marketing firms are that overall Fortune 500 companies have higher levels of brand loyalty with their customers than ever before. So that's pretty good times. And obviously, organizations such as Mastercard and Apple and so forth are gonna be very protective of those customer relationships and make sure that they don't overstep the risks or overstep the bounds of what these trusted relationships can involve. I do see leaders that are doing the right things with AI. They have governance programs, they have executive leadership, they have education and retraining and so forth, taking increasingly bold steps about what they're-- the ways in which they're starting to play with their customer relationships. Mastercard is a good example with their Agent Pay offering, which is starting to help some of their customers shop, peruse options, make decisions, and even process transactions. So Mastercard is starting to play with the crown jewels. It'll be very interesting to see how that pans out in terms of the level of trust and the trusted relationships they have with their customers. But I think the good news for a lot of longstanding data people is that they have been advising executives for decades that data governance needs more investment. Data quality has long-- is, is a longstanding challenge as are privacy risks and so forth. So all these aspects of data governance that really get to trust have not been getting enough attention. But I think executives at the CEO level and at the board level are starting to realize that AI can have some spectacular problems if you don't double down on the AI, on the data fundamentals. So in a sense, this can be a good time for data people to have sort of a renaissance in redoubling and extending their governance programs to address those risks and make sure they have clean inputs for AI models.
[06:06] Christina Ellwood: Aside from having their, your data be clean and prepared for the models, how does governance help you to prevent pro-- errors of the AI? Either whether they're hallucinations or they're just maybe misguided rather than absolutely wrong. How do you-- how does data governance help you with that, and what are you seeing as best practices?
[06:29] Kevin Petrie: Yeah, it's interesting. There aren't easy answers. If we look at traditional data governance, which is not a discipline that many companies would claim to have mastered at this point, there are traditional risks. I mentioned data quality, data privacy, protection of intellectual property are some of the big ones. Organizations need to extend from that domain into the da-domain of models, figuring out how to make sure that models have predictions and outputs that are clean, that are safe. They also need to move to a third domain now, which is agents, and figuring out how to make sure that a-an agent's behavior does not go rogue, does not have subversive intentions. Large popular language models out there right now, such as OpenAI. OpenAI has been very forthright and said, "Look, if we found that our reasoning models, if monitored too closely by humans, will start to try to deceive humans, and will try to hide what they're doing." There are other models out there that will rewrite their own code in order to avoid being shut down. So those are some big challenges that organizations need to figure out how to address from a governance perspective. What we do find, if we take a peek at what tech companies are doing, software companies whose crown jewels are based on automation and AI, the number one hu- tr- control they use for AI governance is humans, human in the loop. And I think that's a very good best practice. If you have smart, vigilant employees who are safeguarding and approving outputs before they go into the wild, that's the safest way to make sure that you're not undermining trust with customers and the public.
[08:13] Christina Ellwood: Do you envision that AI governance and AI model management will move into the purview of the data governance team, or do you see those as complementary but different?
[08:24] Kevin Petrie: I think that those programs need to be addressed comprehensively, and we do see organizations starting to embrace that notion. You have chief data officers that are trying to play catch up, I think, at this point. They've been managing data governance programs for some time. They need... They're partnering with chief AI officers, chief analytics officers, so that they can have data engineers, data stewards working with data scientists, working with ML engineers in order to address these risks in a more comprehensive and holistic way. And when they get into the agentic realm, they need to work with AI-savvy developers who are building the workflows and building the applications. So I do see organizations, the advanced cutting-edge ones, starting to blend their teams and bring them together. And you can start to see it a lot of the job descriptions on LinkedIn elsewhere. If you see an engineering, a data engineer job opening, it's gonna have some machine learning in there. If you see a data scientist job opening, it's gonna have some data engineering in there, and that's as it should be.
[09:26] Christina Ellwood: And are you seeing the agents being developed by the technical teams, or are they being developed by business people who maybe don't have a relationship with the data team?
[09:35] Kevin Petrie: Great question. We're at an interesting point. There are vendors that are rightly understanding that agentic AI is a highly complex, daunting endeavor for organizations. And so we have some clients here at BARC, Boomi and Dataiku, that are helping companies build out agentic applications. And they're doing it with pre-made templates, with ways for potentially business-oriented as opposed to technically oriented people to start to put these agentic workflows into place. And that's great. There's also a group of specialists that are building their own more custom agentic workflows, and that's a higher risk, but potentially higher reward in terms of harnessing and achieving competitive advantage by really stitching those agents into your own proprietary workflows and your own proprietary datasets.
[10:26] Christina Ellwood: How much autonomy are they building in at this point?
[10:29] Kevin Petrie: Not much at this point, and I don't think they should. In fact, agent... Honestly, the notion of agent right now, the strict definition is that these are autonomous bots that are making independent decisions independently of humans, and then even taking action independently of humans. I think in most cases, those agents are really a new word for copilots or even traditional software, which has been automating workflow for 30 years. They can and should be going back to humans to get approval before making a purchase, closing a transaction, and so forth. So there are very cool, inspiring visions of the future where you have armies of agents that go into these on your behalf. They transact with one another in public marketplaces and stuff. It's gonna take a while to get there because you can quickly understand how complexity and risk go through the roof when you have multiple agents, each drawing myriad datasets through multiple models, start to transact with each other in an autonomous fashion. One failure could have a pretty big cascading effect. So companies can and should step cautiously into that realm and really work with strict guardrails and carefully define use cases to start.
[11:46] Christina Ellwood: Yeah, it sounds like we actually need some new capabilities in the data layer to deal with agents. Do you think that's true?
[11:54] Kevin Petrie: I do think that's true. I think that clean inputs are a really good start. Clean outputs help from a model perspective, but you also need-- For example, you can use AI to help monitor what these agents are doing. You can start to have sentiment indicators or natural language processing to understand potentially toxic output that an agent might be putting into place. You can have rules-based checks that will block transactions or actions deemed, deemed unsafe. So there is a new layer, there's a new domain, which is governance of, of agents. I'm actually working on a report on that topic right now.
[12:36] Christina Ellwood: That sounds like it's really needed. And we have-- at AI Realized we have these private roundtable discussions with executives, and while there's a lot of, there's a lot of excitement about agents, there is also a lot of caution around deploying too quickly. In fact, in those discussions, I, I gather that the way they're thinking about agents is a little bit around the interface between existing systems. You think about the applications like their CRM, for example, or their ERP or whatever, treating those as a data repository, treating agents as a way to connect those data repositories together for action, for business automation or business processes. And then below that is the, is the governance layer. That's the model that I'm hearing. Is that what you're hearing too?
[13:24] Kevin Petrie: Yeah. I think that at the end of the day, the goal of an agent is to orchestrate a series of a sequence of tasks based on a decision they've made, based on their own reflections about prior decisions. And those tasks ideally are gonna leverage existing software. And if you look at companies that are based on SAP or Oracle and are using structured data with these functional models such as CRM or supply chain management, a lot of that data is in pretty good shape to drive decisions, and a lot of the applications are already putting out pretty rich content. And so you could start to see how you could string together reasonable tasks that, with human oversight, can connect the dots and help humans automate workflows, whether it's for a mortgage application, invoice processing, a lot of things like that. A lot of the drudgery for human teams, for finance teams starts to go away and free up those workers to do more innovative things.
[14:22] Christina Ellwood: Yeah, for sure. Yeah, we heard, really heard two main use cases in the Agents at Work Roundtable. One was, uh, connecting these, these systems together, so the interfaces between systems, and the other was relieving toil, these drudgery jobs that e-everybody hates, and that, uh, frankly are not a good use of the company's time for having high-paid people doing these very low-level roles. But let's zoom in a little bit to the executive leadership side. So organizations are going to soon be competing against AI-native companies that have a fundamentally different economic underpinning. They may also have a different economic model for how they're selling their capabilities. How do you envision enterprises responding to that?
[15:07] Kevin Petrie: Good question. Our survey findings to date are that organizations, enterprises prefer to work with existing trusted vendors. They might have a bias towards that. There's always a group of, of risk-taking companies, usually on the younger side, born in the cloud and so forth, that are gonna bet the business and be more willing to work with startups and so forth. But I think the, the first good news is that a lot of companies are willing to work with existing vendors. I think that the goal should be to make incremental changes to existing processes, and not try to rip and replace what you have. So most organizations in the Fortune 1000 have had systems in place for a very long time. They have mainframe. They got mainframe sitting on premises, and maybe they're still in-increasing their mainframe, and that's just one way in which a lot of the old stuff doesn't go away. It's really about layering new incremental intelligence onto the aspects of your existing data architecture that work, and then start to create a beachhead of something new, and expand from there in terms of building in new technology and new architectural elements.
[16:15] Christina Ellwood: Do you envision that new beachhead being a new market opportunity or a new product or both? Or what do you envision that new beachhead being?
[16:23] Kevin Petrie: Yeah, good question. So I think that what we're finding is most organizations, and this is rightfully, the primary goal with AI at a broad level is efficiency and productivity. That's great. So take friction out of existing workflows. The next big bucket, which is a little lower, but still probably half of companies are looking to improve revenue, delight customers in new ways and so forth. And that's where you can get innovative and, and start to have some fun. But I think the goal with that second bucket is where you do wanna start a beachhead, you wanna have a project, a sandbox project, have innovative people start to experiment, and you wanna have smart people, smart employees that are vetting any outputs before they start to take it into the wild with customers. If you can achieve something at limited scale that's promising, then you can start to look at how to scale it into the rest of the business. And some of those early use cases can be devising new customer interactions, assisting customer service agents in the background, giving them new information that they can, on a human basis, use to upsell or cross-sell workers, use to give them discounts and maybe escalate problems that where there's clearly a customer sentiment problem. There's definitely a lot of upside, at least from the organization perspective, on software development. You can, uh, really improve the productivity of your programmers, especially mid-level and senior level folks, by giving them gen AI tools. So those are some of the early use cases that I think can be very promising. You get that beachhead of success, and you increase from there.
[17:59] Christina Ellwood: But aren't you hearing from executives some reticence to go to production with their pilot use cases, even in some of those areas that you just described? And if so, what do you think the underlying reason for that re- reticence is?
[18:13] Kevin Petrie: Yeah, I think there's definitely, there's hesitation. Rightfully, innovations always take time. We've known that since the 1800s when we first had the internal combustion engine or other aspects, early aspects of the Industrial Revolution. Companies need to think hard about the risks to their environments. They struggle when you move from limited or lower scale production to higher scale production. It's usually because they did have initially some clean data inputs that were good for a given department or a given data type, and as they start to go more broad, they find that the data, the additional data that's coming in is not as clean and requires some real work. So that's one challenge. Another is that you can have a sandbox project that has the most innovative workers, the folks that are comfortable using AI. Uh, maybe they're younger, they have less baggage and so forth. But making the rest of your organization, encouraging them to use AI responsibly can be a real hurdle. Those are two things. I think that a third issue, which is becoming a concern and will be more of one going forward, is that AI inference costs, while they're coming down, they're coming down sharply, could still be a significant surprise cost similar to cloud usage costs, where you get that surprisingly high bill at the end of the month. So as you go broad and you have a lot of humans in your organization that are consuming AI, and you've got those token usage costs for each unit of data consumed, that can drive up costs in surprising ways, and organizations need to figure out how to govern that.
[19:48] Christina Ellwood: Yeah. Obviously, organizations don't like surprises of any type. In fact, at the Economics of AI Roundtable, we heard the opposite, which was they piloted the project and had X inference cost, which was supported by the, the impact of the use case. But when they deployed, the cost plummeted and their IOI, ROI skyrocketed. And so it was a pleasant surprise, but it was still a surprise, and that make- that's unnerving, especially when as a, as a executive, your, part of your job is to create an environment which people-- that's consistent and dependable that people can operate against. That kind of uncertainty is unnerving. So that's a, a big thing that came out of the discussion was that there is a hesitation based on the uncertainty of the risk. That not just the financial risk, not just the customer risk, not just the data risk, but the overall risk of disruption. You wanna talk about that?
[20:43] Kevin Petrie: Yeah. I think that companies' executives are being a little more forthright about what they don't know, and this is good, identifying unknown unknowns. And one of the things they're saying is, "We don't know how much we're gonna be able to automate current human work." So that's one risk. But there's also the level of disruption about how they engage with customers. If you look at Accenture, their stock is down right now because organization-- because a lot of their clients are starting to, on the margin, say, "You know what? I'm gonna ask ChatGPT for some pretty rich advice. I'm gonna have some smart people start to interact with these tools, start to go to their sources of information and so forth. And for my 20 bucks a month or whatever it is, I'm gonna get some pretty solid advice, and that's gonna defer my need to hire Accenture on some strategic consulting." So there are different ways in which companies are starting to worry about more existential risk to their business. It-- we should also recognize, and I am a, maybe a little controversial in my view, that I think there should be some regulation of these model outputs at this point. If you look at what happened with Grok last week, that was a pretty spectacular governance problem, potentially caused by Elon Musk himself because he adjusted some things. And y- you had a model that was encouraging millions of people on a public social media platform to attack and harm an individual. And, you know, that governance thing ran wild. So I think companies that were planning to put Grok into a customer service workflow should think real hard about toxic content. And Grok's just the most spectacular blowup lately. There are other cases in which OpenAI had to roll back a, a version update because they were being too sycophantic and thereby, um, reinforcing user bias when they were asking for toxic content. So there are a lot of ways in which these models need some work, and that kind of disruption to your trusted customer relationships, your relationships with the public, is creating a lot of justified caution among executives.
[22:46] Christina Ellwood: Yeah, for sure. Data, which is of course a huge asset for companies that's not on their balance sheet. Right. They also have their, their own internal intelligence, the how they do things, how they think, how they operate, their culture, things like that. So they have internal intelligence, what Paul Baier at GAI Insights likes to call own your own intelligence. You should have a model for doing that. I like that. A way in which you own your own intelligence. And that, of course, is a useful way to spark the conversation about whether your model should be trained on your data, it, whether you should be using a private model or a, uh, isolated model, or you should be using a public model. When you're talking with executives, is this a hot topic or is it a settled subject?
[23:30] Kevin Petrie: It's a good question. I think it's nothing settled yet in this space in a lot of ways, but there is recognition. So the way I view it, we're in our third stage already of the AI and the GenAI boom. First one is thought company or thought knowledge workers starting to use ChatGPT, so forth. Second stage is they're using these models within commercial tools like Joule within SAP and, and GitHub Copilot and so forth. The third phase is where adopters can actually get some competitive advantage because they're applying language models to their own proprietary data and their own proprietary processes. So that's where you could uncover real sustainable competitive advantage rather than productivity gains that anyone can achieve. And what I see as a common pattern is organizations putting in place a lot of the large models, but very carefully architecting RAG workflows, retrieval-augmented generation, where you retrieve the right content from your own proprietary data, you use it to augment the user prompt, thereby the GenAI model is more likely to generate a response that's accurate, safe, and so forth. So I think using the public model with RAG underneath is emerging as a common pattern for companies that wanna safely apply the most powerful intelligence to their specific workflows.
[24:51] Christina Ellwood: If you could wave your magic wand and, and have every enterprise operating in an ideal mode with AI, what would you s- what would we see in their infrastructure and in their operational model, and how long do you think it'll take us to get there?
[25:08] Kevin Petrie: Yeah, good question. So I think that I would encourage executives at the CEO level to have delegate to an executive sponsor just below that, chief data officer, chief AI officer, and then maybe chief operating officer, and have them build a tiger team of folks from within the data team, so data engineering leaders, the AI team, data science leaders, and the developer team. Get leaders from those groups to work with specific parts of the business. And it should really all start with the business, where you've got the, the, a certain line of business in a certain region, a certain product group and so forth, that's helping identify points of pain today. What hurts today? Where is there too much friction in the system, too much manual work? And if you can have this tiger team go in and identify ways to ease that pain, they shouldn't rush to say that GenAI is always the answer. It might not be. It might be traditional ML. It might be other things. But start with pain identified within the business, and then have this tiger team of data, AI, and developer people devise solutions to address that on a limited scale. That's what I advise as a way to get started. It does take time to go to full production, and that's okay.
[26:27] Christina Ellwood: So three years from now, will we still have someone reporting to the CEO who is leading the, the coordination of AI across the enterprise and isolated teams that are deploying within the departments? Or do you envision there being a more integrated system?
[26:42] Kevin Petrie: I think we'll still have that individual. That individual is a good person. I was struck that, that Jamie Dimon, CEO of JPMorgan Chase, took the head of AI and pulled her out of tech, out of-- So she's head of data and AI. She doesn't report to the CIO. She reports directly to him and the president. There are ways in which organizations are getting creative about making sure that AI has a, a centralized function and a leader who reports directly to the CEO. Another interesting move was from Moderna, who promoted their chief AR- HR officer to manage AI and manage IT, and that's a, I think, another equally creative look to, at the executive level, figure out how your people work with robots.
[27:31] Christina Ellwood: And an acknowledgement that's a critical part of the success of being able to leverage this technology, unlike previous technologies.
[27:39] Kevin Petrie: Exactly.
[27:39] Christina Ellwood: Um, so you work with a lot of executives and boards to understand how to chart a path forward. What is the most critical leadership skill that you find you need in advising them about AI that maybe is different than when you were advising primarily about data?
[27:58] Kevin Petrie: Oh, good question. So I was an economics minor. I actually-- In grad school, I taught econ, macroeconomics to a lot of younger folks, and I think that it's really important to keep the business perspective, especially now. And yes, AI is new and unprecedented in many ways, but there are so many lessons of history that apply here, and that I think that executives should be aware of. The-- At the end of the day, and I've said this in other forums, the goal is to increase the value that each worker delivers in a given hour. That's why we're better off than our grandparents. They're better off than their grandparents, and so forth. And if you look at prior big bang innovations, the internal combustion engine and so forth Powerful stuff that got unleashed, but it took decades to really transform incrementally how people go about their lives and their work. And so in a good year, we'll boost our productivity, the value of economic output per worker hour, by, say, three, four percent in a year. It actually went down last quarter, so we're in early innings. We have a long, lot of way to do, lot of way to go. But I think what I try to do is remind executives that, yes, the tech is fascinating. Maybe this time is different. It's probably faster more than anything else because of the power of the models and the cost of inference. Power of models going up, cost of inference going down. So it's a transition that's happening faster, but there are lessons from history about figuring out how to safely apply these powerful models to your own complex ways of doing business, and being incremental and methodical is the way to do it.
[29:35] Christina Ellwood: So anchoring them in economics and history as a way to understand how to chart a path forward. Is that fair?
[29:42] Kevin Petrie: Yes. Okay. Don't get too dizzy about the power of what Grok or these other things are handing to you. Take a pragmatic look at your existing business and figure out how to enhance it.
[29:56] Christina Ellwood: Any other thoughts you wanna leave executives with on this podcast?
[30:00] Kevin Petrie: I think this is a great point to reemphasize the need to train people. As I said, human skill gaps are identified as the number two obstacle to AI success in our research. Number one is data quality. Invest in your people. Help them use AI and use it to augment their work. And for young people out there, AI is an incredible opportunity in data science, maybe not strict computer coding, but there are a lot of ways in which organizations need to use AI to their advantage so that they can double down on the advantages that don't change, which is humans' relationships, the power of brands, and the power of, of trust.
[30:44] Christina Ellwood: Wow, very well said. If people wanna learn a little bit more about you or BARC, what should they do?
[30:50] Kevin Petrie: Thank you for asking. That's great. I'm on LinkedIn, pretty active there, promoting our research, engaging with our user community to unpack trends. We also have barc.com, B-A-R-C.com, where we publish our research, and I'm privileged to work with some very smart folks on a lot of the reports that we publish.
[31:10] Christina Ellwood: Great. Kevin Petrie, VP of Research at BARC. Thank you so much for joining me today on AI Realized.
[31:18] Kevin Petrie: Thank you very much. I enjoyed it.