Chris Butler
Chris Butler is a product leader at GitHub, based in Oakland, California. He’s building and guiding human-centered products at the intersection of AI, decision-making, and team alignment, with prior product leadership experience across major platforms including Microsoft, Facebook, KAYAK, and Waze. Chris is a writer and speaker whose work focuses on practical methods for reducing bias and improving outcomes in product development.
Your AI Agents Are Built Wrong
Episode Summary
Chris Butler of GitHub makes a bold case that most enterprises are designing AI agents the wrong way. Instead of building agents that mimic job titles, he says leaders should build agents that create specific artifacts inside governed workflows. In this episode, he explains how that shift changes automation, trust, and cross-functional execution, and why the future of agentic AI belongs to teams that focus on outputs, not impersonation.
Episode Timeline and Summary**0:00 – 1:05 | Introduction
Host Christina Elwood introduces the podcast AI Realized and guest Christopher Penn, co-founder and chief data strategist at Trust Insights. She highlights his background in data science, predictive analytics, and AI applications predating the current AI boom.
**1:05 – 3:33 | Where AI Drives Measurable Business Outcomes
Christopher outlines two fundamental ways to use AI: optimization (doing existing things faster/cheaper) and innovation (doing things never done before). He introduces using AI as a synthetic voice of the customer — virtual focus groups based on ideal customer profiles — citing a peer-reviewed study showing ~90% accuracy in replicating buyer intent.
**3:33 – 9:45 | The Three Branches of AI
Christopher dives into regression AI (uplift modeling, Granger causality for campaign attribution), classification AI (sentiment analysis, named entity recognition, topical focus), and generative AI. He draws on his experience at a PR firm building statistical models to measure campaign lift.
**9:45 – 15:25 | Trust Insights & The 5P Framework
Christina and Christopher discuss Trust Insights' consulting focus on AI implementation. He describes Katie Robbert's 5P Framework (Purpose, People, Process, Platform, Performance) as a structured approach to AI adoption, extending the classic "people, process, technology" model.
**15:25 – 22:00 | Measuring ROI & Proving AI Value
Discussion of organizational friction in AI adoption. Christopher argues that companies can't measure AI ROI if they aren't measuring outcomes already. He introduces the TRIPS framework (Time, Repetitiveness, Importance, Pain, Sufficient Data) for identifying which tasks to hand off to AI. Highlights a real-world example where his CEO achieved 100x productivity gains using Claude after adopting agentic tools.
**22:00 – 34:10 | Revenue Generation with AI
Christopher shares concrete revenue examples:
11% of closed deals came from AI tools recommending Trust Insights (GEO — Generative Engine Optimization)
Using Claude Code to convert workshop materials into a published book in under 6 hours, generating $1,000–$2,000 per title
AI-driven competitive intelligence: analyzing 1,900 competitor job listings to infer 12–18 month strategic priorities for a pharmaceutical client
Christina adds her own example of a company using outside-in AI analysis to generate hundreds of millions in pipeline for enterprise sellers.
**34:10 – 43:00 | Agentic AI & The Five Levels of AI Enablement
Christopher describes his Hermes Agent — autonomously researching speaking opportunities by browsing the web, compiling association contacts, and ranking pitching targets. He outlines the Five Levels of AI Enablement:
Done by you (ChatGPT copy-paste)
Done with you (Gems/GPTs)
Done for you (Claude Code, managing agents)
Done without you (autonomous agents)
Done in advance of you (virtual agencies)
**43:00 – 51:00 | Defining Success & Safety for AI Agents
Christina raises the security risks of autonomous agents. Christopher's advice: treat AI agents like untrusted contractors — isolate them in a DMZ (air-gapped machine, no sensitive credentials). He uses Burrows' Delta as an example of a quantifiable success metric, emphasizing that AI cannot succeed if it doesn't know what success looks like.
**51:00 – 56:00 | AI in Sales & Marketing
Christopher recommends hooking agentic AI into your CRM and sales playbook to generate next-best-action recommendations for every deal. He also describes using AI to analyze call transcripts to identify where individual sales reps fall short (e.g., "step three framing in challenger methodology") and build personalized coaching tools.
**56:00 – 1:01:30 | Key Takeaways & Resources
Christopher's top takeaway: add the sentence "Ask me questions until you have enough information to succeed at the task" to every AI prompt — this one change produces dramatically better results. He directs listeners to trustinsights.ai for his blog, podcast, and YouTube content.
**1:01:30 – 1:05:30 | Leadership Skill for the AI Era
Christina asks what leadership skill matters most right now. Christopher's answer: project management — specifically the ability to design complex projects and anticipate failure points, which translates directly to working effectively with agentic AI systems.
**1:05:30 – End | Post-Recording Conversation
Off-the-record discussion between Christina and Christopher about potential collaboration — including executive roundtables on local AI + security/data privacy, the AI Realized community structure, hybrid events, and Christopher potentially speaking or contributing content. Covers Google's TurboQuant and MTP papers and the case for local AI models for enterprise data privacy.
Full Transcript[00:00:00] Christina: Welcome to AI Realized, the podcast for enterprise executives leading AI
[00:00:04] Christina: deployments. From tackling security, data, and operational challenges to navigating organizational transformation, AI deployment offers a unique opportunity to redesign our organizations from the inside out. I'm Christina Elwood, your host for today's episode, and today we are joined by Christopher Penn, co-founder and chief data strategist at Trust Insights.
[00:00:28] Christina: Christopher's been working in data and AI science and data science long before the current generation of AI, building practical applications in attribution modeling, predictive analytics, and customer journey intelligence. He's also a globally recognized keynote speaker and co-host of the long-running Marketing Over Coffee podcast.
[00:00:48] Christina: What makes his perspective especially valuable is his focus on turning AI into measurable business outcomes, helping organizations connect data to real decisions, real performance, and real [00:01:00] ROI. Christopher, welcome to AI Realized.
[00:01:03] Christopher: Thank you for having me.
[00:01:06] Christina: We're very happy to have you, and this whole area of AI in the vertical functional groups within our organizations, like sales and marketing, is the sort of topic of the day.
[00:01:16] Christina: So at a very high level, where do you see AI actually working to drive measurable business outcomes in sales and marketing?
[00:01:25] Christopher: The ways that it's used best these days the-- the-- I should s- take a step back. There's two fundamental ways to use AI, and w- this incorporates all three branches of it regression AI, classification, and generative AI.
[00:01:39] Christopher: And those three branch-- Those two ways are optimization and innovation. Optimization is do what you've always done, but do it bigger, better, faster, cheaper, and a lot of folks these days are emphasizing faster and cheaper. And then innovation, which is do what you've never done before because you've never had the capabilities.
[00:01:55] Christopher: And this i- I'll give a very trivial example. Imagine taking something [00:02:00] like an RFP response and turning it into a country music song. Could you do that with AI? Yes. Should you? Eh, it's debatable. But it is something that you traditionally would never have done before because you just didn't have the capabilities of doing so.
[00:02:15] Christopher: When we talk about AI in the enterprise, and in particular i- in useful ways of using it, one of the most valuable ways is to have it be a synthetic voice of the customer. Back in summer of 2025 there was an academic paper, and I'm struggling to remember the name of the paper but it showed in peer-reviewed research that generative AI models accurately repr- replicate purchase intent, buyer intent with about 90% accuracy.
[00:02:44] Christopher: So- If you were to, for example, build, take your existing ideal customer profiles and put them into the generative AI tool of your choice, or even better, an agentic AI tool of your choice, and have them function as a virtual focus group, [00:03:00] you basically have a customer on tap 24/7 to ask questions of. "Hey, I've got two subject lines.
[00:03:07] Christopher: I've got this, I've got that. I've got a new product idea. I w- I need to raise my prices." Tell me, machine masquerading as my customer, what my customers' probable reactions are going to be to this, and how can I message it in a way that will minimize risk or maximize the revenue opportunity?
[00:03:27] Christina: That's a fabulous example of using generative AI.
[00:03:31] Christina: Do you have examples for the other two types?
[00:03:34] Christopher: So regression AI is the classic this is, goes back to the Eisenhower administration in the 1950s. And one of the most powerful tools here is what's called uplift modeling, and there are statistical methods like Granger causality, et cetera. One of the pr- challenges that marketers face is imagine you have all of this data.
[00:03:54] Christopher: You have your web traffic, and you have your CRM data, and you have your social media data, and this, that, and the other thing. [00:04:00] And you say, "Hey, we have this campaign in flight," or whatever, something that you did. And you say, "I wanna know what the impact of that thing was." But at the same time, you've got Google Ads running.
[00:04:11] Christopher: You've got YouTube ads running. You've got emails going out the door. You've got billboards all around the city. It's very difficult because it's a lot of noise. Good multivariate regression techniques like uplift modeling, propensity score modeling, Granger causality, et cetera, can be used by you in partnership with your favorite generative AI tool to say, "Given all of this data, can we build a statistically valid model that shows the uplift of this thing?"
[00:04:39] Christopher: This is something we get from bioinformatics, right? When you say I have all this information about this patient, and we administered a treatment. What is the effect of the treatment on the patient, especially given how much other noise there is?" I first started working with this type of modeling back in 2013 when I was working at a PR firm, because public relations is notoriously [00:05:00] difficult to measure because there's not a direct click stream between positive press coverage and something else.
[00:05:05] Christopher: And so I was brought into the firm I was working at the time to try and help them build models like this that would help them understand what the true lift of a PR campaign was. So that's regression. And the second one, classification, is one of my favorites because it's all about organizing your data.
[00:05:25] Christopher: If you've got a bunch of data that has no structure or things, you can bring structure to that data and then bring insights out of it. For example, things like basic sentiment analysis, f- topical focus. Given a piece of informati- a pool of information, what in this information is relevant and what is not?
[00:05:42] Christopher: So imagine you have vast quantities of social media data, and you wanna know which of this stuff is actually about us, particularly if you have a company like our company's named Trust Insights. That's a fairly common phrase, so how do you use these tools to [00:06:00] identify things like named entities, then be able to measure h- your presence within a pool of data?
[00:06:06] Christina: Is that the work of Trust Insights as an organization?
[00:06:12] Christopher: It is partially. So we are a consulting firm. We focus on, these days, AI implementation and enablement, helping organizations. My CEO and co-founder, Katie Robbert, is our organizational change and change management expert. She's the one who comes in t- to an enterprise and says, "Yeah, you've got the shiny new toys, but toys themselves are not enough.
[00:06:33] Christopher: You need to enable the people and your processes to match the platforms." And then she came up with this framework called the 5P Framework by Trust Insights, which is purpose, people, process, platform, performance. Why are you even doing the thing, and then how do you measure success as the bookend to your traditional people process technology, which was originally from what, H.J.
[00:06:53] Christopher: Levitt in 1964 was, the diamond framework back then became the people process technology trope, [00:07:00] and then Katie extended that to say having purpose and performance as the bookends for it.
[00:07:05] Christina: Yeah obviously the organizational friction is the largest impediment to the adoption of AI.
[00:07:12] Christina: From the time that AI Realized was first formed in '20- in October of 2024 we heard this from the stage, that 100% of the companies were struggling with that friction. It remains the most cited source of friction today. It sounds like that's part of the work that you've been doing. So what are the underlying measurement and decision systems?
[00:07:34] Christina: What do those look like inside of those organizations, and how do they relate to the friction of adoption?
[00:07:41] Christopher: One of the biggest interesting pieces of feedback we get from people who are resistant is the immediate demand, "Oh show me the ROI of AI." Which we always say So how are you measuring the ROI of this task now?
[00:07:57] Christopher: And the answer is we're not. Then you can't measure [00:08:00] the ROI of AI 'cause you're not measuring it now. You have no basis for comparison. And so that's one of the things that we immediately tend to flag. But you can me- everything has to come back to three basic imperatives, right? You're either saving time, saving money, or making money.
[00:08:15] Christopher: Everything you do has to, in some way, have line of sight to one of those three objectives, ideally more than one, in order for you to demonstrate impact. When we work with organizations, the first imperative that they go after is always saving time. How can we free up time? How can we free up resources and stuff like that?
[00:08:33] Christopher: And that's a great first place to start because there are a lot of process inefficiencies, and there are a lot of capabilities that today's agentic AI systems can do that can take on a task completely. I'll give you an example. This, again my CEO is a non-technical person. I'm like the, I'm like the button-pushing propeller head in the company.
[00:08:55] Christopher: But we've been using Claude Cowork a lot, which is a non-technical agentic [00:09:00] system. And we measured her output in terms of important deliverables, strategic blueprints, this, that, and the other thing for ourselves and for other clients in the period, the six-month period prior to work, to Claude Cowork coming out, and then in the three-month period after when she started using it.
[00:09:17] Christopher: And her productivity level as a senior executive, as a leader, is literally 100x what it used to be. That is how much more productive she is at doing things like budgeting, forecasting, strategy strategic council, and all the way down to website design, like rebuilding slide decks, all of the stuff that there's so much manual drudgery in, like making a slide deck, that if you can use your brain to get the valuable stuff out of your head and then hand off the typing essentially to a tool, you can dramatically increase the results you get.
[00:09:52] Christopher: And so that's the first enablement that we tend to go after with people is to say, "Let's get your to-do list [00:10:00] under control personally or organizationally." If your department has a punch list- We have a framework we call TRIPS, time, repetitiveness, importance, pain, and sufficient data, and we score tasks by that.
[00:10:13] Christopher: When we do a department or company-wide audit with somebody, we'll say, "Okay, let's score all of your deliverables and tasks by this matrix and identify the top 10 tasks that are just a waste," right? They, the, it consumes a lot of time, highly repetitive, tons of examples of success, and nobody likes doing it.
[00:10:31] Christopher: Those are the tasks that immediately you should be handing off to AI because no one's, no one gets upset about it, right? There's a lot of conversation about, the future of work. Nobody is we've, and ze- zero clients have ever said, zero people ever said, "You know what? I would like to keep doing my expense reports by hand."
[00:10:49] Christina: This is the this is the first piece of advice I think that people took away when this conversation was started, which is start with something people hate. And that's a wonderful example [00:11:00] of that. And I myself have found Claude Cowork to be an extremely valuable boost to my productivity as well, so I resonate with that.
[00:11:08] Christina: But I think the revenue one is the one that needs some attention. And I'm very curious what you're seeing AI actually do to improve attribution, for example, or conversion, or qualification, any of the elements of moving a deal through the funnel. Can you illuminate can you illustrate some examples there?
[00:11:29] Christopher: Sure. So a couple of really obvious ones. Number one what is now called, among other things, GEO. There's so many different variants to the... But AI basically making recommendations. 11% of our business in the last six months has come from AI recommending us because we have been s- working, and we've understood the space and we've been working in the space for a while.
[00:11:47] Christopher: We've been doing what we know works based on the architecture of the technology and how the technology functions. We've been planning ahead for years for this, and it works. That's 11% of our closed won [00:12:00] deals come from generative AI tools recommending us. So we- we'd love to be case study zero for a lot of these things.
[00:12:07] Christopher: In terms of revenue generation, one of the things these tools enable is y- is better product market fit and new products. So As an example, I t- I typically deliver, I'll go 20, 30 workshops in the in the spring and the fall to all kinds of different industries. Now, with tools like Claude Code, I can take my workshop that I did, take all the materials and outputs from it, put it through Claude Code with a recipe that I built, plus a bunch of just some custom Python code, and in about two and a half hours, it spits out a polished book that then goes immediately into our company bookstore, and it goes up for sale, and typically generates, $1,000 to $2,000 of extra revenue.
[00:12:53] Christopher: We just published, Generative AI for Destination Marketers. That product did not exist, and instead of taking six to nine months [00:13:00] to bring it to market, I can have it to market in under six hours and immediately begin monetizing it. At a broader level, rationalizing your product portfolio or looking at product market fit, you...
[00:13:12] Christopher: those customer profiles we were talking about earlier. If you have a product and it's not selling, you take your ideal customer profile, and you literally say, "Why is," "Why are you not buying this?" Especially if you have other signal data that you can provide, like your inbox, your call center data, et cetera, to better understand this is why the, our customers are not buying this thing.
[00:13:33] Christopher: You have macroeconomic data. Why are people booking fewer hotels? 'Cause everything costs more money. Gas is $5 a gallon. People aren't traveling. That's why. And so revenue generation can be improved product market fit. It can be net new products that you didn't have for sale before, especially in ways that you've never done before information products and things like that.
[00:13:55] Christopher: And of course, referrals from these new tools that are, [00:14:00] if you have done a good job of being present and having stuff out there for the machines to learn from, become a word-of-mouth tool, but the word of mouth is being spread by a machine instead of human.
[00:14:12] Christina: Yeah, so I should say for our listeners that we have done a number of things on GEO recently in the AI Realized community.
[00:14:19] Christina: We have a webinar that you can find on on the YouTube channel. We've got three articles on GEO, and we just did a roundtable and published a readout from the roundtable discussion. So this area of GEO for marketers and for founders and for CEOs is really important, and it plays back into something you said earlier, which was the importance of PR.
[00:14:41] Christina: Can you walk us through a specific example where AI changed a business outcome in sales or marketing, not just the analysis, but the decision and the result?
[00:14:51] Christopher: In terms of an outcome this is gonna be a tricky one to thread the needle very carefully. We had a pharmaceutical client and they wanted [00:15:00] to know-- They're a very large company.
[00:15:01] Christopher: They wanted to know what their largest competitor was doing. So what we did was- We took 1,900 of the competitor's open job listings, downloaded them, digested them with a language model, and then essentially did a large-scale inference t- model to cut, to say "Where... What is this company's 12 to 18-month strategic priority?"
[00:15:24] Christopher: Because you don't hire people for things that are unimportant, right? You don't add headcount for stuff that's- ... not relevant. You add headcount for stuff that is strategically important. And what- ... we found was that there were three different product lines, but one in particular that this competitor was hiring like crazy for.
[00:15:42] Christopher: We handed that to our client. Our client said, "We don't have a good business answer to this competitive challenge," and they spun one up and essentially. Now, we don't know we don't know what happened to their version- ... of that product line because we were not privy to those conversations.
[00:15:59] Christopher: But [00:16:00] the people who were our stakeholders were like, "This is probably the most important thing that we've ever done with AI because we can now anticipate what a competitor's doing i- in their 12 to 18-month strategic horizon based on the hiring data."
[00:16:15] Christina: Yeah, absolutely. And in, in fact, I have interviewed a founder of a company that does outside-in analysis using AI, and their wedge business case is with B2B sellers, and they are able to bring them opportunities that were identified using outside analysis of the target accounts for their KPIs, their gaps in their performance, and deliver those opportunities with a built-in business case for why the company should be buying.
[00:16:42] Christina: Now, that's especially powerful when you're looking at complex product portfolios. So if you've got, 50, 100, 200 products that you're trying to sell into enterprises that are large and complex, you simply cannot do that mapping with human beings, and AI is ideally suited to doing that. And they're c- they're creating hundreds of [00:17:00] millions of dollars worth of new pipeline, which are essentially SQLs on a silver platter for an enterprise seller.
[00:17:06] Christina: Now, that's just not something that could have been done without having an A- not just the AI, but the system sitting on top of it. So the business logic layer that sits on top of it that does all of the triangulation and root cause analysis. Now, that's something I'm hearing you say, too, when you say especially with agents.
[00:17:24] Christina: I think that's what you're referring to, is that the agent systems are the, where the logic is living in addition to the automation that allows for some of these decisions to be taken. Can you double-click on that?
[00:17:36] Christopher: So I've got one running right now, an assistant called Hermes Agent, that is going out and it is identifying every trade association that's having an event in, within two hours of my house within the next 12 to 18 months at all the different venues and stuff like that.
[00:17:51] Christopher: So it's going out, it's browsing the web, it's grabbing all the data, it's assembling the data, it's identifying the contacts for those people, either by email address or LinkedIn profile, and then it [00:18:00] provides me a rank ordered list of here's the associations that I should pitch as a keynote speaker to speak at their events.
[00:18:06] Christopher: I am doing none of this. I gave it a 13-page project plan to start, and then I'm hands-off after that. It does everything else downstream of that. That, and that is what today's agentic systems are capable of. We have what we call the five levels of AI enablement, and it maps to product market fit.
[00:18:26] Christopher: Done by you, done with you, done for you, done ahe- without you, done in advance of you. So done by you is level one. Chat, ChatGPT. You're the copy-paste monkey. You're typing all the time. You're copy-pasting, and it... You get some gains, but not much, because you the human are essentially the machine operator.
[00:18:44] Christopher: Level two are things like Gems and GPTs, standard operating procedures you've baked into little mini apps in these tools that increase efficiency, but you the human are still copy-pasting an awful lot. Level three is where you go from individual contributor to manager of an [00:19:00] agent, of a, almost like a virtual employee.
[00:19:02] Christopher: These are systems like Claude Code, Claude Cowork, et cetera, where you are now managing, you are delegating tasks. Level four is systems like Hermes, OpenClaude, Dear Flow, you name it, that all the different autonomous agents. You give them a project plan, maybe even a job description, and they go off and do the job.
[00:19:23] Christopher: And then level five is not just an autonomous person but an aut- autonomous agency or company where, a system like Paperclip, for example, or any kind of control plane similar, you are delegating the project to this ag- virtual agency that just does all those tasks that you need to bring something to, to market.
[00:19:46] Christopher: I've got another tool running right now. I am g- I don't know how it's gonna turn out. It could be, it could go well, it could go horribly wrong. But it's tied into an investment app, and I've given it the charter "I'm gonna give you $25. [00:20:00] You have to f- you have to, using these APIs and these platforms, you have to turn this into $100.
[00:20:05] Christopher: You can issue trades, you can choose which assets to buy, et cetera, but you figure it out. And, grab all the data you need to do back testing, select which algorithms, select which statistical, causal algorithms make the most sense, and then run a live test and see if you can turn 25 bucks into a thou- into 100."
[00:20:24] Christina: Okay, that's that's ambitious, , on your part- ... to maybe doing that. I applaud your creativity there.
[00:20:30] Christopher: Where should- if it works, you'll never hear from me again because- Exactly ... I will be a billionaire. You'll be retiring.
[00:20:34] Christina: Amen. Absolutely. I appreciate that, that spirit. So where should people think about, or listeners think about doubling down right now to get more value?
[00:20:44] Christina: What's actually compounding and working for people right now?
[00:20:49] Christopher: So go back to the five P framework by Trust Insights, which is purpose. Why are you doing the thing? Forget AI. What are the [00:21:00] five biggest things that are, have got your hair on fire right now? Take those five things and decompose them into their individual tasks.
[00:21:09] Christopher: What, and then what of these tasks are follow the trips framework that you can clearly identify this is a task that a machine should be doing. There, no human should be doing this particular task. It doesn't have to be the whole job, but if there are... That problem is composed of tasks.
[00:21:25] Christopher: Decompose that and then map the people, the process, and then the platform to those tasks, and then c- for each task, identify a measurable, quantifiable, objective outcome that a machine can tune against and come up with answers that, that will deliver value. That's how you get value out of AI is by take, by putting it in its place in the larger strategic structure, and then saying, "How can we decompose this problem to the individual pieces, and then which of those pieces can we hand off to a machine, and where are we still gonna have humans as blockers?"
[00:21:59] Christopher: A [00:22:00] lot of people and this is something my CEO, Katie Robbert, talks about all the time, is a lot of people start with the technology. "Oh, we've got to use AI for this. We've got to use AI for this." No, you've got to figure out what is the problem and how do you measure the solution to the problem in clear, objective, quantifiable ways.
[00:22:18] Christopher: This, it's the old management trope. If you can't measure it, you can't manage it. That has never been more true than with AI. For example, very st- simple example, let's say you're doing some creative writing or just doing any kind of writing, and you have a writing style guide. There is a measure of text s- author, authorship similarity called Burrows' Delta that a machine can analyze a piece of text and understand.
[00:22:42] Christopher: You can run this mathematical comparison. If you say to a machine, "I want you to write like me," you will use Burrows' Delta with, and the value cannot exceed 1.25. Now, a language model like a cloud code can iterate and iterate until it hits that success number, until, because it [00:23:00] knows what success looks like.
[00:23:02] Christopher: So many people don't get value out of AI because they do not know what success looks like. They cannot quantify it, and as a result, the machine can't quantify it, and the machine has no idea whether it has succeeded or not
[00:23:15] Christina: How do you recommend people factor in the safety element? 'Cause you're right, they can't do what you...
[00:23:20] Christina: If you can't tell what the outcome is, it can't achieve the goal. How do you do that and make it safe at the same time?
[00:23:27] Christopher: What do you mean by safety? Define safety specifically.
[00:23:30] Christina: So agents can go rogue, for example. You can have data exfiltrated, you can have them be a portal in for a hacker.
[00:23:39] Christina: There's different ways in which the agentic system falls outside of our security systems today. And there's been quite a bit written about that both in our Substack newsletter as well as in in, in the w- in the wild by various security experts. Given that you are using agents in autonomous fashion, that is the higher risk profile [00:24:00] for an agent is when they are autonomous rather than having a human in the loop, how are you factoring in safety and what is your advice to others as they consider building autonomous agents?
[00:24:11] Christopher: The, go back to the way you've always done it. How do you handle an untrusted contractor? You don't sit them in the middle of the office on the executive floor with a, root level passwords to everything, right? That would be a disaster. You say, "No, consultant, you're gonna sit here in this little locked room with this laptop that's been air gapped from our network and you do your work in this little concrete room."
[00:24:30] Christopher: It's like what all the three letter agencies do. They have a little concrete room in the basement of Langley where the those untrusted entities get to work. So in my agentic setup, for example, I have a little B link box. It's it's its own little computer. It's, it is firewalled outside my, my, my main setup, so it can't even cross into my main...
[00:24:52] Christopher: It's basically, it's on its own little island, and I can, terminal into it and things like that. But on that box, there's no sensitive data. There's no way for [00:25:00] it to be able to use credentials it doesn't have access to. There is no way... So even if it runs across hostile code and, it gets prompt overridden by some hostile code, it's in a d- it's in a DMZ.
[00:25:12] Christopher: It's the same thing we've been doing for, what? 30 years with Wi-Fi. We have DMZs for, in our Wi-Fi networks to keep the main network safe from guests. This is not new, it's just people forget how we've done security and safety in other places.
[00:25:28] Christina: I think to be fair, many executives have never had to stand up their own system.
[00:25:33] Christina: You have business people who are creating these agents and so forth, and it's the first time they've ever built a product, if you will. And they haven't had to think that through. Let's do a little thought experiment. So for the things that you run on your laptop, which are, is connected to the internet, is connected to systems of record, and is your your main terminal, when do you decide, "Oh, this needs to go over here on my DMZ isolated, air-gapped, s- [00:26:00] super-duper secure computer over here"?
[00:26:02] Christina: What's an example of an app you would do in one versus the other?
[00:26:06] Christopher: Any time that I'm using any system where there's an option that built in... So Claude, for example, calls it dangerously skip permissions, right? And if you want a task to run autonomously, you use dangerously skip permissions. That is the mental red flag to say, this does not belong on the production system.
[00:26:24] Christopher: The moment you have to say, "I want this to be less hands-on. I wanna run this in auto mode. I wanna run it in YOLO mode," that's when you know it goes to the other machine. When you, the human, are constantly hitting, "Do you approve this? Do you approve this?" And you actually read the, read what it's asking you to do that's when you can have a greater degree of safety to go, to know, okay, this is asking me permissions.
[00:26:48] Christopher: Or better yet, if you know what you're looking at in a tool, you can say, "Okay, you're asking me for permission to this command. No, you may not do that." "Hey, can I remove all these files?" "No, [00:27:00] you may not." Another thing that really helps is having strict guardrails and permissions within the apps themselves.
[00:27:06] Christopher: Now, yes, they can be hijacked in the worst case scenarios, but most bad things happen because people provide insufficient guardrails. A simple example, in a Claude code, you have list of permissions that you can give it out of the box. One of them should be you are forbidden to delete files.
[00:27:27] Christopher: You b- you know, you may not use this command ever. If you want, if you don't think of something relevant, move it to the archived folder so that I can move it back if you're wrong. So it's it's defining what those rules are, your first principles. Every project should have first principles.
[00:27:41] Christopher: Some are gonna be universal, like you should use existing software that's pr- known and proven to work rather than re- you know, reinvent the wheel every single time you approach a task. Other things are those, you say like you can only operate in this folder and depending on the system you're on, you, like I said, you may need to [00:28:00] move it to a separate box so that even if it does decide, to override its own permissions, it can't leave its little containment box.
[00:28:09] Christopher: All of this is stuff that should be at part of your AI council and your AI governance, that these rules and things should be decided up front
[00:28:20] Christina: so let's talk about applying this in the world of sales and marketing since that's the area that you are an expert in. Is there any specific advice for the sales and marketing executive and team that is adopting AI either agentically for task automation or autonomous automation?
[00:28:41] Christina: Is there anything specific that you would recommend to them? My number one r- recommendation is work with your tech team, obviously, but hook into your CRM, use, and using the data that you have, and your sales playbook. If you don't have a sales playbook written down, you should. Once you have that, [00:29:00] you could have, and you should have a either an agentic system or a who, who cares what the fancy label is, that looks at every deal in your CRM against your sales playbook and says, "What is the next best action that we need to take on this deal to get it to move forward?"
[00:29:14] Christopher: 'Cause your sales playbook should have your main sales tactics, your methodology. Do you use solution selling? Do you use insight selling? Do you use challenger? What is ... whatever the methodology is that you use at your shop should be embedded in there. You should have battle cards in there.
[00:29:29] Christopher: You should have objection handling in there so that you can extract all the data from your CRM, maybe from your call center system, particularly if you use something like Gong, put it into a language model programmatically through a tool like a Claude code or whatever, and build and then augment back into your CRM.
[00:29:48] Christopher: This is the next best action for this deal. If you want this deal to move forward based on our sales playbook, this is what you should do. And this, and you can do this i- we built this for a couple of clients now. Not [00:30:00] only does that help advance sales more quickly, but then you can take things like call transcripts, map them back against the sales playbook and say, "These are our sales reps who are out of compliance with the sales playbook."
[00:30:11] Christopher: We use challenger methodology, and this, the rep here is just not doing a good enough job of, step three framing. On all their calls, they fall down at step three framing. They need coaching most there. Because the tools can identify in a call transcript where a person is skilled and where a person falls, falls short, and then you can coach that person to say "Okay, you need to work on this," and then build them a a- an interactive tool, again, with the language models of your choice and ideal customer profile to help them train, to help them have that simulated conversation where the simulated prospect comes back with an objection, "Oh, I don't wanna sign a 12-month contract."
[00:30:50] Christopher: Okay how, what is, what have you learned from your training that will help you overcome that objection?
[00:30:56] Christina: Gotcha. So if our listeners could take one thing [00:31:00] away from our conversation today, what would you want them to take away?
[00:31:04] Christopher: At every level in the use of generative AI, if you ask it
[00:31:10] Christopher: If you give it one sentence, everything will get better immediately. And that one magic sentence is, "Ask me questions until you have enough information to succeed at the task." Ask me questions until you have enough information to succeed at the task. Whether it is Claude Code, whether it's with Agent, whether it's ChatGPT Basic Edition, everyone forgets that they, that the, these AI tools do not generally have enough information, and so they infer and guess a lot.
[00:31:42] Christopher: And when you're using these tools, whether you're using it for management, for sales, for marketing, for whatever, you are forgetting that it does not know. Even if you're providing it with, some data or you've got it connected to your internal systems, it still doesn't know everything that you think it knows.
[00:31:58] Christopher: And if you add that one [00:32:00] sentence, especially when you're starting out on a project of any kind, you will immediately get 5X better results because the model will stop and say I have some questions like who's this for?" Or, "Have you tried to solve this problem already?" And at every level, every grade of experience, I use this all the time, and I consider myself to be a fairly f- proficient user of these tools.
[00:32:24] Christopher: I use this all the time, and I am constantly surprised by how the machine challenges me to think deeper and to provide information. And I ... More often than not, I'm going, "Ugh, why didn't I think to include that?" And now I do. I
[00:32:38] Christina: have that experience myself. I appreciate that. So what resources would you recommend for listeners who wanna learn more about your work and about adopting AI for their sales and marketing organizations?
[00:32:51] Christopher: The best place to start is trustinsights.ai. And then you can get to our blog, our podcast, our YouTube channel, our live stream, our Instagram, all these [00:33:00] places. There's so many, but the starting point is trustinsights.ai, and you can find everything from there.
[00:33:07] Christina: All right, great. In the AI revolution, you've been around the track a few times.
[00:33:12] Christina: You've been a leader in a number of different contexts. In this AI era, what's the leadership skill that you find most valuable right now?
[00:33:23] Christopher: Project management. Project management, being a, being good at des- d- designing complex projects because if you're really good at that, like my CEO Katie is, you can anticipate all the things that are likely to go wrong.
[00:33:39] Christopher: And when you're working particularly with agentic systems, if you can anticipate what's gonna go wrong and get ahead of it, you will get 10X better results than the person who's just next to you winging it and hoping that they can chat their way through it. As models get smarter, this is something Ethan Mollick of Wharton says all the time, which I love.
[00:33:57] Christopher: As AI gets smarter, it makes [00:34:00] smarter mistakes that are harder to detect. So you have to stop being its chat buddy and start thinking like a project manager and say, "This is the task." Using the five P framework by Trust Insights, what is the purpose? What does success look like? Who are the people involved?
[00:34:16] Christopher: How do we do this? What tools and technologies should we use? If you have those skills, you are well-positioned to, to get heads and shoulders better results out of AI than somebody who doesn't.
[00:34:31] Christina: Christopher Penn, co-founder and chief data scientist at Trust Insights, thank you so much for sharing your experience and your advice today with the AI Realized audience.
[00:34:43] Christina: We really appreciate you being with us.
[00:34:45] Christopher: Thank you for having me.
[00:34:50] Christina: Okay. We're out. We're- ... we're still recording, just so you know. The recording is still on, but that helps my editor know when we think we're done. Exactly. He [00:35:00] looks for "out." Nice. Searches for my saying, "We're out." That was really fun. I really enjoyed your... you're very energetic and passionate and clear and give really solid advice, and I really appreciate that.
[00:35:15] Christina: I can see why you're such a popular keynote speaker. I'm very curious. I-- So I would love your little your agent that's finding your speaking opportunities, because I have the reverse problem in looking for the right speakers. So let me tell you a little bit about our organization, because I think there's probably work that we could be doing together.
[00:35:35] Christina: So AI Realized was founded back in 2024 because we observed that the adoption was top-down. So the early adopters were executives. Now, every early adopter cohort needs a network to be able to get through that phase where everybody's groping to figure out how to do this new thing. So we our [00:36:00] intention was to bring together people who were in enterprises who were leading these AI adoption efforts to share their experiences with each other and form their own network, okay?
[00:36:09] Christina: So that was our intention. That happened. We had a couple hundred people in the room, and everyone on the stage joined the audience and spent the entire day, because they were really there to meet other people who were doing the work and to learn from each other. They wanted more, and that's what led to the Substack newsletter, the podcast, the roundtables, the executive roundtables.
[00:36:32] Christina: And of course we do the annual summit. So the, We have two more roundtables this year. I'm interested in what you think would be a great topic for an executive roundtable. We have the format is Chatham House Rules. We have groups of six to eight executives who, with a facilitator, discuss the topic, do a readout to each other, and then the AI release team [00:37:00] writes up the readout for the community and it's all done anonymously.
[00:37:04] Christina: So that's the format, and we've done agents at work, we've done the economics of AI, the deployment friction. We've done GEO. So those are the ones we've already done. I'm interested in what you would suggest for an executive roundtable that would be really meaty and beefy for people that are trying to do this adoption internally and is not what they're finding in other venues.
[00:37:28] Christopher: That's, that is a tricky question because it's not that they're not finding it, but they're not doing any of the necessary work to make the stuff work, like in agentic AI. There's no shortage of information, but A, part of it's figuring out what's credible or not, and B, people aren't doing the work.
[00:37:47] Christopher: They're just saying, okay, it's like shelfware. I see this a lot with our clients where, they're all, "We're gonna be an AI-forward organization," and they end up with a lot of binders and a lot of PowerPoints, but [00:38:00] nobody actually does anything or does anything in a systematic way.
[00:38:04] Christopher: In terms of topics, I-- One of the things that is The, I'll give you two different ideas. One's tactical and one's more high level. The tactical one is understanding the difference between models and harnesses, because that is a big deal, and especially with if you are using AI on premise, like in with local inference, stuff like that, you need to understand that because what you can do today with local AI is insane, and it's about to get 18 times better, quite literally, because of two papers that were released in the mass- last month that dramatically changed the economics of local AI to make it much, much more capable.
[00:38:46] Christopher: So that's the topical one. And then, Can you tell
[00:38:48] Christina: me what those two papers are too?
[00:38:50] Christopher: They're both by Google. One's called TurboQuant and the other is called MTP Multi-Token Prediction. And you can find them both on Google's developer blog. They change [00:39:00] the calculus of h- of the speed and the s- intelligence at which local models can run to the point where a model like Gemma 4 or Qwen 3.6 can do 90% of what a foundation model does at the cost of electricity, right?
[00:39:14] Christopher: I have a friend who I would not necessarily classify as being, a super advanced technologist who bought himself a DGX Spark and is running a local model and getting amazing results out of it. And he's "Yeah, I don't need to pay for AI outside of this now because this does everything I want it to do, and it just sits on my desk."
[00:39:32] Christopher: All of our enterprise e-executives that are using AI in either their product or in their operations are running local models in addition to using cloud-based models. And I asked you about security. One of the companies that is doing some really interesting work here is offering the encryption of AI tokens on inference outputs.
[00:39:55] Christina: Like for local models they can it can encrypt the tok- output [00:40:00] tokens. It can also be used in a chip, which is not of any interest to people like you and me and it can be used at the edge. If you're doing something like a self-driving vehicle, a drone, an IoT device, something like that, it can be used there.
[00:40:11] Christina: So I predict and, really in my opinion from way back when this was first heating up, that we need to be local because we are leaking all of our corporate intelligence out into the world. And so to my mind, the local model is what makes the most sense for all companies to be doing. And t- the...
[00:40:30] Christina: You still need to secure that, and securing the token so they're never in plain text is the piece of the puzzle that's missing, particularly for agents, because it can secure them in memory as well as secure them in all of those other places where it shows up, right? So I think that those two are very interesting, and we could put them together, right?
[00:40:52] Christina: We could talk about local AI and security and governance, and we could talk about the protection of int- of the [00:41:00] corporate intellectual capital, right? So that could be... Now this one of our executives who's doing a lot of work in this area is also talking about cost reduction and how he has lowered the cost of their implementation by, it's 70%.
[00:41:14] Christina: And he's done it by, a combination of things. Not one thing but, running things locally on different kinds of servers and tuning the servers and all sorts of things that he's done. So it might be good to include him in that kind of conversation. So I like that. Local AI is a good one.
[00:41:29] Christina: What was your second topic?
[00:41:31] Christopher: Oh more on the s- sort of the data privacy part, because again, this is an area where a lot of people do not understand The lack of data privacy that you have, even in enterprise agreements, because of one little tiny clause in every contract which says every provider retains data for a certain period of time, period, because so that they can answer lawful requests for information by [00:42:00] the government.
[00:42:01] Christopher: Without getting too political, I don't trust the government, and I don't like the, the government- You shouldn't ... can decide what lawful means at any given time. Yeah. Lawful may mean- You shouldn't ... as, as people who look like me do. 1941, lawful meant I get to go to a concentration camp in Nevada.
[00:42:14] Christopher: ... Local AI is the only private AI. If you have data that is sensitive and confidential, it is the only one. And so helping people even understand that there is commercially safe, yes, but it is not guaranteed private, and helping people understand the distinctions and the different levels, because the number one area people have a problem with at the enterprise is some donkey, on 14 levels down the org chart is using free ChatGPT all day to copy and paste their entire job, and guess what?
[00:42:44] Christopher: All of your data has exfiltrated to a third party.
[00:42:47] Christina: No, it's exactly right. It's exactly right. Okay, I love that topic. That's awesome. Now, if we do the, so the way... So we're a volunteer-based organization. Let me s- just say that again because I don't know if I said that. Would [00:43:00] love to have you submit articles for the Substack newsletter or to participate in the executive roundtables.
[00:43:08] Christina: You could be a participant. You could be a facilitator, whatever. Whatever makes sense for you to try to expand the visibility and awareness of the work that you're doing. When f- when it comes time for summit we may also see if you're interested in speaking or sponsoring. I don't know if Trust Insights does any sponsorship.
[00:43:27] Christina: But that's actually the way... Yeah, that's actually the way that we end up funding what we do, is with with sponsorships. I totally understand that you don't do that. I don't think you need to do that given the work that you're doing and given all of your, your free this and free that.
[00:43:42] Christina: That's... You should be earning your your business that way. Yeah. And good for you that you are. So but I make that offer to you if you are interested in contributing content or interested in leveraging this community you're welcome to do where are you based?
[00:43:55] Christopher: Boston
[00:43:56] Christina: In Boston. Yeah, I remember seeing something about Boston in [00:44:00] your work.
[00:44:00] Christina: We don't hold any of our events currently remotely or in Boston, but we do some that are remote. So if you wanted to host an event as a way to create some visibility for your work in this space, we could do a hybrid event where you have some people who are local and some people who are remote that we could do from your offices in Boston if that's an area of interest for you.
[00:44:23] Christina: If we wanted to do, for example, the executive roundtable on local AI, you want to participate, but you're not in San Francisco, we could have a Boston and a San Francisco. You could be the host of the Boston. My team could be the host of the San Francisco. We already do them hybrid so people can dial in remotely.
[00:44:41] Christina: So something like that. If that's of interest to you. I'm just putting it out there. Enjoy the dinner. Your work is really powerful, and I appreciate the chance to interview you for the podcast. I hope you found it valuable.
[00:44:54] Christopher: It was fun.
[00:44:56] Christina: Good. I hope you find it valuable. I hope it turns out to actually [00:45:00] lead to some business for you.
[00:45:01] Christina: We will edit it and post it for next Thursday morning. Okay. And you will have a audio card that you get that we use in our LinkedIn promotions that we'll take a little clip from this and play the audio automatically on LinkedIn, which you can use for promoting the podcast if you wish on your LinkedIn channel or on your company site or wherever you would like to use it.
[00:45:23] Christina: Okay?
[00:45:23] Christopher: All right. Thank you so much.
[00:45:25] Christina: Okay, thank you. Any questions for me before we ring off?
[00:45:28] Christopher: Not at this time. Thank you.
[00:45:30] Christina: All right. Thank you, Christopher. Take care, and I really appreciate your time today.
[00:45:33] Christopher: You too. Take care. Bye-bye.