Agentic AI and Revenue Work: What Actually Pays Off

GTMAI Adoption
Written By Christina Ellwood

Episode Summary

Christopher Penn, co-founder and Chief Data Scientist at Trust Insights, argues that agentic AI is displacing whole workflows rather than individual tasks, and that most organizations cannot prove it because they were not measuring outcomes to begin with. He walks through using AI as a synthetic voice of the customer, the frameworks he uses to decide which work to hand over, where revenue has actually shown up, and the security posture he recommends for autonomous agents. The through line is measurement: what to instrument, what to ignore, and how to prove lift to a finance team that has heard the pitch before.

Key takeaways

  • There are two ways to use AI: optimization, doing what you already do faster or cheaper, and innovation, doing what was not possible before. Most organizations only attempt the first

  • AI can act as a synthetic voice of the customer. Penn cites peer-reviewed work showing generative models reproduce purchase intent at roughly 90 percent accuracy

  • You cannot measure AI ROI if you were not measuring outcomes before. The frameworks come second, the baseline comes first

  • Roughly 11 percent of Trust Insights’ closed deals came from AI tools recommending them, which is what generative engine optimization actually looks like on a P&L

  • Treat an autonomous agent the way you would treat an untrusted contractor. Isolate it, and do not hand it credentials it does not need

  • An agent cannot succeed if it does not know what success looks like. Give it a quantifiable target it can iterate against

  • •    Add one sentence to every prompt: ask me questions until you have enough information to succeed at the task. Penn calls this the single highest-leverage change most people can make

About Christopher Penn

Christopher Penn is co-founder and Chief Data Scientist of Trust Insights, where he works at the intersection of marketing, analytics, and artificial intelligence. He has worked in data science and predictive analytics since long before the current AI boom, and is known for making complex technical concepts usable by real teams. He co-hosts the Marketing Over Coffee podcast and the In-Ear Insights podcast, and is a speaker, author, and educator focused on what AI can do, what it cannot, and how to apply it responsibly for measurable results.

 

In this episode

00:00 Welcome and guest introduction
01:25 Optimization versus innovation, and the synthetic voice of the customer
03:34 Regression AI, and measuring campaign lift
05:00 Classification AI, and what it is good for
06:12 What Trust Insights does
07:05 Organizational friction as the real impediment
07:41 Why you cannot measure ROI without a baseline
09:00 The 5P framework
10:00 TRIPS, and choosing what to hand to AI
10:49 The advice people actually took away
11:29 GEO, and 11 percent of closed deals
12:30 Turning workshop material into a published book
14:12 What AI Realized has been doing on GEO
14:51 Competitive intelligence from 1,900 job listings
16:15 Outside-in analysis and enterprise pipeline
17:36 Hermes Agent, and autonomous research
20:34 Where to start
20:49 The five levels of AI enablement
23:15 What safety actually means here
24:11 Treat an agent like an untrusted contractor
25:28 Why most executives have never stood up their own system
26:06 Defining success so an agent can hit it
28:20 Applying this in sales and marketing
29:14 Your sales playbook, your methodology, and coaching from call transcripts
30:56 The one thing to take away
32:51 Resources
33:23 Leadership skill: project management
34:31 Wrap-up

In Christopher’s words

“There’s two fundamental ways to use AI. Optimization, doing the things you already do faster, better, cheaper. And innovation, doing things you could not do before.”

— Christopher Penn (01:25)

“How do you handle an untrusted contractor? You don’t sit them down in front of your most sensitive systems and walk away.”

— Christopher Penn (24:11)

“Ask me questions until you have enough information to succeed at the task. If you give it that one sentence, everything gets better.”

— Christopher Penn (31:04)

“You can’t measure the ROI of AI if you weren’t measuring the outcome in the first place.”

— Christopher Penn (07:41)

“Project management. Being good at designing complex projects, because that is exactly what working with agents is.”

— Christopher Penn (33:23)

 

Resources

Christopher Penn and Trust Insights

•    Christopher Penn on LinkedIn: linkedin.com/in/cspenn

•    Trust Insights: trustinsights.ai. The starting point he names on air, leading to the blog, podcast, YouTube channel, and live stream

•    Trust Insights Bookstore: trustinsights.ai. Where they publish books generated from workshops, one of the revenue examples in this episode

•    Marketing Over Coffee: marketingovercoffee.com. His long-running podcast with John Wall

•    In-Ear Insights: trustinsights.ai. The Trust Insights podcast on analytics, AI, and data

•    So What? live stream: trustinsights.ai. Thursdays at 1pm Eastern

•    Christopher Penn’s personal site: christopherspenn.com. Blog, newsletter, and speaking information

Frameworks

•    The 5P framework: trustinsights.ai. Purpose, People, Process, Platform, Performance. Created by Katie Robbert, extending People, Process, Technology by bookending it with why you are doing the thing and how you will measure success

•    The TRIPS framework: trustinsights.ai. Time, Repetitiveness, Importance, Pain, Sufficient Data. A scoring matrix for deciding which tasks to hand to AI

•    Five levels of AI enablement: Done by you, done with you, done for you, done without you, done in advance of you. Penn’s maturity model

•    Leavitt’s Diamond: Harold J. Leavitt, 1964. People, Tasks, Structure, Technology, the model that became People, Process, Technology, which 5P extends

•    Challenger, solution selling, insight selling: The sales methodologies he names as things your playbook should encode so AI can evaluate rep compliance

Research and data

•    LLMs Reproduce Human Purchase Intent via Semantic Similarity Elicitation of Likert Ratings: PyMC Labs and Colgate-Palmolive, October 2025. The roughly 90 percent accuracy figure behind the synthetic voice of the customer

•    Burrows’ Delta: A stylometric measure of authorship similarity. Penn uses it as an objective success metric a model can iterate against, for example a value that cannot exceed 1.25

•    Uplift modeling, propensity scoring, Granger causality: Statistical techniques from bioinformatics applied to marketing attribution. He began applying them to PR measurement in 2013

•    Google TurboQuant: Google Research, ICLR 2026. Vector quantization for KV cache compression, cited as one of two papers transforming local AI economics

•    Google Multi-Token Prediction: Released for Gemma 4, May 2026. Speculative decoding delivering up to 3x faster inference without quality loss

Related AI Realized episodes and events

 

Frequently Asked Questions

  • Christopher Penn describes building virtual focus groups from ideal customer profiles and putting questions to them the way you would to a real panel. He grounds it in peer-reviewed work from PyMC Labs and Colgate-Palmolive showing generative models reproduce human purchase intent at roughly 90 percent accuracy. His framing is that this is innovation rather than optimization: not a faster version of research you already do, but research most organizations could never afford to run at that frequency.

  • Because they were not measuring the outcome beforehand. Penn is blunt that this is the most common blocker he encounters, and no framework fixes it, since a framework applied to an unmeasured baseline still produces nothing a finance team will accept. His related point is that organizational friction, not technical capability, is the real impediment to adoption.

  • TRIPS stands for Time, Repetitiveness, Importance, Pain, and Sufficient Data. It is a scoring matrix for deciding which work to hand to AI. The strongest candidates consume significant time, are highly repetitive, have plentiful examples of what success looks like, and nobody enjoys doing them. It pairs with the 5P framework, which sets purpose and performance around people, process, and platform.

  • Purpose, People, Process, Platform, Performance. Created by Katie Robbert at Trust Insights, it extends the familiar People, Process, Technology model, which itself descends from Leavitt’s Diamond, by bookending it with two questions organizations usually skip: why are you doing this, and how will you know whether it worked. Penn uses it as the structured entry point for AI adoption.

  • Penn gives concrete numbers rather than projections. Roughly 11 percent of Trust Insights’ closed deals came from AI tools recommending the firm, which is generative engine optimization showing up on a P&L. Workshop material converted into a published book in under six hours generates a thousand to two thousand dollars per title. And competitive intelligence drawn from around 1,900 competitor job listings inferred a pharmaceutical client’s 12 to 18 month strategic priorities.

  • Penn’s answer is to stop treating it as a novel problem. Handle an autonomous agent the way you would handle an untrusted contractor: isolate it, give it only what it needs, and do not seat it in front of sensitive systems unsupervised. In practice that means an air-gapped or DMZ environment without sensitive credentials, plus using the permission controls the tools already provide rather than defaulting to full access.

  • Done by you, basic chat and copy-paste. Done with you, where SOPs are baked into small purpose-built apps. Done for you, managing agents the way you would manage a virtual employee. Done without you, fully autonomous agents. And done in advance of you, an autonomous agency operating ahead of your involvement. Penn maps the levels to what an organization is actually ready for, rather than treating the top as the goal.

  • Adding one sentence to every prompt: ask me questions until you have enough information to succeed at the task. Penn names this as his top takeaway of the conversation, and says the change in output quality is dramatic and immediate. It works because it converts a one-shot instruction into a short requirements conversation, which is closer to how you would brief a capable colleague.

 

  • [00:00] Christina Ellwood: Welcome to AI Realized, the podcast for enterprise executives leading AI deployments. From tackling security, data, and operational challenges to navigating organizational transformation, AI deployment offers a unique opportunity to redesign our organizations from the inside out. I'm Christina Ellwood, your host for today's episode, and today we are joined by Christopher Penn, co-founder and chief data scientist at Trust Insights. Christopher's been working in data and AI science and data science long before the current generation of AI, building practical applications in attribution modeling, predictive analytics, and customer journey intelligence. He's also a globally recognized keynote speaker and co-host of the long-running Marketing Over Coffee podcast. What makes his perspective especially valuable is his focus on turning AI into measurable business outcomes, helping organizations connect data to real decisions, real performance, and real ROI. Christopher, welcome to AI Realized.

    [01:03] Christopher Penn: Thank you for having me.

    [01:06] Christina Ellwood: We're very happy to have you, and this whole area of AI in the vertical functional groups within our organizations, like sales and marketing, is the sort of topic of the day. So at a very high level, where do you see AI actually working to drive measurable business outcomes in sales and marketing?

    [01:25] Christopher Penn: The ways that it's used best these days the-- the-- I should s- take a step back. There's two fundamental ways to use AI, and w- this incorporates all three branches of it regression AI, classification, and generative AI. And those three branch-- Those two ways are optimization and innovation. Optimization is do what you've always done, but do it bigger, better, faster, cheaper, and a lot of folks these days are emphasizing faster and cheaper. And then innovation, which is do what you've never done before because you've never had the capabilities. And this i- I'll give a very trivial example. Imagine taking something like an RFP response and turning it into a country music song. Could you do that with AI? Yes. Should you? Eh, it's debatable. But it is something that you traditionally would never have done before because you just didn't have the capabilities of doing so. When we talk about AI in the enterprise, and in particular i- in useful ways of using it, one of the most valuable ways is to have it be a synthetic voice of the customer. Back in summer of 2025 there was an academic paper, and I'm struggling to remember the name of the paper but it showed in peer-reviewed research that generative AI models accurately repr- replicate purchase intent, buyer intent with about 90% accuracy. So- If you were to, for example, build, take your existing ideal customer profiles and put them into the generative AI tool of your choice, or even better, an agentic AI tool of your choice, and have them function as a virtual focus group, you basically have a customer on tap 24/7 to ask questions of. "Hey, I've got two subject lines. I've got this, I've got that. I've got a new product idea. I w- I need to raise my prices." Tell me, machine masquerading as my customer, what my customers' probable reactions are going to be to this, and how can I message it in a way that will minimize risk or maximize the revenue opportunity?

    [03:27] Christina Ellwood: That's a fabulous example of using generative AI. Do you have examples for the other two types?

    [03:34] Christopher Penn: So regression AI is the classic this is, goes back to the Eisenhower administration in the 1950s. And one of the most powerful tools here is what's called uplift modeling, and there are statistical methods like Granger causality, et cetera. One of the pr- challenges that marketers face is imagine you have all of this data. You have your web traffic, and you have your CRM data, and you have your social media data, and this, that, and the other thing. And you say, "Hey, we have this campaign in flight," or whatever, something that you did. And you say, "I wanna know what the impact of that thing was." But at the same time, you've got Google Ads running. You've got YouTube ads running. You've got emails going out the door. You've got billboards all around the city. It's very difficult because it's a lot of noise. Good multivariate regression techniques like uplift modeling, propensity score modeling, Granger causality, et cetera, can be used by you in partnership with your favorite generative AI tool to say, "Given all of this data, can we build a statistically valid model that shows the uplift of this thing?" This is something we get from bioinformatics, right? When you say I have all this information about this patient, and we administered a treatment. What is the effect of the treatment on the patient, especially given how much other noise there is?" I first started working with this type of modeling back in 2013 when I was working at a PR firm, because public relations is notoriously difficult to measure because there's not a direct click stream between positive press coverage and something else. And so I was brought into the firm I was working at the time to try and help them build models like this that would help them understand what the true lift of a PR campaign was. So that's regression. And the second one, classification, is one of my favorites because it's all about organizing your data. If you've got a bunch of data that has no structure or things, you can bring structure to that data and then bring insights out of it. For example, things like basic sentiment analysis, f- topical focus. Given a piece of informati- a pool of information, what in this information is relevant and what is not? So imagine you have vast quantities of social media data, and you wanna know which of this stuff is actually about us, particularly if you have a company like our company's named Trust Insights. That's a fairly common phrase, so how do you use these tools to identify things like named entities, then be able to measure h- your presence within a pool of data?

    [06:06] Christina Ellwood: Is that the work of Trust Insights as an organization?

    [06:12] Christopher Penn: It is partially. So we are a consulting firm. We focus on, these days, AI implementation and enablement, helping organizations. My CEO and co-founder, Katie Robbert, is our organizational change and change management expert. She's the one who comes in t- to an enterprise and says, "Yeah, you've got the shiny new toys, but toys themselves are not enough. You need to enable the people and your processes to match the platforms." And then she came up with this framework called the 5P Framework by Trust Insights, which is purpose, people, process, platform, performance. Why are you even doing the thing, and then how do you measure success as the bookend to your traditional people process technology, which was originally from what, H.J. Levitt in 1964 was, the diamond framework back then became the people process technology trope, and then Katie extended that to say having purpose and performance as the bookends for it.

    [07:05] Christina Ellwood: Yeah obviously the organizational friction is the largest impediment to the adoption of AI. From the time that AI Realized was first formed in '20- in October of 2024 we heard this from the stage, that 100% of the companies were struggling with that friction. It remains the most cited source of friction today. It sounds like that's part of the work that you've been doing. So what are the underlying measurement and decision systems? What do those look like inside of those organizations, and how do they relate to the friction of adoption?

    [07:41] Christopher Penn: One of the biggest interesting pieces of feedback we get from people who are resistant is the immediate demand, "Oh show me the ROI of AI." Which we always say So how are you measuring the ROI of this task now? And the answer is we're not. Then you can't measure the ROI of AI 'cause you're not measuring it now. You have no basis for comparison. And so that's one of the things that we immediately tend to flag. But you can me- everything has to come back to three basic imperatives, right? You're either saving time, saving money, or making money. Everything you do has to, in some way, have line of sight to one of those three objectives, ideally more than one, in order for you to demonstrate impact. When we work with organizations, the first imperative that they go after is always saving time. How can we free up time? How can we free up resources and stuff like that? And that's a great first place to start because there are a lot of process inefficiencies, and there are a lot of capabilities that today's agentic AI systems can do that can take on a task completely. I'll give you an example. This, again my CEO is a non-technical person. I'm like the, I'm like the button-pushing propeller head in the company. But we've been using Claude Cowork a lot, which is a non-technical agentic system. And we measured her output in terms of important deliverables, strategic blueprints, this, that, and the other thing for ourselves and for other clients in the period, the six-month period prior to work, to Claude Cowork coming out, and then in the three-month period after when she started using it. And her productivity level as a senior executive, as a leader, is literally 100x what it used to be. That is how much more productive she is at doing things like budgeting, forecasting, strategy strategic council, and all the way down to website design, like rebuilding slide decks, all of the stuff that there's so much manual drudgery in, like making a slide deck, that if you can use your brain to get the valuable stuff out of your head and then hand off the typing essentially to a tool, you can dramatically increase the results you get. And so that's the first enablement that we tend to go after with people is to say, "Let's get your to-do list under control personally or organizationally." If your department has a punch list- We have a framework we call TRIPS, time, repetitiveness, importance, pain, and sufficient data, and we score tasks by that. When we do a department or company-wide audit with somebody, we'll say, "Okay, let's score all of your deliverables and tasks by this matrix and identify the top 10 tasks that are just a waste," right? They, the, it consumes a lot of time, highly repetitive, tons of examples of success, and nobody likes doing it. Those are the tasks that immediately you should be handing off to AI because no one's, no one gets upset about it, right? There's a lot of conversation about, the future of work. Nobody is we've, and ze- zero clients have ever said, zero people ever said, "You know what? I would like to keep doing my expense reports by hand."

    [10:49] Christina Ellwood: This is the this is the first piece of advice I think that people took away when this conversation was started, which is start with something people hate. And that's a wonderful example of that. And I myself have found Claude Cowork to be an extremely valuable boost to my productivity as well, so I resonate with that. But I think the revenue one is the one that needs some attention. And I'm very curious what you're seeing AI actually do to improve attribution, for example, or conversion, or qualification, any of the elements of moving a deal through the funnel. Can you illuminate can you illustrate some examples there?

    [11:29] Christopher Penn: Sure. So a couple of really obvious ones. Number one what is now called, among other things, GEO. There's so many different variants to the... But AI basically making recommendations. 11% of our business in the last six months has come from AI recommending us because we have been s- working, and we've understood the space and we've been working in the space for a while. We've been doing what we know works based on the architecture of the technology and how the technology functions. We've been planning ahead for years for this, and it works. That's 11% of our closed won deals come from generative AI tools recommending us. So we- we'd love to be case study zero for a lot of these things. In terms of revenue generation, one of the things these tools enable is y- is better product market fit and new products. So As an example, I t- I typically deliver, I'll go 20, 30 workshops in the in the spring and the fall to all kinds of different industries. Now, with tools like Claude Code, I can take my workshop that I did, take all the materials and outputs from it, put it through Claude Code with a recipe that I built, plus a bunch of just some custom Python code, and in about two and a half hours, it spits out a polished book that then goes immediately into our company bookstore, and it goes up for sale, and typically generates, $1,000 to $2,000 of extra revenue. We just published, Generative AI for Destination Marketers. That product did not exist, and instead of taking six to nine months to bring it to market, I can have it to market in under six hours and immediately begin monetizing it. At a broader level, rationalizing your product portfolio or looking at product market fit, you... those customer profiles we were talking about earlier. If you have a product and it's not selling, you take your ideal customer profile, and you literally say, "Why is," "Why are you not buying this?" Especially if you have other signal data that you can provide, like your inbox, your call center data, et cetera, to better understand this is why the, our customers are not buying this thing. You have macroeconomic data. Why are people booking fewer hotels? 'Cause everything costs more money. Gas is $5 a gallon. People aren't traveling. That's why. And so revenue generation can be improved product market fit. It can be net new products that you didn't have for sale before, especially in ways that you've never done before information products and things like that. And of course, referrals from these new tools that are, if you have done a good job of being present and having stuff out there for the machines to learn from, become a word-of-mouth tool, but the word of mouth is being spread by a machine instead of human.

    [14:12] Christina Ellwood: Yeah, so I should say for our listeners that we have done a number of things on GEO recently in the AI Realized community. We have a webinar that you can find on on the YouTube channel. We've got three articles on GEO, and we just did a roundtable and published a readout from the roundtable discussion. So this area of GEO for marketers and for founders and for CEOs is really important, and it plays back into something you said earlier, which was the importance of PR. Can you walk us through a specific example where AI changed a business outcome in sales or marketing, not just the analysis, but the decision and the result?

    [14:51] Christopher Penn: In terms of an outcome this is gonna be a tricky one to thread the needle very carefully. We had a pharmaceutical client and they wanted to know-- They're a very large company. They wanted to know what their largest competitor was doing. So what we did was- We took 1,900 of the competitor's open job listings, downloaded them, digested them with a language model, and then essentially did a large-scale inference t- model to cut, to say "Where... What is this company's 12 to 18-month strategic priority?" Because you don't hire people for things that are unimportant, right? You don't add headcount for stuff that's- ... not relevant. You add headcount for stuff that is strategically important. And what- ... we found was that there were three different product lines, but one in particular that this competitor was hiring like crazy for. We handed that to our client. Our client said, "We don't have a good business answer to this competitive challenge," and they spun one up and essentially. Now, we don't know we don't know what happened to their version- ... of that product line because we were not privy to those conversations. But the people who were our stakeholders were like, "This is probably the most important thing that we've ever done with AI because we can now anticipate what a competitor's doing i- in their 12 to 18-month strategic horizon based on the hiring data."

    [16:15] Christina Ellwood: Yeah, absolutely. And in, in fact, I have interviewed a founder of a company that does outside-in analysis using AI, and their wedge business case is with B2B sellers, and they are able to bring them opportunities that were identified using outside analysis of the target accounts for their KPIs, their gaps in their performance, and deliver those opportunities with a built-in business case for why the company should be buying. Now, that's especially powerful when you're looking at complex product portfolios. So if you've got, 50, 100, 200 products that you're trying to sell into enterprises that are large and complex, you simply cannot do that mapping with human beings, and AI is ideally suited to doing that. And they're c- they're creating hundreds of millions of dollars worth of new pipeline, which are essentially SQLs on a silver platter for an enterprise seller. Now, that's just not something that could have been done without having an A- not just the AI, but the system sitting on top of it. So the business logic layer that sits on top of it that does all of the triangulation and root cause analysis. Now, that's something I'm hearing you say, too, when you say especially with agents. I think that's what you're referring to, is that the agent systems are the, where the logic is living in addition to the automation that allows for some of these decisions to be taken. Can you double-click on that?

    [17:36] Christopher Penn: So I've got one running right now, an assistant called Hermes Agent, that is going out and it is identifying every trade association that's having an event in, within two hours of my house within the next 12 to 18 months at all the different venues and stuff like that. So it's going out, it's browsing the web, it's grabbing all the data, it's assembling the data, it's identifying the contacts for those people, either by email address or LinkedIn profile, and then it provides me a rank ordered list of here's the associations that I should pitch as a keynote speaker to speak at their events. I am doing none of this. I gave it a 13-page project plan to start, and then I'm hands-off after that. It does everything else downstream of that. That, and that is what today's agentic systems are capable of. We have what we call the five levels of AI enablement, and it maps to product market fit. Done by you, done with you, done for you, done ahe- without you, done in advance of you. So done by you is level one. Chat, ChatGPT. You're the copy-paste monkey. You're typing all the time. You're copy-pasting, and it... You get some gains, but not much, because you the human are essentially the machine operator. Level two are things like Gems and GPTs, standard operating procedures you've baked into little mini apps in these tools that increase efficiency, but you the human are still copy-pasting an awful lot. Level three is where you go from individual contributor to manager of an agent, of a, almost like a virtual employee. These are systems like Claude Code, Claude Cowork, et cetera, where you are now managing, you are delegating tasks. Level four is systems like Hermes, OpenClaude, Dear Flow, you name it, that all the different autonomous agents. You give them a project plan, maybe even a job description, and they go off and do the job. And then level five is not just an autonomous person but an aut- autonomous agency or company where, a system like Paperclip, for example, or any kind of control plane similar, you are delegating the project to this ag- virtual agency that just does all those tasks that you need to bring something to, to market. I've got another tool running right now. I am g- I don't know how it's gonna turn out. It could be, it could go well, it could go horribly wrong. But it's tied into an investment app, and I've given it the charter "I'm gonna give you $25. You have to f- you have to, using these APIs and these platforms, you have to turn this into $100. You can issue trades, you can choose which assets to buy, et cetera, but you figure it out. And, grab all the data you need to do back testing, select which algorithms, select which statistical, causal algorithms make the most sense, and then run a live test and see if you can turn 25 bucks into a thou- into 100."

    [20:24] Christina Ellwood: Okay, that's that's ambitious, , on your part- ... to maybe doing that. I applaud your creativity there.

    [20:30] Christopher Penn: Where should- if it works, you'll never hear from me again because- Exactly ... I will be a billionaire. You'll be retiring.

    [20:34] Christina Ellwood: Amen. Absolutely. I appreciate that, that spirit. So where should people think about, or listeners think about doubling down right now to get more value? What's actually compounding and working for people right now?

    [20:49] Christopher Penn: So go back to the five P framework by Trust Insights, which is purpose. Why are you doing the thing? Forget AI. What are the five biggest things that are, have got your hair on fire right now? Take those five things and decompose them into their individual tasks. What, and then what of these tasks are follow the trips framework that you can clearly identify this is a task that a machine should be doing. There, no human should be doing this particular task. It doesn't have to be the whole job, but if there are... That problem is composed of tasks. Decompose that and then map the people, the process, and then the platform to those tasks, and then c- for each task, identify a measurable, quantifiable, objective outcome that a machine can tune against and come up with answers that, that will deliver value. That's how you get value out of AI is by take, by putting it in its place in the larger strategic structure, and then saying, "How can we decompose this problem to the individual pieces, and then which of those pieces can we hand off to a machine, and where are we still gonna have humans as blockers?" A lot of people and this is something my CEO, Katie Robbert, talks about all the time, is a lot of people start with the technology. "Oh, we've got to use AI for this. We've got to use AI for this." No, you've got to figure out what is the problem and how do you measure the solution to the problem in clear, objective, quantifiable ways. This, it's the old management trope. If you can't measure it, you can't manage it. That has never been more true than with AI. For example, very st- simple example, let's say you're doing some creative writing or just doing any kind of writing, and you have a writing style guide. There is a measure of text s- author, authorship similarity called Burrows' Delta that a machine can analyze a piece of text and understand. You can run this mathematical comparison. If you say to a machine, "I want you to write like me," you will use Burrows' Delta with, and the value cannot exceed 1.25. Now, a language model like a cloud code can iterate and iterate until it hits that success number, until, because it knows what success looks like. So many people don't get value out of AI because they do not know what success looks like. They cannot quantify it, and as a result, the machine can't quantify it, and the machine has no idea whether it has succeeded or not

    [23:15] Christina Ellwood: How do you recommend people factor in the safety element? 'Cause you're right, they can't do what you... If you can't tell what the outcome is, it can't achieve the goal. How do you do that and make it safe at the same time?

    [23:27] Christopher Penn: What do you mean by safety? Define safety specifically.

    [23:30] Christina Ellwood: So agents can go rogue, for example. You can have data exfiltrated, you can have them be a portal in for a hacker. There's different ways in which the agentic system falls outside of our security systems today. And there's been quite a bit written about that both in our Substack newsletter as well as in in, in the w- in the wild by various security experts. Given that you are using agents in autonomous fashion, that is the higher risk profile for an agent is when they are autonomous rather than having a human in the loop, how are you factoring in safety and what is your advice to others as they consider building autonomous agents?

    [24:11] Christopher Penn: The, go back to the way you've always done it. How do you handle an untrusted contractor? You don't sit them in the middle of the office on the executive floor with a, root level passwords to everything, right? That would be a disaster. You say, "No, consultant, you're gonna sit here in this little locked room with this laptop that's been air gapped from our network and you do your work in this little concrete room." It's like what all the three letter agencies do. They have a little concrete room in the basement of Langley where the those untrusted entities get to work. So in my agentic setup, for example, I have a little B link box. It's it's its own little computer. It's, it is firewalled outside my, my, my main setup, so it can't even cross into my main... It's basically, it's on its own little island, and I can, terminal into it and things like that. But on that box, there's no sensitive data. There's no way for it to be able to use credentials it doesn't have access to. There is no way... So even if it runs across hostile code and, it gets prompt overridden by some hostile code, it's in a d- it's in a DMZ. It's the same thing we've been doing for, what? 30 years with Wi-Fi. We have DMZs for, in our Wi-Fi networks to keep the main network safe from guests. This is not new, it's just people forget how we've done security and safety in other places.

    [25:28] Christina Ellwood: I think to be fair, many executives have never had to stand up their own system. You have business people who are creating these agents and so forth, and it's the first time they've ever built a product, if you will. And they haven't had to think that through. Let's do a little thought experiment. So for the things that you run on your laptop, which are, is connected to the internet, is connected to systems of record, and is your your main terminal, when do you decide, "Oh, this needs to go over here on my DMZ isolated, air-gapped, s- super-duper secure computer over here"? What's an example of an app you would do in one versus the other?

    [26:06] Christopher Penn: Any time that I'm using any system where there's an option that built in... So Claude, for example, calls it dangerously skip permissions, right? And if you want a task to run autonomously, you use dangerously skip permissions. That is the mental red flag to say, this does not belong on the production system. The moment you have to say, "I want this to be less hands-on. I wanna run this in auto mode. I wanna run it in YOLO mode," that's when you know it goes to the other machine. When you, the human, are constantly hitting, "Do you approve this? Do you approve this?" And you actually read the, read what it's asking you to do that's when you can have a greater degree of safety to go, to know, okay, this is asking me permissions. Or better yet, if you know what you're looking at in a tool, you can say, "Okay, you're asking me for permission to this command. No, you may not do that." "Hey, can I remove all these files?" "No, you may not." Another thing that really helps is having strict guardrails and permissions within the apps themselves. Now, yes, they can be hijacked in the worst case scenarios, but most bad things happen because people provide insufficient guardrails. A simple example, in a Claude code, you have list of permissions that you can give it out of the box. One of them should be you are forbidden to delete files. You b- you know, you may not use this command ever. If you want, if you don't think of something relevant, move it to the archived folder so that I can move it back if you're wrong. So it's it's defining what those rules are, your first principles. Every project should have first principles. Some are gonna be universal, like you should use existing software that's pr- known and proven to work rather than re- you know, reinvent the wheel every single time you approach a task. Other things are those, you say like you can only operate in this folder and depending on the system you're on, you, like I said, you may need to move it to a separate box so that even if it does decide, to override its own permissions, it can't leave its little containment box. All of this is stuff that should be at part of your AI council and your AI governance, that these rules and things should be decided up front

    [28:20] Christina Ellwood: so let's talk about applying this in the world of sales and marketing since that's the area that you are an expert in. Is there any specific advice for the sales and marketing executive and team that is adopting AI either agentically for task automation or autonomous automation? Is there anything specific that you would recommend to them? My number one r- recommendation is work with your tech team, obviously, but hook into your CRM, use, and using the data that you have, and your sales playbook. If you don't have a sales playbook written down, you should. Once you have that, you could have, and you should have a either an agentic system or a who, who cares what the fancy label is, that looks at every deal in your CRM against your sales playbook and says, "What is the next best action that we need to take on this deal to get it to move forward?"

    [29:14] Christopher Penn: 'Cause your sales playbook should have your main sales tactics, your methodology. Do you use solution selling? Do you use insight selling? Do you use challenger? What is ... whatever the methodology is that you use at your shop should be embedded in there. You should have battle cards in there. You should have objection handling in there so that you can extract all the data from your CRM, maybe from your call center system, particularly if you use something like Gong, put it into a language model programmatically through a tool like a Claude code or whatever, and build and then augment back into your CRM. This is the next best action for this deal. If you want this deal to move forward based on our sales playbook, this is what you should do. And this, and you can do this i- we built this for a couple of clients now. Not only does that help advance sales more quickly, but then you can take things like call transcripts, map them back against the sales playbook and say, "These are our sales reps who are out of compliance with the sales playbook." We use challenger methodology, and this, the rep here is just not doing a good enough job of, step three framing. On all their calls, they fall down at step three framing. They need coaching most there. Because the tools can identify in a call transcript where a person is skilled and where a person falls, falls short, and then you can coach that person to say "Okay, you need to work on this," and then build them a a- an interactive tool, again, with the language models of your choice and ideal customer profile to help them train, to help them have that simulated conversation where the simulated prospect comes back with an objection, "Oh, I don't wanna sign a 12-month contract." Okay how, what is, what have you learned from your training that will help you overcome that objection?

    [30:56] Christina Ellwood: Gotcha. So if our listeners could take one thing away from our conversation today, what would you want them to take away?

    [31:04] Christopher Penn: At every level in the use of generative AI, if you ask it If you give it one sentence, everything will get better immediately. And that one magic sentence is, "Ask me questions until you have enough information to succeed at the task." Ask me questions until you have enough information to succeed at the task. Whether it is Claude Code, whether it's with Agent, whether it's ChatGPT Basic Edition, everyone forgets that they, that the, these AI tools do not generally have enough information, and so they infer and guess a lot. And when you're using these tools, whether you're using it for management, for sales, for marketing, for whatever, you are forgetting that it does not know. Even if you're providing it with, some data or you've got it connected to your internal systems, it still doesn't know everything that you think it knows. And if you add that one sentence, especially when you're starting out on a project of any kind, you will immediately get 5X better results because the model will stop and say I have some questions like who's this for?" Or, "Have you tried to solve this problem already?" And at every level, every grade of experience, I use this all the time, and I consider myself to be a fairly f- proficient user of these tools. I use this all the time, and I am constantly surprised by how the machine challenges me to think deeper and to provide information. And I ... More often than not, I'm going, "Ugh, why didn't I think to include that?" And now I do. I

    [32:38] Christina Ellwood: have that experience myself. I appreciate that. So what resources would you recommend for listeners who wanna learn more about your work and about adopting AI for their sales and marketing organizations?

    [32:51] Christopher Penn: The best place to start is trustinsights.ai. And then you can get to our blog, our podcast, our YouTube channel, our live stream, our Instagram, all these places. There's so many, but the starting point is trustinsights.ai, and you can find everything from there.

    [33:07] Christina Ellwood: All right, great. In the AI revolution, you've been around the track a few times. You've been a leader in a number of different contexts. In this AI era, what's the leadership skill that you find most valuable right now?

    [33:23] Christopher Penn: Project management. Project management, being a, being good at des- d- designing complex projects because if you're really good at that, like my CEO Katie is, you can anticipate all the things that are likely to go wrong. And when you're working particularly with agentic systems, if you can anticipate what's gonna go wrong and get ahead of it, you will get 10X better results than the person who's just next to you winging it and hoping that they can chat their way through it. As models get smarter, this is something Ethan Mollick of Wharton says all the time, which I love. As AI gets smarter, it makes smarter mistakes that are harder to detect. So you have to stop being its chat buddy and start thinking like a project manager and say, "This is the task." Using the five P framework by Trust Insights, what is the purpose? What does success look like? Who are the people involved? How do we do this? What tools and technologies should we use? If you have those skills, you are well-positioned to, to get heads and shoulders better results out of AI than somebody who doesn't.

    [34:31] Christina Ellwood: Christopher Penn, co-founder and chief data scientist at Trust Insights, thank you so much for sharing your experience and your advice today with the AI Realized audience. We really appreciate you being with us.

    [34:45] Christopher Penn: Thank you for having me.

 
Listen to other episodes
 
 
 
Join AI Realized
 
 
 
Subscribe to the newsletter
 
 
 
Christina Ellwood
Previous

Artifact-Scoped Agents: Stop Mimicking Job Titles
Next

AI Search Visibility: When AI Says Your Company Is Dead