Analytics as Code: Why AI Stops Guessing With Data
Episode Summary
Data work is supposed to be the source of truth, which is why a model that hallucinates one percent of the time was unusable for it. Chris Parmer, co-founder of Plotly, explains what changed: AI got remarkably good at writing code, and Plotly’s customers were already doing analytics in code rather than in point-and-click tools. That shifts the question from whether you trust the answer to whether you can read the code that produced it. He covers who inside the enterprise picks this up first, why organizations end up with tens of thousands of dashboards, and what he has learned running proof of concepts with customers on technology this new.
Key takeaways
The hallucination objection changes shape when AI writes code rather than answers. Data work has to be accurate, and a model wrong one percent of the time cannot support decisions, but code can be read and verified
Analytics as code is not new. Quantitative finance, bioinformatics, and classical engineering have worked this way for years, in Python, Jupyter, Spark, and SQL. What is new is that AI can now write it
Organizations accumulate thousands or tens of thousands of dashboards because a new one gets built for every question, and only the trained BI person can build them. Natural language removes that bottleneck
The people adopting fastest are the stakeholders who used to wait on a data scientist, in quantitative finance, trading, portfolio management, energy, and utilities
Parmer’s rule of thumb for switching tools: it has to be at least one and a half times more productive before people jump ship. Much of AI clears that bar easily, taking weeks to days and days to hours
A blank prompt box is paralyzing rather than freeing. Auto-generating several ways to look at the data beats asking the user what they want
About Chris Parmer
Chris Parmer is co-founder and Chief Product Officer of Plotly, the company behind the Plotly graphing libraries and Dash, the framework for building analytic and data applications in Python. His work centers on making sophisticated data analysis available to people who are not professional data scientists, and on analytics as code: doing analysis by writing code rather than clicking through a business intelligence tool. He writes regularly on building products on top of AI.
In this episode
| 00:42 | Welcome and guest introduction |
| 01:33 | The hallucination problem in data work |
| 01:58 | What changed when AI got good at writing code |
| 04:07 | Where analytics as code is actually used |
| 05:41 | What this means for executives who are not building products |
| 06:17 | The career path for analysts leveling up |
| 07:04 | Why companies end up with tens of thousands of dashboards |
| 08:14 | The executive who keeps asking for another cut of the data |
| 10:10 | Prompting, and why a blank slate is paralyzing |
| 12:31 | The audience that arrived once the barrier dropped |
| 14:52 | Where adoption is concentrated in the enterprise |
| 16:26 | Continuous monitoring and real-time dashboards |
| 17:29 | Sales and marketing use cases |
| 18:00 | Product analytics, done internally |
| 20:01 | Tracking engineering velocity from GitHub data |
| 20:39 | The organizational side of adoption |
| 21:28 | Letting people try any tool |
| 22:19 | Why it has to be 1.5 times better before anyone switches |
| 22:43 | Culture change comes from the conversation, not the tool |
| 23:35 | Three-month proofs of concept, and staying close to the vendor |
| 24:28 | Where AI changes the enterprise next |
| 25:47 | Guidance for executives early in adoption |
| 26:14 | Resources |
| 27:01 | Leadership skill: experimentation |
| 28:22 | The one thing to remember |
| 28:59 | Wrap-up |
In Chris’words
“Data work is meant to be accurate. It’s meant to be the source of truth. If an AI model hallucinates 1 percent of the time, you can’t make decisions off of that.”
— Chris Parmer (01:58)
“You end up with companies that have thousands or tens of thousands of dashboards, because there’s a new one created for every single question.”
— Chris Parmer (07:04)
“A lot of product experiences show you a blank slate, and it’s what do you want? You can have anything. That maybe is freeing, but it’s also a little paralyzing.”
— Chris Parmer (10:10)
“I have this theory that it’s gotta be at least one and a half times more productive for people to really jump ship.”
— Chris Parmer (21:40)
“Being very comfortable with throwing away work and throwing away approaches, and not getting tied too early to things.”
— Chris Parmer (27:20)
Resources
Chris Parmer and Plotly
• Plotly: plotly.com. The graphing libraries and Dash, the framework for building analytic and data applications
• The Plotly blog: plotly.com/blog. Where Chris writes frequently on building products on top of AI
• Plotly on LinkedIn: linkedin.com/company/plotly
What Chris reads
• Simon Willison’s blog: simonwillison.net. He reads it almost every day for what is coming next on the technology side
Tools and technologies discussed
• Dash: dash.plotly.com. Plotly’s framework for data applications, and the thing stakeholders used to wait on a data scientist to build
• Jupyter: jupyter.org. Named as a long-standing home for exploratory analysis in code
• Apache Spark: spark.apache.org. Another example of analytics done as code
• Excel, Power BI, Tableau: The point-and-click tools he contrasts with the code-based approach
Related AI Realized episodes and events
• Agentic AI and Revenue Work: What Actually Pays Off: Christopher Penn of Trust Insights on agentic AI, measurement, and proving lift.
• Connecting AI Agents to Live Enterprise Data: Deepti Srivastava of Snow Leopard on the gap between AI agents and dependable structured data.
• Artifact-Scoped Agents: Stop Mimicking Job Titles: Chris Butler of GitHub on scoping agents to what they produce, and where the human belongs.
Frequently Asked Questions
-
Chris Parmer’s answer is that the objection changes shape rather than disappearing. Data work is meant to be the source of truth, and a model that hallucinates one percent of the time cannot support decisions. What changed in the last year is that AI became remarkably good at writing code, and Plotly’s customers already do analytics by writing Python. When AI produces code rather than an answer, the work becomes inspectable, which is a different proposition from trusting an assertion.
-
Doing data analysis by writing code rather than clicking through a business intelligence tool. Parmer points out this is well established rather than novel: quantitative finance, pharmaceutical bioinformatics, and classical engineering disciplines have worked this way for years, using Python, Jupyter notebooks, Spark, and SQL. Python has become the standard language for advanced analysis because it offers more flexibility and power than point-and-click tools.
-
Because a new one is built for every question, and only one trained person can build them. Parmer describes the pattern: a dedicated person is trained on the BI tool, everyone else has to ask them, and stakeholders do not have time to learn the interface themselves. He also challenges the framing that code is the hard thing to learn, noting that sophisticated BI tools have their own steep learning curves.
-
The stakeholders who were previously dependent on data scientists. Parmer sees it concentrated where data is core to decisions and operations: quantitative finance, trading firms, portfolio management, and also energy and utilities. The people with Python skills had become the bottleneck on those teams, and this frees them while letting the stakeholders analyze the data themselves.
-
Parmer’s rule of thumb is at least one and a half times more productive. His reasoning is that a 10 or 20 percent improvement never feels like enough to justify the learning curve, which is why past technology switches stalled. Much of the current wave clears that bar comfortably: he describes work moving from weeks to days and days to hours.
-
Parmer’s account of what worked at Plotly is to let anyone try any tool, especially early. Many of these products have consumer versions, so people develop their own preferences and the enterprise versions can follow. They run Slack channels where people share tips. His observation is that the cultural change comes from the conversation between people about what they are finding, more than from any individual adopting a tool.
-
Parmer runs dedicated three-month proofs of concept with customers, and stresses closeness to the vendor. His reasoning is that the technology is new enough that the vendor holds most of the accumulated practice while remaining eager to learn from customers. You may need new ways to audit the AI, or find it not returning what you expect, and that feedback loop improves both the product and the relationship quickly.
-
Parmer expects roles and responsibilities to stay recognisable while the way they are performed changes fundamentally. His specific prediction is a shift toward monitoring and multitasking, with AI systems working in the background and individuals taking on more of a managing, steering, and orchestrating role. He describes this as how his own work has already evolved over the past year.
-
[00:42] Christina Ellwood: Welcome to AI Realized, the podcast for enterprise executives leading AI deployments. From tackling security, data, and operational challenges to navigating organizational transformation, AI deployment offers a unique opportunity to redesign organizations from the inside out. I'm Christina Elwood, your host for today's episode, and we're talking today with Chris Parmer, the co-founder of Plotly. In this episode, we're gonna talk about ex- accurate data analytics using AI, and Chris is a particularly valuable person for us to talk with about that because besides being the co-founder of Plotly, he brings a rare mix of perspectives, having been trained as an electrical engineer and as a self-professed data nerd. Chris, welcome to AI Realized.
[01:31] Chris Parmer: Thank you. Great to be here.
[01:33] Christina Ellwood: It's lovely to have you, and I'm interested in talking about this topic because I too have a bit of a data background, and have, I think, somewhat of the misconception that many people have that using AI for data analytics is a bit of an oxymoron. You wanna talk a bit about how do you do that accurately when you're having, yeah, using a probabilistic system to do something that I think of as being deterministic?
[01:58] Chris Parmer: Totally. It's such a good question, and I think, you know, the first few years that we were working with AI, we noticed how much it hallucinates, right? And data work is meant to be accurate. It's meant to be the source of truth, right? You're not supposed to-- if it, a AI model hallucinates 1% of the time, you just, you can't make decisions off of that. Now, what's really changed lately is in the last year, AI became remarkably good at writing code. And for us at Plotly, and for a long time, our primary customers are doing data analytics through code. So they're writing Python, primarily code to do data analytics, rather than dealing with point-and-click tools like Excel or Power BI or Tableau. And by working in Python, they get a lot more flexibility and power, uh, and it's become kind of the standard language for advanced data analytics. So the approach that we take for doing data analytics with AI that we've taken in our latest product, Plotly Studio, is we don't have the AI analyze the data directly. We don't send-- If you have an Excel file or a CSV file with 100,000 rows, we don't send those 100,000 rows to the LLM directly and ask it questions. Instead, we take a schema of that data, and then we ask the LLM to write code that would analyze the data through that code according to the person's question. So if the person asks, "Okay, what's the most common value in this data set?" The LLM won't look through the data set directly. The LLM will write five lines of Python code that's very standard and very unlikely to get wrong, and it's also verifiable. You can see the code that it's writing and audit it, just like you would your own code or a colleague's code. It'll write those five lines of code to find the most common value in that data set, and then run that code, and then show you the result. So we're almost using AI less as that sort of magical oracle to tell you the answer, and more as just this tool that will help you write remarkable code remarkably quickly.
[04:07] Christina Ellwood: That makes so much sense to me. This whole idea of using code for data analytics, I think deserves a double-click as well. So is that primarily done in situations, Chris, where you're trying to do data analytics within a software system, like a SAS system, for example. You wanna show the analytics of use or something like that, and s- you've got this code that is generating the UI, it's doing all of the workflows and things like that, and you wanna have analytics within that?
[04:42] Chris Parmer: Mm-hmm. It's used for that, but also as a means to an end just for day-to-day data analytics or data science work, exploratory data analytics. If you're a, if you're a data analyst and you need to do more sophisticated data analytics, like some data science methods, some statistical methods, then you're doing it primarily in code. And so for a long time, this was just done in industries like classical engineering, electrical engineering, mechanical engineering, physics, and then for, in finance, quantitative finance has been doing this in code for a long time. Pharmaceutical bioinformatics, writing code, working with things like Jupyter Notebooks to do anything from exploratory data analytics to more sophisticated modeling on top of that. And even tools like Spark that you might hear about are all doing data analytics as code in this way. Even SQL, I guess writing SQL queries is what I consider data analytics as code as well.
[05:41] Christina Ellwood: In many ways it is as well. Okay, so that's very helpful, and I'm just, I'm mindful of that many of our executives that are deploying AI are not building products, but they are using it for their own internal operations and workflows. So would you advise them to take a similar approach, and rather than use a BI tool, to have someone on the team who knows how to do analytics, to use coding as their method for doing their data exploration? Because in many ways, this is data that has insights that we don't really know what we're looking for yet.
[06:17] Chris Parmer: Mm-hmm. Yeah, absolutely. And I think that's been the career path for many folks who are professional data analysts within large organizations to level up. They might take a six-week or a 12-week Python bootcamp or data science bootcamp to learn Jupyter notebooks and Python and, and writing SQL to, to make their data analytics more powerful and more flexible, and be able to automate it easier. And that's the way the industry's been going for a long time. But now we have this massive shift where you don't need to take a six-week bootcamp anymore because AI can help you write that code, which is, which
[06:51] Christina Ellwood: is- So when Vinny asked that question is I was thinking, like democratizing many other things- Yeah like product development and other areas, this is really democratizing who can do data science, isn't it?
[07:04] Chris Parmer: Yes. Yeah, absolutely. And it allows you also... For a long time, we've been thinking that, okay, code is a thing that's hard to learn. But even sophisticated BI tools that have complex user interfaces, those have a learning curve themselves. And so we see a lot of organizations where there's the dedicated person that was trained in using the BI tool, and if you have a question, you've got to ask that person to make the dashboard for you. And you end up with companies that have thousands or tens of thousands of dashboards 'cause there's a new one created for every single question that an executive or a stakeholder or a customer might ask. And that's because those folks don't have the time to go into a BI tool and click around and learn it and all of that. But now that you can do this with natural language, those stakeholders can just ask the question directly themselves. And the underlying technology that I think all data analytics will be built on in the future is they'll ask a question, and then code will get written that will be the data analytics code that would answer that question, and then they'll see the result. And that's the technology we've been building our latest product off of, and really our hypothesis, Plotly.
[08:14] Christina Ellwood: I gotta tell you, I was one of those executives asking for the dashboard from the BI person- Mm-hmm ... and always needing another cut of the data or another type of question answered or whatever. So I really relate to this, this challenge, and I love the Plotly product because for the first time, I can take my data files and I can do the analysis on them myself, and all of my what if questions can be answered easily. And what, one of the things I like about the, the sort of- initial experience is that y- you automatically get these predefined tables. So I didn't have to figure out what table I was looking for. I feed it my data, it shows me all these representations, these graphs and tables and so forth, and from there I can decide, "Oh, I want to know a different cut," or whatever. And even though all the types of graphs are not what I'm, I need, it doesn't matter. I just ignore the ones I don't need. So it isn't, it doesn't get in my way. It actually helps me way more than I ever thought it would- Mm-hmm ... to have it be pre-structured for me in that way. Mm-hmm. I find when I show it to people, that's one of the things that really lights them up, is they're like, "Wow! Yes. All of these different cuts of this. This is amazing. This is like magic." And when you go to, to take the next step and use the natural language to, to change what you're looking, looking at, as long as it's a filter, it's really easy. But if you need to do a graph that isn't automatically generated for you, or you need to really change the way the graph is structured, there is a little bit of a learning curve there to learn how to do that. Are you finding that, am I typical- Yeah ... in your, of your user who's, wh- business user who isn't a technologist running into that challenge? Or is that- Mm-hmm. Yeah? Okay.
[10:10] Chris Parmer: Yeah, yeah. Totally. Yeah. There's this learning curve around prompting, right, and it's, was called prompt engineering for a while. And I really wanna get the product to this point where you feel like you don't need to l- know all these secret tricks about how to structure your language and what you want, and we're getting there, so that you can more easily express your queries and get a really good answer out of the box. But, and I think your point too about just seeing some things immediately is enormously helpful. A lot of product experiences out there just show you a blank slate, and it's what do you want? You can have anything. And that, like maybe is freeing, but it's also a little paralyzing, too. And so a lot of, and I think with data e- especially, there's a lot of different ways to look at it, and now that we can just auto-generate it, we like to present users with lots of different views that are semantically interesting to them about their data. And there might be an experience where you actually, you're doing very little prompting yourself, and you're just clicking through the suggestions that we're auto-generating for you and giving you that inspiration. Because you might not have the types of questions that you wanna ask or the ways you answer them at the tip of your fingertips. I've found that even myself using it, doing data viz for 15 years now, I'm presented with really interesting suggestions, ones that, that I wouldn't have come up myself
[11:31] Christina Ellwood: Yeah, I like that too. I also think you've done a nice job on that UI with breaking, so that people don't have to write a perfect prompt. There, there are little windows that you can put the portions of the prompt- Mm-hmm ... so the prompt ends up being very well structured because- Mm-hmm ... you've pre-structured these little fields so that people are not having to know the proper way to do the structuring. Yeah. So I, I think that's, that was a wise move on your part, too. Now, I know you, Plotly has a very large open source community because you are an open source core, and you also have an enterprise version of the product. And I've had the pleasure of attending some of your local, you're in San Francisco, I'm in San Francis- been to some of your local community events and met some of your users. I'm really curious how it's being received, how Plotly Studio, the AI version of your product, is being received in the open source community, and is there a difference in how it's being received in the enterprise community?
[12:31] Chris Parmer: Yeah, it's a great question. We found that when we released it, suddenly this new audience became
[12:39] Christina Ellwood: delighted by our technologies. This audience that I think aspired
[12:43] Chris Parmer: to do kind of code-based analytics before, was on the edge of the community or getting in the, into the community a little bit, and then were suddenly able to create things that would've otherwise taken them weeks to create immediately. We have so many folks in our community, and in, and this is shared across our enterprise customers as well, where they're very data literate and they are beginner code literate. And- They are now able to
[13:12] Christina Ellwood: focus more on their data literacy
[13:15] Chris Parmer: in exploring their data and working with the product, knowing that it's fully backed by all the flexibility that they desired as they were starting to get interested in code, but without needing to spend all of that time writing the code or learning it. And there's been some reticence, too. I think there's folks that are working in code in our community that really love to do, and prompting is almost like learning a new programming language. I've given some talks where I talk about, uh, we're coding in English now. And so I think there's, for some folks, there's maybe a little bit of grief of losing that craft or thinking they might lose it. I personally have found that a lot of the skills and the ways of thinking as being a software engineer translate to still prompting. I personally find it still to be a pretty creative experience. And then among our enterprises, every enterprise we talk to, AI, incorporating AI is, like, in their top three company goals. It's just everywhere, right? And there's a lot of experimentation that we're seeing that's happening, and it still feels like we're in that era. And we get a lot of the same questions around, okay, how do you trust it? How do you verify it? The point you brought up earlier about structuring the prompts is a good one because we get a lot of questions about, okay, how do you reproduce it? If in a typical chat conversation it might be hundreds of messages long, and if you have a second person that wants to audit it, you know, they're not gonna wanna read through your hundred message long chat scram... chat transcript. So that's some of the things we're doing in the product around structuring the prompts really nicely so you can reproduce things easily and audit them and all of those types of things.
[14:52] Christina Ellwood: And that's interesting. Who in the enterprise is adopting the AI version of the product most? Yeah. Like, where, where are the hotspots in the
[15:02] Chris Parmer: enterprise? Yeah. It's really exciting because it's, it's the stakeholders or c- the consumers of the folks that were Previously reliant on the data scientists to build them a Dash app or which is our other technology for building what we call analytic or data applications. And now they are able to go in and create things themselves. And so this is still in industries where data is core to the decision-making and operations. So for us, that means in quantitative finance, trading firms, portfolio management, where you're running models, looking at data every single day to make trading decisions, highly technical. And, and a lot of the folks that were, that had the sort of Python chops were the bottleneck in a lot of those teams. And so now they're able to be freed up because the stakeholders can visualize and analyze the data themselves in a more sophisticated way. Energy, utilities, also just tons of data with IoT devices out in the field measuring pipelines or measuring the electric grid or measuring equipment out in the field, and just constant data analysis done on all that data that's coming in to see if things are failing or working properly or not. And so a lot of the managers of these teams that were previously reliant on phoning up their engineer to do the data analysis are able to be freed up to do it themselves, which is really exciting.
[16:26] Christina Ellwood: Do you have anybody who's do- using it in a continuous monitoring environment? I, because a lot of those, what you just described, are continuous monitoring pieces. Are people pipelining their data into Plotly Studio and using it as a-- 'cause your graphs are all interactive. I could see them being- Right ... dynamic as well as interactive.
[16:44] Chris Parmer: Totally. Yeah, we have so many of our existing customers that have set this stuff up with our core technology, with Dash and Plotly, and building these real-time dashboards. And so now doing data exports from that into Plotly Studio when they need to do a deeper diagnosis. And so there's a bit of a mix of you, you-- there's still a place for building sort of the real-time dashboard that has a long shelf life, and that's done by our customers with our, our Dash libraries and our Plotly libraries, and these things are maintained and live up on a TV screen for years. But those just show a fixed number of views, right? So then when an issue comes up, you want to be able to take that data out and explore it really freely and flexibly, and that's where Plotly Studio comes in. So it complements a lot of the existing technology that we've provided really well.
[17:29] Christina Ellwood: And as a, on the sales and marketing side of the house, the out- the outwardly facing side of the house, there's many of those use cases where you wanna look at the data in a more flexible way, but you don't actually need it to be a continuous live feed. So it's, it's a little between the two of those. Do you see any of those cases maybe in finance or operations or product teams or things like that where people are using the product for non-mission critical but complex data analytics?
[18:00] Chris Parmer: Totally. One of my favorite examples is actually internally. We do a lot of product analytics, see at what points in the product are people falling off or not using it or whatever, and it's pretty complicated stuff. You might A/B test a feature that you release or things like that, and tools out there like Mixed Panel or Amplitude or Google Analytics have been the standard for a long time. And at Plotly, I exported all of that data that we had and export it every hour and use AI and Studio to do all my analytics now. So instead of going into one of those tools and using the graphical user interface, which I always found to be like a little bit inflexible, huge learning curve to understand how it works, I'm just typing every day I'm opening it up and I'm asking a new query. Okay, what's the window size that people use most, most frequently? Or what's the startup time difference on Mac versus Windows of the desktop application? Or all these types of questions that might take 30 minutes to create the chart by point and clicking, figuring it out. I was pretty reliant on other members of my team to create a lot of those things. I've asked just hundreds of questions over the last months. Any curiosity that I have is just immediately there, and you can, you can just type it and ask and get the answer. Isn't that awesome? It's remarkable. It's just remarkable. Yeah.
[19:21] Christina Ellwood: Yeah. Data, uh, we have this problem in, in SEO and other types of analytics on the marketing side too where it's just awful to try and get your reports, and the tools that are out there are- You know, are just, let's just say they're l- lacking in flexibility and in, in usability. So it's just fabulous when you get rid of that obstacle. I don't know if that... Um, and it's interesting that you're finding that on the product side. This is for product-led growth you're referring to, of course. In the product development side, are you using it there too and looking at bugs and frequency of bugs or areas of bugs or whatever? Using it there too?
[20:01] Chris Parmer: Yeah. Yeah. We've gotten some exports of our GitHub data just to track velocity and things like that. And then in the product development itself of software engineering, we're all using AI as well to accelerate the development itself. And if-- And so you start seeing, wow, it's, wow, the whole company starts to feel like we're all working through natural language. We're writing natural language to write the code. We're writing natural language to analyze the data that is tracking events that was written in the code. It starts to feel like this new universal interface instead of a mouse and a point and clicking, you're working out of a terminal all day.
[20:39] Christina Ellwood: Yeah, no, that's really good. And one of the things that has persisted as a challenge for enterprises adopting AI since long before we started AI Realized more than a year ago, is the organizational change, the challenge of people changing how they do things. And humans are reluctant to change. But- Yeah ... since you mentioned that you have it so pervasively used inside your organization, do you have some advice for the e- executives on the call about how to help people get over the hump and how to lead your team in such a way th- that these, this approach is culturally designed in instead of layered on top?
[21:28] Chris Parmer: Yeah. I think what worked well for us at Plotly was really encouraging anyone to try out any tool
[21:38] Christina Ellwood: in, especially in the beginning.
[21:40] Chris Parmer: Right. Because a lot of these tools are available on the consumer side as well, and so people start developing their own preferences about what they might like to use, and then being able to bring in the enterprise versions of those. And we have Slack channels and stuff where everyone is sharing their favorite tips and tricks with each other. I think there's definitely a learning curve, and I think what we've seen in our organization is some people will start picking it up, and then we'll see the productivity improvements. And I have this theory that it's gotta be at least one and a half times more productive for people to really jump ship.
[22:19] Christina Ellwood: And I think a lot of reluctance in us switching technologies in the past is 'cause it might be 10% more productive or 20% more productive,
[22:25] Chris Parmer: and that's, that doesn't always feel like enough to get over the learning curve. But with a lot of these tools, it's just night and day. It's weeks, it's weeks to days, days to hours. And so once you can see your colleagues start doing it, at least at Plotly, that's when things have started really taking off.
[22:43] Christina Ellwood: Yeah. It strikes me that it's the communication between people that's leading to the sort of cultural change. Mm-hmm. Rather than the individual adopting it, it's the conversation about the adoption or about the tool or about the- Yeah benefit or whatever that's leading to that change. And I think that's, it, that's especially challenging for organizations that are remote or hybrid because there's less water cooler talk kind of thing. Slack does seem like a good way to close that gap, but not everyone, every culture i- is oriented to a digital communication platform like that is universal across the company. They have different tools for different areas of the company. H- do, have you observed other ways that people have been able to create that cultural shift that are not dependent on a digital channel?
[23:35] Chris Parmer: Mm-hmm. Yeah. Within, at least with the customers that we've worked with, because this is so new, just starting out with these really dedicated three-month proof of concepts. We do that a lot with our customers. To come in, 'cause especially with AI technology- Qualia as the vendor, we know a lot of the best practices already. And we're very eager to learn more from our customers, and they, they're very eager to get the tips and tricks. And because it's such a new technology, having those really close relationships with the vendors I think is really helpful. And you might need new types of ways to audit the AI, or you might see that the AI isn't returning the answer that you'd expect. But because things are changing so quickly, that feedback between yourself and the vendor can really improve the product and the relationship really quickly, and yeah.
[24:28] Christina Ellwood: That's a good, that's a good, a good observation and takeaway. So where do you see AI transforming the enterprise in the coming years? I think that
[24:42] Chris Parmer: the way that a lot of the roles and responsibilities that we have will remain the same, but the way that we are doing them will be fundamentally different. The products that we're adopting to do, or the productivity tools that we're doing will look fundamentally different. The user interfaces and everything will be very different, and I think there's no shortage of work to be done, so I think we will just, we'll be getting a lot more done. But the way we do it will be very different, in that a lot of incumbent tools around how we work will look different because we'll be, we'll be I think more monitoring and multitasking many things that are happening at once, where the AI systems are plugging away in the background, and as individuals, we're taking on a little bit more of a managing, steering, orchestrating kind of role. That's how I've seen my own work really evolve over the last year, and I, I think that will make its way into the role of all enterprises as they become more comfortable and as the technology stabilizes.
[25:47] Christina Ellwood: What's your guidance for executives who are early in their AI adoption journey?
[25:55] Chris Parmer: I think for them to use it themselves as well. I think once you really start to use it yourself and see the productivity, it becomes so much more exciting and clear where it should be used throughout the organization.
[26:14] Christina Ellwood: Do you have any resources that you recommend for listeners who wanna learn more about you or about Plotly or about AI?
[26:21] Chris Parmer: Our blog is really good. I write in it very frequently with lay's thoughts about building products on top of this and stuff. So that's just, you can find that right at plotly.com/blog. And then other newsletters and things that I read, one of the execs at Anthropic has a Substack that is very good. I'll get-- We can get the link for folks after this. Very forward-thinking, very kind of geo- geopolitical and interesting. And then on the technology side, I read Simon Willison's blog almost every day. It's a little more cutting edge on what's coming out in the technology and, uh, yeah, more on the cutting edge.
[27:01] Christina Ellwood: Yeah, those are great resources. Thank you for that. And as a AI leader yourself in an executive in a technology company, what skill, leadership skill do you find most valuable when leading an AI initiative?
[27:20] Chris Parmer: I think experimentation, especially in this early era, being able to try multiple things at once and throw away work, and being very comfortable with throwing away work and throwing away approaches and not getting tied too early to things, especially as the technology is changing so quickly. And especially now as evaluating technologies is easier than ever because of the ability to use AI to do that work.
[27:46] Christina Ellwood: Do you have a structured way you do experiments?
[27:50] Chris Parmer: No, I, I don't. I, yeah, I follow what feels inspiring or exciting. I try to have, juggle a few different things at once. I have some R&D experiments that I might be thinking about or running for weeks at a time. I like to have a few different ideas always top of mind or in the back of my mind at once, and pick them up and shuffle them, and share often with the team about the experiments. And as a technologist, like really building something that you can demo, and demoing it often internally to the team.
[28:22] Christina Ellwood: Yeah. That's a, that's always a good strategy, right? It's a more scientific approach than an engineering approach. If our li- listeners remember just one thing from you today, what should it be and why?
[28:34] Chris Parmer: That AI is gonna completely change how we do data analytics, and, and the concerns that we have about hallucinating aren't, aren't as valid with this new approach of analytics as code, and AI writing code to do analytics. So it's gonna completely change the whole business intelligence and analytics market. The way we do, the products that we use in the next few years are gonna be completely different than what we use today.
[28:59] Christina Ellwood: So thank you, Chris Parmer, co-founder of Plotly, for joining me today on AI Realized.
[29:06] Chris Parmer: Great to be here. Thank you.