Smaller Models, Bigger Wins: Verify Before You Answer

AI AdoptionLLMs
Written By Christina Ellwood

Episode Summary

The industry has been conditioned to expect that better AI means more GPUs, more power, and more memory. Jason Williamson, CEO of MythWorx, is building in the opposite direction: domain-specific expert systems at a fraction of the footprint, which verify an answer before producing it rather than generating one probabilistically and asking you to check. He is careful that this is not a replacement for large language models but a complement, and he is specific about where it fits: engineering, physics, math, and the compute-constrained places where a rack of GPUs is not an option.

Key takeaways

  • The approach inverts the usual order. Rather than producing a probable answer and asking what you think, it verifies first: equations balanced, units consistent, simulators hitting their targets

  • It uses neuro-symbolic ideas plus quantum-inspired math, assembled as a mixture of solvers. The closest familiar analogy is a mixture of experts in the LLM world

  • Williamson is explicit that this is not a replacement for LLMs. He calls it a yes-and, aimed at domains where a wrong answer is worse than no answer: engineering, physics, construction, math

  • Three deployment cases: organizations that do not want to spend millions when they do not have to, edge devices like vehicles and robotics that currently need GPUs onboard, and places constrained by compute, power, budget, or heat

  • Most enterprise models already running are deterministic. Risk models, stress tests, algorithmic trading, engine simulation, supply chain prediction. The use cases are deterministic; the models are the mismatch

  • His pitch to CIOs is about position, not technology. Most are not driving innovation, they are reacting to what marketing and other functions bring them, the way public cloud arrived by credit card

About Jason Williamson

Jason Williamson is CEO of MythWorx, which builds domain-specific deterministic AI systems that verify results before producing them, at a fraction of the compute footprint of a large language model. He previously ran startups, venture, and research at Oracle, and is a United States Marine Corps veteran. He has also led mission-based and faith-based organizations, including work against human trafficking, and he spoke at the AI Realized Summit before this conversation.

 

In this episode

00:42 Welcome and guest introduction
02:41 Why more GPUs is not the only path
06:25 Where no answer beats a possibly wrong answer
06:54 Neuro-symbolic ideas and quantum-inspired math
07:48 Verifying first instead of guessing
08:53 A mixture of solvers
09:23 How you would actually buy and run it
11:01 Why the cost profile is a fraction
12:56 Why one size does not fit all
13:34 Three deployment cases
14:28 Mini data centers, power, and heat
15:17 Satellites and compute-constrained environments
16:25 What this means for a CIO or CAIO
17:25 Why most CIOs are not in the innovator seat
21:27 Getting adoption top-down and bottom-up at once
23:46 Finding a deterministic use case
23:57 Why most enterprise models already are deterministic
24:37 The next two years, and the end of experimentation
26:22 Under-resourced by design
26:56 Resources
27:49 The one thing to remember
28:52 Leadership: human connection
31:54 Wrap-up

In Jason’s words

“We’re not trying to find the answer in a probabilistic way and then saying, "Hey, what do you think of this?" We verify everything first, and then go on top of that.”

— Jason Williamson   (07:48)

“This is not a one or the other kind of scenario. We think we are a yes-and when it comes to LLMs.”

— Jason Williamson   (02:41)

“Most of the models that are being executed today are deterministic models.”

— Jason Williamson   (23:57)

“If you want to get different results, you have to do things differently. And right now, to do things differently isn’t more power, more chips, more data centers. It is do more with less.”

— Jason Williamson   (27:49)

“Safra said, "If everyone is telling you you’re crazy, you might be onto something."”

— Jason Williamson   (27:49)

 

Resources

Jason Williamson and MythWorx

Where he points listeners

  • Intel neuromorphic computing: intel.com. His recommended starting point for understanding this class of approach

  • Cerebras: cerebras.ai. Named for small compute at the edge

Concepts discussed

  • Neuro-symbolic AI: The family of approaches MythWorx builds on, combining neural and symbolic methods

  • Mixture of experts: The LLM-world analogy he offers for his mixture of solvers architecture

  • Satisfiability: Used in the verification step rather than in generating the answer

Related AI Realized episodes and events

 

Frequently Asked Questions

  • Jason Williamson describes an approach that verifies before it answers, rather than producing a probable answer for a human to check. In practice that means confirming an equation balances, that units are consistent, and that simulators hit their targets, then building on top of that verified base. He is explicit that this is not a replacement for large language models but a complement, aimed at domains where a wrong answer is worse than no answer at all.

  • In engineering, physics, construction, and math, where correctness is not negotiable. Williamson’s broader point is that most models already running in enterprises are deterministic in nature: risk models, monetary stress tests, algorithmic trading, engine simulation, supply chain prediction. The use cases are deterministic. The mismatch is in the model being applied to them.

  • Neuro-symbolic ideas, plus what Williamson calls quantum-inspired math, using tensors that borrow from many-body physics rather than any quantum computer. The system is a mixture of solvers, which he offers as roughly analogous to mixture of experts in the LLM world. Learning happens across many small event-driven systems with reward modulation he compares to dopamine.

  • On-premises, in a public cloud, or hosted, and at a fraction of the usual cost. Williamson’s comparison is a car or chemical manufacturer spending millions a year training a domain-specific model, then adding their own data, then standing up an inference engine on top. His value claim is dropping that inference onto low-energy chips instead of racks.

  • Williamson names three. Organizations that do not want to spend millions when they do not have to, which covers engineering firms, manufacturers, chemical companies, and law firms. Edge devices that currently need GPUs onboard, such as a vehicle deciding whether an obstacle is a speed bump or debris. And environments constrained by compute, power, budget, or heat, including satellites, where a rack is not an option.

  • Often not first, because most CIOs are not in the innovator seat. Jason Williamson argues they got into technology to do interesting work, but by the time they reach the role the job is keeping the lights on and not getting called. His example is public cloud, which arrived because a marketing person put AWS on a credit card for an application IT was not delivering, leaving the company to formalize it afterwards.

  • The same way the current ones did, top-down and bottom-up at once. Williamson’s account of how AI arrived: a couple of pilots run by enthusiastic engineers, plus business sponsors with line of sight to revenue who could see what they would get if it worked. He is also clear you would not switch everything, since a solver built for mathematics is not the tool for open-ended generation.

  • Expect a bumpy, fuzzy front end with real disruption. Williamson expects uncertainty about the workforce to grow as adoption deepens, and cost pressure to intensify because compute prices are not falling and GPU access remains difficult. His summary of where buyers now are: experimentation is over, and the demand is to show returns much faster than the market has been delivering them.

 

  •  

    [15:28] Christina Ellwood: And the third is where you have compute constrained and you have high-value applications.

    [15:33] Jason Williamson: Right. Right.

    [15:36] Christina Ellwood: Are th- Are those, is that a correct assumption? That's

    [15:38] Jason Williamson: ex- You, you've hit that exactly on the head. So I think- All

    [15:41] Christina Ellwood: right ...

    [15:41] Jason Williamson: those are very large areas that we're focused on, and I think is super transformative.

    [15:47] Christina Ellwood: It is. It definitely is, and I think we're gonna see more and more of this, Jason. I think we're gonna see other types of models. I think we're gonna see other types of energy profiles, sizes. One of the things John Sfiocla, who spoke at summit as well, talked about is mobile is a coming, right? We are gonna have the AI in our mobile phones, and that is gonna change how we use our mobile phones and where we use our mobile phones, and obviously the size of a model you can put in a car is different than the size of the model you can put in a phone. It's, it seems, I don't know if your models can go that small,

    [16:24] Jason Williamson: but- Absolutely they can. Uh-huh.

    [16:25] Christina Ellwood: Okay. There you go. So maybe this is an answer to the phone conundrum as well. So bringing this down to the executives who are listening, if you were sitting in the CIO or CAIO, we don't have a whole lot of those, but you're sitting in their chair at a Fortune 500 company. So it's-- I find it really helpful to go to the biggest of the big because for our audience, even though most of our audience is not in that particular department, because they have every degree of complexity. They're global, they're multi-layer organizations, they have many lines of business, they're in multiple verticals, and they're, they're driven by access to capital. So these parameters make it useful for that reason. That's why I'm modeling it there. So if you're a CIO of a Fortune 500 company and you want to stay ahead and contribute to keeping your company competitive in an America-friendly way, what are two or three non-obvious decisions you would wanna make?

    [17:25] Jason Williamson: I think this sounds really non o-- This sounds obvious, but is actually quite non-obvious. I think CIOs are really focused today on, we got into this business, we as technologists got in this business to, like, do cool things and be cutting edge. And often, once someone reaches the level of the CIO, their life is just, "Let me keep the lights on and make sure no one calls me." And they often find themselves not in the innovator seat. I would argue that most CIOs are not the ones driving technology innovations within their company. They're ref- they're reacting to what marketing needs and this needs and that needs, right? Look at the emergence of public cloud was guided by IT because a marketing person slapped down a credit card and bought a bunch of AWS because there's a, an application they needed that they weren't getting internal. And now the company's forced to realize, "I gotta go public cloud." So I think the non-obvious thing is for a CIO to carve out a chunk of their time and embrace this notion of AI and understand how that's gonna fit within their organization. And I talk about this at the summit, too. It's understanding what your maturity model is and, like, understanding where you are within your organization. So step one is, like, embrace innovation. I know that sounds like all CIOs do that. I would argue most do not. I would say embrace that, that, that... Don't be afraid of it. Embrace it. And then step two would be understand where your company is on the maturity model of AI adoption. You could be at level zero, you could be at level five. You're probably not at five, but and it's understanding, like, how do I get, how do I get out of this, like, experimentation phase and understanding just like we did with virtualization, just like we did with cloud, just like every technology. This was the argument I made during the summit, is that AI adoption's not no different than when we went off the mainframe, and we went to distributed, we went to virtualization. Like, it's, there has to be va- there has to be an ROI on that. And so if you can get really good at articulating where the line of sight to revenue's gonna be for the organization, you can start not tr- just being a cost center reactionary, and instead be able to provide, like, you know, ways for the company to move forward. 'Cause at the end of the day, your business only is profitable with two levers. You only get two levers: lower cost or increased revenue, and AI has a chance to do both of those. So as a CIO, you need to start, like, re-engaging the language of finance for the purpose of driving EBITDA at your company. And AI can definitely do that, but being able to put guardrails and wrap words around where you are in maturity and be able to get that along, uh, is really good. And one of the reasons why we love what we do at Mythworks with the transparency is that you can often get through the very difficult compliance hurdles because we always see that blocker. And as soon as you invite legal in the room, everything stops. And, and in this world, it's especially true because there's a lot of- But in this space. And running ahead of that is very important. So with the way we do these things is we're not, we can't hallucinate. We're not doing probabilistic pre-predictions on language. We solve things or we don't, and here's how it happened, and it's transparent. So being able to adopt technologies and running ahead of the, of compliance is gonna help you get there. And then understanding, the last thing I'll say on that is understanding how to articulate what that risk looks like You know, 'cause risk is just a probability of something going bad and a cost associated to it. And so if you can figure out how to get through those humps, then you can start turning your, "I just need to keep the lights on and not get fired," to like, "We're gonna move this company forward because of technology." And I would say I've seen that in a lot of companies that have transformed. I'm an ex-Cap One guy. I was a part of the IPO way back in the day where it was a little regional bank in Virginia and, and became a large credit card issuer. And technology and the role of the CIO there, in fact, the current SO- CIO there is extremely influential in how a banking company moves forward with what they do technically. And looking at companies like that's a great case study in how we get these things done.

    [21:27] Christina Ellwood: Jason, thinking about what you guys are doing at Mythworks with your different type of AI model, it strikes me that taking what you just said about the CIO becoming more of the innovation voice maps nicely to that because they can say, "Well, this is a different AI model that is cheaper, lower energy, et cetera, et cetera, and deterministic, so we lower our risk." So we've lowered all of the things that I'm responsible for. Now we're gonna have to have the second part of this conversation, which I'd love for you to answer, which is how do we go-- how does the CIO go to the people in the organization who are currently using LLMs, who could be using this model, and explain to them why they should switch, and what is the process of switching? How do I get onto your model- Sure ... and off of the one that I'm on today, or the ones, 'cause they're all using ones.

    [22:26] Jason Williamson: Well, exactly. And I would argue that maybe you don't switch them all, 'cause we're not a generalist where, you know, if I'm gonna ask, I'm doing a, a, an equation in math and I wanna get a recipe for soup, we're not-- We could absolutely build that. We're just not. We're focused more on, on a mixture of solvers. But I-- Side note, inside of a year or two we'll be in that level for sure. But I think it's the same way that you gotten them to adopt the fir- the ones in the first place. And so I like to see it as like a top-down, bottom-up approach at the same time. And so how did this happen at the company? They probably had a couple pilots. You had some cool kids with stickers on their laptops who had great haircuts, who was like, "Oh, I'll go do it." And then you had some business sponsors that had, again, line of sight to revenue to understand if this thing works, I get this out of it. And so I feel like it, it's the same way that they got the LLM to adopt before. But as I- Pretty sure I mentioned this on the panel from the summit. It's a, it's an OB issue as much as a technical issue. It's an organizational behavior issue, otherwise. Because the fact there's LLMs there, that's like the hardest part. Like, just getting people over the hump that AI's, like, a good thing is probably the hardest battle. So if you can get them to go, "Hey, there's another way. Let's go try this now," then you've already done the hard sell, is the organizational behavior side of the house.

    [23:46] Christina Ellwood: Let me ask a naive question. Would I also be wise, as this fictitious CIO, to go find someone who has a use case that's deterministic by nature?

    [23:57] Jason Williamson: Feel like they all are. Like, if you're at a bank, you're trying to figure out risk, risk models. You're trying to do monetary pressure tests. You're trying to do algo tr- algo trading. You're trying to asset real- Those are all deterministic. If you're at a company that makes things and you gotta do simulations on your engines, you've gotta look at predictive analytics, prediction on your supply chain. We're sitting on... There's, I would say most of the models that are being executed today are deterministic models.

    [24:25] Christina Ellwood: Meaning they're deterministic use cases, but the models- deterministic use cases. Models are the problem. Exactly. Look at that. Right? You're correcting me. Yes, exactly. Yeah, yeah.

    [24:33] Jason Williamson: Most of it is not like, I need to create a new novel. Yeah. You know?

    [24:37] Christina Ellwood: Fair enough. So- Let's step back even further for a minute. Where do you see AI transforming the enterprise in the next two years?

    [24:49] Jason Williamson: I think we're still going to probably engage in a bumpy, fuzzy front end phase for the next two years. There's gonna be some disruption. As the more AI gets adopted, I think you'll find the more uncertainty that gets injected, meaning, what does this do to my workforce? How do I re-engage them? I can't just ... Do I fire them all? How do I ... So I think the enterprise is gonna go through a little bit of a crisis of, what do I do with this? I s- I think there's still gonna be cost pressures, because as the price of compute does not decline, and the cost of GPUs and the accessibility is, continues to be more difficult, there's gonna be price pressure to make sure that there's a return on investment quick, much quicker than we are. We've been e- We're out of experimentation. Yes, we get it. It does cool stuff. You need to show it to me now. And I've just been going through some infrastructure purchases, and it's really interesting out there right now. Price pressure's a pretty real thing in the infrastructure space right now. So I think in the enterprises they adopt, we're gonna see that, and then we'll see some uncertainty. I think it's a up to ... It's a strong ... That, that's not a headwind for us. That's a tailwind. So as the price pressures continue and availability and accessibility continues, 'cause the lack of big data centers are ... Like, that's good for me. Give me the small stuff. It's ... I was just talking to somebody just the other day. I was in the Marine Corps, and we pride ourselves on having less and being il- il-funded's not the right word, but like-

    [26:22] Christina Ellwood: Under resourced ... less,

    [26:23] Jason Williamson: under resourced, less gear. Minimum

    [26:25] Christina Ellwood: water. Not the extreme water.

    [26:27] Jason Williamson: That's right. I want the, I want least. And so for us, it's come on, let's go. We're, we don't have, we don't need it to get it going. So the future is going to be those things which for a select few of us in this space that are delivering technologies in a way that our resource constraint is a, is an upside for us to take some of that market share, just from a selfish perspective, but it's also good for the customer. And so I think that's our future

    [26:56] Christina Ellwood: What's your, what resources do you recommend to listeners who wanna learn more about what the work that you're doing and about Mythworx and your technology?

    [27:04] Jason Williamson: Yeah. So I think you can go look and see what Intel's been doing, some, like the chi- there are some chip manufacturers that are, like Intel is doing this, Cerebras is doing small compute on the edge. I think go just take a look and see what's happening there. There's some, a lot's being done in the academic space. There's a couple companies that are, I mean, a couple like us, that are out there doing this for real. So I, I would say start at Intel and, and go take a look at what they're doing in neuromorphic, and then just let your curiosity keep taking you away from there.

    [27:42] Christina Ellwood: If our listeners remember just one thing from our conversation today, what would you like it to be and why?

    [27:49] Jason Williamson: I think the one thing that I would like people to understand and take away is the way that we move... If you wanna do things differently, if, if you wanna get different results, you have to do things differently, right? And so right now, the, to do things differently isn't more power, more chips, more data centers. It is do more with less. And so that's what I want people to think about, like how can you do more with less? So if everyone is running that way, I think maybe we just, Sony needs to run the other way. Uh, and last, and, and I'm just remember something that Safra Catz said. Safra was now the chairman of Oracle. She was CEO there for a really long time, and she, uh, you mentioned this earlier, I ran startups and venture and research for them for a while, and Safra said, "If everyone, um, is telling you you're crazy, you might be onto something." And so I think that's what I'd love people to take away, like when they're engaged in this AI space. If they're telling you you're crazy 'cause everyone's running that way, then you might be onto something, 'cause we certainly think that's true.

    [28:52] Christina Ellwood: Okay. Gotcha. Your background is so diverse and interesting, Jason. You've been a leader, y- you, in mission-based organizations like your human trafficking and your faith-based work. You've been involved as a, your military leader. You've been a corporate enterprise leader. You're leading a startup at this point. I mean, if I've left anything out, please, I don't mean to diminish. It's a- No, please ... it's popping into mind. Yeah. As a leader in this era, in the AI era- Yes ... what's your, what is the skill or the leadership capability that you find most valuable in the AI rev- evol- the AI, AI era? The AI era

    [29:34] Jason Williamson: is the s- it's the, it's connection with people. I just recently had-- I'm not gonna tell you, like be good at prompt engineering and be good... Like, it's the human connection. One of the organizations that I'm involved with in the startup portfolio, we just had a off-site in Charlottesville, Virginia, which is where I live, just south of Washington, so we're spoiled here being in Appalachians. And it was a chance to, like these, all these people work remotely. A couple of them are in together, but that's just the way it is. And being able to have like human connection to look at vision and look, and why is the vision there, and to understand things about each other and understand what motivates them and what demotivates them, and getting on the same page. Like, that stuff is magic and it's gold. And as we move into this AI space, we're continuing to get pulled apart as people and as humans. We saw that accelerated in COVID, and so we have to, again, we gotta run against the grain a little bit and get back into connective tissue because- That is probably the single most thing a leader can do is really understanding the humans that are involved there. Not just like your DevOps and your technology and make sure you're hiring the smart- smartest people. Like, all those things are important, but it always comes down to the organizational behavior aspect of that, and the culture that you build, and the, the ability to navigate, like, adversity together and all those kinds of things. Absolutely. That is, as a leader, hands down, it's all about the people. That sounds kind of cliché, but it's true.

    [31:13] Christina Ellwood: I- it sounds like you feel like it's m- even more true in this era than in previous- That is- ... roles you've been in, 'cause you're- W- ... calling it out as this ... 'Cause obviously, that was important in other roles that

    [31:26] Jason Williamson: you were in. Always, yeah. Or in the military, in, in the mission field, and you're, like, going to, like, rescue kids and all that, like, people are important. But we're in a culture that is constantly driving us apart. We're in a culture that's constantly telling us the people that don't agree with you are evil, and they're We're constantly in a culture that is it ... Entropy is a thing. It's real, and it's real in humans. And so we have to go against that. And- Love it ... you're gonna ... Yeah. I could keep ta- We could have a whole session on that,

    [31:54] Christina Ellwood: so. We could indeed. And you are, you're a fascinating person and bring a lot of interesting insights to the party, so I hope we do have you back again soon. But for now, let me say thank you, Jason Williamson, CEO of Mythworks, for joining me today on AI Realized.

    [32:11] Jason Williamson: Thank you. Grateful for the time.

 
Listen to other episodes
 
 
 
Join AI Realized
 
 
 
Subscribe to the newsletter
 
 
 
Christina Ellwood
Previous

From Firefighting to Fire Prevention in IT Operations
Next

The Competitive Advantage No One Is Protecting