Steven Forth is Ibbaka’s Co-Founder, CEO, and Partner. Ibbaka is a strategic pricing advisory firm. He was CEO of LeveragePoint Innovations Inc., a SaaS business designed to help companies create and capture value.
In this episode, Steven discusses the challenges of pricing AI, primarily due to the limited availability of data. He emphasizes the importance of shortening the time required to build value models in order to easily establish pricing.
Podcast: Play in new window | Download
Why you have to check out today’s podcast:
- Discover the complaints and challenges associated with pricing AI
- Enhance transparency in establishing the price point with creating value models
- Find out the most recent advancements in AI pricing concerning language models and software, which greatly enhance productivity
“Once you have a good value model, coming up with pricing is maybe not trivial but it’s certainly much easier.”
– Steven Forth
Topics Covered:
01:15 – Discussing complaints about AI through the Value Models
07:57 – Steven’s added thoughts to Mark’s suggested solution to achieve pricing transparency [limitations in creating value models]
11:43 – The need for more trainings for language models used in pricing
14:25 – What is Copilot by Microsoft and what it is capable of doing that can help salespeople
17:04 – How is Copilot might disrupt the market especially Google Workspace user
17:50 – Survey of people’s willingness to pay $30 a user for Copilot [and whose group is more willing to pay more]
22:20 – How were the users segmented, are they all Copilot users, and finding out people’s reaction to AI
23:37 – The amazing capabilities of these two AIs
24:57 – What could happen if Mark Stiving will use AI to write his fourth book [The need for great prompt engineering]
28:12 – How are AIs going to be priced
29:59 – Microsoft Copilot as a ‘Will I’ question
Key Takeaways:
“If you can build a value model and validate a value model, then you can fairly easily, I believe, derive pricing from the value model.” – Steven Forth
“We should be able to customize a language model based on our training and integrate mathematical AI from a place such as Wolfram/Alpha to greatly scale up our ability to build value models so that we could legitimately do a thousand a year or so.” – Steven Forth
Resources/People Mentioned:
- Craig Zawada: https://impactpricing.com/podcast/ep95-defining-no-touch-and-self-serve-pricing-with-craig-zawada/
- PROS: https://pros.com/
- Vendavo: https://www.vendavo.com/
- Zilliant: https://www.zilliant.com/
- McKinsey: https://www.mckinsey.com/
- Wolfram/Alpha: https://www.wolframalpha.com/
- iGenius: https://www.igenius.ai/
- Hugging Face: https://huggingface.co/
- OpenAI: https://openai.com/
- Figma: https://www.figma.com/
- Notion: https://www.notion.so/product
Connect with Steven Forth:
- Email: [email protected]
- LinkedIn: https://www.linkedin.com/in/stevenforth/
Connect with Mark Stiving:
- LinkedIn: https://www.linkedin.com/in/stiving/
- Email: [email protected]
Full Interview Transcript
(Note: This transcript was created with an AI transcription service. Please forgive any transcription or grammatical errors. We probably sounded better in real life.)
Steven Forth
Once you have a good value model, coming up with pricing is maybe not trivial but it’s certainly much easier.
[Intro]
Mark Stiving
Welcome to Impact Pricing, the podcast where we discuss pricing, value, and the sneaky relationship between them. I’m Mark Stiving and our guest today is, once again, the only, Steven Forth. Here are three things you wanna know. Oh no, you know everything there is to know about Steven by now. Hey, Steven, let’s talk about AI today.
Steven Forth
Sounds good. It’s something we’ve been spending a lot of time on here at Ibbaka.
Mark Stiving
Yeah, I was gonna say, when you say we, you mean you and your team not me, because I almost ignore it.
Steven Forth
Me and my team. There are some really interesting things happening.
Mark Stiving
Okay. Can I give you a couple of my complaints or my thoughts about AI and you could tell me why I’m so wrong, and then I want to hear this data you’ve collected and all the really cool things that are going on in AIand how, and what it has to do with pricing.
Steven Forth
Yep.
Mark Stiving
So I have two big complaints with AI and pricing, maybe three. So one is the lack of data in B2B worlds, the lack of data on value. And I don’t think that’s a big deal. I mean, I just think that’s gonna come over time. So, am I wrong on that or is that okay to complain about?
Steven Forth
It’s okay to complain about it because by complaining about it, maybe we can get something to happen about it. So let’s complain.
Mark Stiving
Okay, got it. So that’s one. Number two, it feels to me that we don’t have a right answer to train it with. Last time you were on the podcast, we talked about how AI could learn how to play from scratch, just give me the rules. And because I know if I won or if I lost, I can figure out how to win the game eventually better than anybody. Now in pricing, I know if somebody bought or didn’t buy, but I don’t know if they paid the most they were willing to pay. So I don’t know the truth. How do we deal with that in AI?
Steven Forth
So I think that part of the difference here is that my approach to setting prices starts with a value model. So if you can build a value model and validate a value model, then you can fairly easily, I believe, derive pricing from the value model. So to me, the question starts a little bit earlier. Can we build value models using AI? I’d like to take a step back here though, because I think we need to talk about two different things.
Mark Stiving
Can you define a value model for everyone? And then we’ll take that step back. I just wanna make sure everybody’s on the same page.
Steven Forth
Yeah. So, in Ibbaka’s language, a value model is something like Tom Nagle’s Economic Value Estimation or EVE, and basically it’s a set of equations that estimates the economic value of a solution for a customer or possibly a customer segment. And then one can layer into that other models, for example, we’re doing a lot of work these days with sustainability models. Now, the sustainability equations can be an input into the value model, but the difference between the two is that the economic value model gives a dollar output, whereas the sustainability model gives an output like energy savings or reduced greenhouse gas emissions, or in the case of a couple of our customers reduced water consumption.
Mark Stiving
Got it. Okay. So now let’s take a step back.
Steven Forth
Sure. So, I think there are two different themes here. So, I differentiate them as pricing AI and AI pricing. So I tend to use pricing AI to mean, using AI in pricing and AI pricing is how we are going to price all of this new functionality and we hope new value that is being created by AI-based systems, and more of it, because work is on the latter on AI pricing than on pricing AI. But I do have some thoughts to share about both of these.
Mark Stiving
Oh, so you were gonna say, I’m complaining to the wrong guy?
Steven Forth
Not necessarily. Shall we start with pricing ai?
Mark Stiving
Sure.
Steven Forth
So first let’s remember the big heavy metal pricing systems such as PROS and Vendavo and Zilliant and Price Effects. They’ve been using AIs for a long time. They haven’t been using the new generation of transformers, which is the innovation that Google created in 2017 that is generating most of the buzz today. But AI is not new to pricing. And the complaint that I’ve had about AI in pricing is that they’re black boxes. So let’s just say that you used your favorite price optimization engine, and it’s suggested that the willingness to pay for some customers is 10% higher than they’re currently paying. So what are you gonna do? Are you gonna go and say to that buyer, hey, my fancy dandy AI says that you should be willing to pay 10% more, so I’m gonna increase your prices by 10%. Well, that’s not gonna get you very far, but one of the problems and challenges with AI is it can be hard to figure out why it’s coming up with the recommendations that it does. It’s a black box. And I think that is one of the big challenges for pricing AIs in B2B. It’s not just the lack of data, it’s the difficulty in explaining the systems, explaining how they work, why they work, and why they’re giving the recommendations that they do.
Mark Stiving
Yeah, that was gonna be my third complaint, by the way. But it turns out that I also asked Craig Zawata this morning. He had made a LinkedIn post this morning, and I’d asked him a question about that black box issue, and he said, they’re now using what they call transparent AI. Now, I don’t know anything else beyond that. I’m sure it’s worth looking into to see what they’re doing. Yeah. But they do recognize it’s a problem and they’re working to get around it.
Steven Forth
Yeah. So there’s a whole trend and DARPA has funded a whole set of research into what DARPA calls ExAI explainable AI. And it’s going to be pretty critical. So I’m glad that Craig and PROS are doing that. But unless the explanation helps you defend the new price point, it’s still not very useful. If the explanation says that we analyze price elasticity of demand based on a whole whack of factors and based on that analysis and based on our predictive models, we predicted that WTP would be higher. Okay, fine. That explains what they did. You and I can imagine how the algorithms could work. It still doesn’t help me defend my price to a buyer.
Mark Stiving
Okay, so I’m gonna make a huge leap all of a sudden, and I’m gonna say, gee, if we had a value model and we used AI to predict the, I guess I’m gonna use the word coefficient, but I’m not sure that’s the right word, but the importance or coefficient of each of the value drivers, then I would have much more transparency, and I would be able to understand why this specific buyer should be willing to pay me more. What would I do?
Steven Forth
Exactly. Yep. That’s exactly, thank you. I’ll slip you your $20 later, that’s exactly what we have. But the problem that we have, is that creating value models, today is a highly skilled performance. Yeah. And it is a bit of a performance. There’s a little bit of the magician happening in it. I was talking to a head of pricing at a major European company, and his comment was, look, if I were to adopt this approach, I would need to build a thousand value models to cover my portfolio and segments, who knows, maybe 3,000 and Ibbaka certainly does not have the ability to build a thousand value models, and I don’t think McKinsey does either. So I can’t go down this path. Fair enough. McKinsey maybe does have the ability to build a thousand or 3,000 value models in a short period of time and update them and keep them current and relevant. But it would cost you a non-trivial amount of money. And, he’s right. Ibbaka does not have that capacity. We could probably build at most 100k a year today. And even that might be stretching it. So this is a real problem for Ibbaka, how do we become more scalable? Part of the answer could well be that will be able to use some combination of generative AI and mathematical AIs to dramatically shorten the time that it takes to build value models. Now it’s going to be human in the loop. I don’t see that we are going to fully automate that in the next three years, but we should be able to customize a language model.
So transformers build things called language models and we should be able to customize a language model based on our training and integrate mathematical AI from a place such as Wolfram/Alpha to greatly scale up our ability to build value models so that we could legitimately do a thousand a year or more. So I think that in itself would be transformative and would have a huge impact on pricing. And as I said earlier, I mean, once you have a good value model coming up with pricing is maybe not trivial but it’s certainly much easier. That’s an example of pricing AI that is one step to remove from setting the end price. And I think we’re going to have, pricing is a very human thing, especially B2B pricing. So I think we are going to have humans in the loop for quite some time to come. But we’re going to need to leverage and take advantage of the power that pricing AIs have to get more personal, more customized pricing, to better understand our customers. So, the impact may be a bit more indirect than direct, at least for the next couple of years.
Mark Stiving
So what’s funny is, I could imagine given where AI is today and it’s actually pretty good, there’s a lot of complaints and a lot of mistakes it makes, but I could imagine ChatGPT or the equivalent doing an interview with customers to find out what their problems are, what their results they were looking for were, and being able to synthesize that into the right set of value drivers, especially if you did enough customers, you’d say, oh, this seems to be common across these customers. And so I could imagine that working well.
Steven Forth
And I think we will see that happening within the next 12 months. Everyone last week was talking about Microsoft’s pricing of copilot at $30, per user, and we should talk about that a bit later. But there was a far more important announcement, last week, which was that meta relay released LAMA 2.0, and made it available through hugging face. Weird name, right? But hugging face is emerging as one of the most important companies in the AI ecology. And LAMA is an open model so that we can train it and modify it fairly easily ourselves using some of the hugging face tools and environments. And I think that what we’re going to find is that large companies and innovative companies and companies like Ibbaka start training their own language models for use in pricing. Now, having said that, there’s still the issue that we use a lot of mathematics in pricing and language models have real shortcomings when it comes to math because they’re based on human understanding of language which does not always map well into mathematics. But there are solutions to that emerging as well from the people I mentioned earlier, Wolfram/Alpha, from iGenius which is another sort of a trending company. So this problem is going to be solved sooner rather than later.
Mark Stiving
Nice. Okay. So I should watch for Ibbaka to solve the problem of pricing AI.
Steven Forth
We hope we’re gonna be one of the companies making a major contribution to it. But there are lots of people starting to work on this and there are lots of different paths to a solution. It’s gonna be super exciting and fun to watch this over the next few years.
Mark Stiving
Excellent. So let’s jump to the other side and talk about AI pricing, which is how these AI companies are actually doing their pricing.
Steven Forth
Yeah. Which is a really interesting question. So let’s start with Microsoft, because it has got so much attention and hey, it’s announcement that it was pricing co-pilot at $30 per user, which is more expensive than many Microsoft 365 packages today. It had a tremendous impact.
Mark Stiving
Do the audience a favor and describe the Copilot. I got to see it at the PROS conference, but I wouldn’t have known that it existed had I not been there.
Steven Forth
Yeah. So copilot is, um, Microsoft’s second major implementation of GPT four. So GPT 4 is the world’s largest and most sophisticated large language model or transformer model as they’re also known. And it comes from a company called OpenAI, which is a company that Microsoft has invested billions of dollars in. And they may have ended up investing billions of dollars in a future competitor. But anyway, that’s one of the things that makes the world interesting. So OpenAI has one of the world’s largest and possibly by some metrics, the world’s best large language model today. There are other important ones from Google. We mentioned Meta’s LAMA earlier, Hugging Face, you can go and find about 280,000 different models. So this is a large and diverse space, but GPT isa really impressive accomplishment.
Mark Stiving
What, and so when I think about these, I gotta say you and I think very differently, Steven, you’re thinking about the technical details of what each one is doing, and I’m sitting here going, well, who cares? And what are we gonna do with it? And the guys that I saw, the Microsoft folks I saw doing the copilot demo were essentially showing how salespeople write emails, follow up with clients. It’s like the AI is doing most of the work for them, and it makes it really easy for them to do a salesperson’s job, or at least parts of the job that they despise. And it keeps them in tune and in touch with our customers. And it was very impressive.
Steven Forth
It is super impressive, and it does a lot more than that too. So it will with a few simple instructions, it will build a presentation for you in PowerPoint. With a few simple instructions it will reconfigure your Excel spreadsheet and give you all sorts of insights and do all sorts of analytics, including things that it might never have occurred to you to ask. So in addition to the word part of it, the writing words, it also works with spreadsheets. It works with PowerPoint, it works increasingly well with code. I don’t know about you, but coding macros in Excel is one of my least favorite things to do. With Copilot, I will almost never have to code another macro. I’ll tell the spreadsheet what I want it to do, and it will do it for me. This is a huge, huge productivity booster. And
Mark Stiving
Are you using Copilot today?
Steven Forth
I have the great misfortune of being a Google Workspace person, and if Google does not respond in the very near future, Ibbaka as a company may flip from Google to Microsoft which is what Microsoft wants, right? So the value of Microsoft, not just the $30 per user, although that’s billions of dollars, it’s the fact that they may well take market share from Google if Google does not respond very, very quickly. So, okay.
Mark Stiving
And so it feels to me like 30 bucks is nothing given what you just said it could do. How much time it’s gonna save me productivity. Heck, I’d spend that much just to have an extra hour to go, fly an airplane or something,
Steven Forth
Well, that’s how you, and I think, it’s been really interesting to read through the comments and reactions online, and to look at the survey data. So Ibbaka has over the last week done a Van Westendorp survey of people. We got about 250 responses, which is not huge, but it’s enough to get some insights. And it’s really interesting to see how divided the world is about this. So there are a set of people who think that paying $1 a month is excessive. And there are some people who say, I would pay $500 a month for this.
And there’s just a huge diversity in price acceptance for Copilot. But the super interesting thing is when we do the analysis the Van Westendorp recommends a price of say, between $15 and $20. But the revenue optimizing price is around $29 or $30. And so Microsoft, I assume that they did similar analysis and that they have access to more data than we do. And they picked the revenue-optimizing price, which makes good sense at this stage in the market’s evolution, because Microsoft has great market access. They don’t need to worry about having a low price to provide early market access. And they want to establish a reasonably high reference price. But remember, they have to do this across all of the different office users.
One of the other really interesting things we discovered, which surprised me, was that there is a group of people who were willing to pay significantly more than the $30, which is the revenue optimizing price, or the range suggested by Van Westendorp. So we compared business consultants, designers, and software engineers. And I won’t ask you to guess since this is your podcast, but I have asked a lot of people to guess, and almost nobody gets this right. The people with the highest willingness to pay are designers and the lowest willingness to pay are business consultants. Now, you would think it would be the other way around, but I suspect what’s happening is that there is a lot of fear among designers that these AIs could take away a lot of their work.
So there is a lot of interest and awareness of using them. And platforms like Figma have done a great job of integrating AI into Figma. So the designers are actually ahead of us both in fear of losing work but also in adoption and understanding and acceptance that they’re going to need to use these tools and these tools can be incredibly valuable. And most of the people with the really high willingness to pay, I’m overstating a bit here, but many of them come from the design role.
Mark Stiving
Yep. So a couple of pricing observations from what you just told us. First off, Van Westendorp when you get a few hundred results or a couple hundred 50, you actually have the ability to do some market segmentation. You were able to do that. I usually tell people, you might hate this answer, Steven, but I usually tell people, I want you to get 50 answers. I want you to get 50 respondents. But if you get 12, I’ll bet you the answers don’t change much after you get to 12. Right. So if you took the first 12 responses, I’ll bet you your curves and your answers look similar, but you can’t do your market segmentation with only 12.
Steven Forth
Yep. I think you’re generally right. I didn’t look at this data. I’m not a huge fan of doing Van Westendorp because I hate asking people what price they should pay. I think you don’t normally, if you want to know what price someone will pay, that’s one of the worst questions you can ask them. But in this case, you know, Microsoft had already gone out and established a price and it seemed like the right tool for the task at hand. And I do think we got some interesting responses. And it was kind of amazing to see, you know, 29, 30 dollars pop up as the revenue maximizing price.
Mark Stiving
Yeah, it’s interesting. Now the other thing that was interesting though is, did you make sure that everybody who did the survey was using a Microsoft platform?
Steven Forth
Nope. And there are differences between Google. Before we started this conversation, I was just comparing Google versus Microsoft and we also asked some questions about attitude towards AI. Oh, this is the other thing that shocked me. For the overall data, the majority of people are opposed to AI and are opposed to the adoption of AI. So it’s around 50%, but depending on how you cut the data, it can be as high as 60%. And I don’t think I found any cut where it was lower than 40%. And the number of people who support widespread use of AI, ranges between 10% and 20% depending on how you cut the data. And if I started cutting it too fine, the N would get too small. So maybe designers on Google in North America would I suspect that would be the highest group, but my N is not large enough to really cut the data that much.
Mark Stiving
So, count me in the majority. But I know you are definitely in the minority. But here’s the thing, it’s scary, but I’m old and I don’t wanna go learn something new.
Steven Forth
Oh, come on. It’s absolutely fascinating. So I think, Mark, that one of my passions is poetry and writing poetry. So here’s one of the things I’m doing. So I’m taking LAMA 2, this new model recently released by Meta. And I’m using the tools available to me on Hugging Face, and I am training it with my own writing. And I’m doing a couple of different approaches. One is, I’m actually training one model just on my writing. And then I’m adding the writing of people that really influenced me a lot and then I’m also taking another approach where I take an existing much larger and richer model and augment it with my own writing. And this is giving me insights into my own habits and patterns of thought that I don’t think I could get any other way.
Mark Stiving
I am positive. This is fascinating. I’m positive . Okay, here’s my question, Steven. Can I take my first three books, feed them into whatever you just said? Tell it what I want my fourth book to be, and it will write my fourth book for me at least the first draft of it.
Steven Forth
It could, yes. However, I’d suggest that it would be a boring book.
Mark Stiving
You don’t know the first three?
Steven Forth
I do. I’ve actually read them. It would be derivative, right? So I would start there. If I were you, I would absolutely do that. Or I would go and find some high school kid that wants to learn this stuff and pay him a bit of money to do it for me. So I would absolutely do that. And I would take all of the other writing that you’ve done, because you write pretty much every week you publish something on pricing. And I would feed that in and I would pause there and then I would start working with it and see what sort of content it was generating. But then I would consider taking some books that have been very influential for you and go beyond just sort of straight pricing, maybe some strategy books, a couple of business, history books, whatever it is that has really helped you grow and think and learn and train them on that as well. So, I would look beyond just myself for influences, because that’s how our brains work, right? We bring in other things. And I can assure you that many people are starting to do this.
Mark Stiving
I would feel so much like a cheater if I did that.
Steven Forth
Why? So working on your own thoughts and your own ideas.
Mark Stiving:
I don’t know. Writing is like, I don’t know, it’s a personal thing.
Steven Forth
Yeah. And I think what you’ll find is that it turns into a conversation between you and this alter you. I think part of the art of using these AIs is how you ask them questions. How you frame up the question for them. And this is referred to these days as prompt engineering and prompt engineering, in order to get useful information from AIs is going to be incredibly important. And it requires a lot of domain knowledge. So people who really understand pricing are going to actually be at a huge advantage when it comes to using AIs for pricing. Because, you know, let’s face it, one of the biggest skills that you and I have is we know what questions to ask.
Mark Stiving
And, and the other funny thing is, when you use the word pricing, in most worlds, it has a million meanings, right? And it shows up everywhere. And so you can’t ask a simple pricing question of chat GPT and get a good answer.
Steven Forth
That’s right. And there are several reasons for that, right? One is it’s being trained too generically. So augmenting one of these models with stuff that’s specific to training is going to be very powerful. And secondly, most people don’t know how to do the prompts. There’s lots of articles out there that say, I asked chat GPT X Y Z, and look at the dumb answer it gave me. And yeah, you ask dumb questions, you get dumb answers. I’m not impressed.
Mark Stiving
So it’s kind of like, when we were used to be good at writing Google search prompts.
Steven Forth
Yeah.
Mark Stiving
You had to be trained to do that.
Steven Forth
Yeah. It’s that on steroids. I mean, it’s a super interesting new world, but, you know, coming back to how AI’s going to be priced? One thing that I’m noticing, if you look at Microsoft, you know, $30 per person, per user, that’s about the same as the whole Microsoft suite is priced at. So basically what they’re saying is AI, Copilot is worth just as much as Word, Excel, PowerPoint, and all those other things put together. And, it’s not just Microsoft. That’s saying that if you look at there’s a company called Notion AI that’s gotten super popular in the project management space. Same thing there. Notion AI costs the same as notion. And I’m seeing this quite a bit that people are charging as much for the AI as they are for all of their other technology.
Mark Stiving
Okay? I’m gonna give you a pricing explanation for that. You’re gonna accept it or not. You’ve probably heard me talk about ‘will I’ and ‘which one’ a lot. Copilot for Microsoft is a ‘will I’ decision because people are not gonna switch off the Microsoft platform. And so they’re saying, am I gonna get this AI add-on or not? And so there is no competitive alternative. Do I want this capability? And so I could easily see how they get away with that. That is, if you’re trying to sell Excel or Word or PowerPoint, you’re up against the Google equivalents of each one of those, which are absolutely free at this point in time for most of us. So I could see how they could get away with that or they could even get away with more.
Steven Forth
Yeah, I completely agree. I think that’s a very, very accurate analysis. So right now for Microsoft Copilot is a ‘will I’ question. It’s not a which one.
Mark Stiving
Yep. Steven, I just noticed we’re over time already.
Steven Forth
So even get to talk about how GPT prices GPT, but we’ll talk about that next time.
Mark Stiving
We’ll talk about that next time. Thank you very much for your time today. Just so everybody can remember, how can they get a hold of you if they want to?
Steven Forth
Easiest way is by email, [email protected]. I am also super easy to find on LinkedIn and I promise I will respond to your LinkedIn message as long as you are not trying to sell me the Generation Services or IT services.
Mark Stiving
Oh, you don’t get web services sold to you either.
Steven Forth
Web services. Yeah. I must get 10 of each of those a day.
Mark Stiving
I know. It’s insane. Alright, to our listeners, thank you so much for your time. If you enjoyed this, would you please leave us a rating and a review? You can get instructions at ratethispodcast.com/impactpricing. And if you have any questions or comments about this podcast or pricing in general, feel free to email me, [email protected]. Now, go make an impact!
