The State of AI Tokenomics: Top Takeaways

00:00Cold Open

Mission critical AI it’s not like you can just turn off the tap, right? Because business stops when we do that. Three in four enterprises said they could not show value to their CFO on their AI spending. It wasn’t that the AI is costing too much, it’s that they couldn’t prove value on it. But is power user measured by how many tokens they Mm-hmm

or is it measured by their output? We all know lines of code is not the right measure. No, not commits, not PRs. Number of tokens is also not the right measure. What exactly are they building? Okay. What are they shipping- Yeah... into production? That’s really the value there.

00:39Who Took the Survey

So state of tokenomics We launched survey data collection on this just a few weeks ago. And in that small amount of time, we got a lot of signal. We had 472 companies take the survey, and they were big companies. They represented collectively about 4.6 trillion in revenue. A lot of Fortune 100s from the US in this group.

So first we asked where they are running their AI. Not a shocker. Everybody’s using frontier, right, from the big providers. We also saw a very high cardinality of lots of models being delivered in lots of different places. So not surprising, a lot of frontier, a lot of open weights. But a pretty meaningful number, 10, 18, 16% of first party models.

Models that people are training or fine-tuning on their own, sometimes hosted and sometimes on their own hardware. Any surprises today from conversations or from the data? Yeah. So I would say from a macro perspective, I do think the trends and the observations align with, you know, the conversations that we have with our partners, whether they be software companies or professional services, or even what we’re doing in-house.

but I think the... So no major surprises from that perspective, but, you know, I think some of the numbers- Hmm... have been surprising. How so? So for example the percentage of companies who already have first party models, right? I think that’s interesting. That was surprising to me too. Yeah. Yeah.

02:04Meet Vanessa Yiu

Can you tell us a little bit about, you know, your role? First of all I am the CTO for technology infrastructure at Deutsche Bank. And our infrastructure organization pretty much covers all of the horizontal platforms that serves the bank. You think about, you know, public cloud, private cloud you know, databases you...

Even all the way through to end user devices. Obviously in the remit of the CTO role, I’m kind of thinking about more strategically where we’re Hmm... In terms of investments, our organization. But I also do have direct responsibility for a couple of functions, which includes AI ops, automation and also observability engineering.

02:43Frontier to Open Weights

So you can see how some of this would tie back to this AI economics concept. We then asked, in 12 months, where do you expect your open weights usage to be? And we see a pretty significant shift in the balance from current frontier model usage to a more balanced mix of frontier and open weights models.

Couple points to call out here. 51% of the folks said that they are, like, 80 to 100% frontier models, but only 24% of those expect to be there next year. The heaviest users of frontier models, those that selected 10, had the biggest drop, the biggest shift expected. And only 17% of the companies reported that they are going to shift more to frontier next year.

So 83% were gonna shift more toward open weights. What’s your, you know, balance of frontier and open weights? Mm-hmm. Do you have private AI? What, what does that look like in terms of a ecosystem? Yeah. So predominantly frontier, right? And though I think, you know, reflecting on the survey data, I think we are gonna see this trend of moving towards more towards open weights and first-party models.

and I think for a number of reasons cost is one of them. But I think more importantly we own a lot of financial data in the bank, right? Mm-hmm. Mm-hmm. And let’s be honest, LLMs are not necessarily trained with financial services data. So for the things that we have to do you know then, then perhaps there is the opportunity then to do more of the first-party models.

04:18The Value Gap

So we got into visibility, and we asked, "How confident are you that you can show visibility within your organization of all of your AI spending?" And there was a mix. I was actually happy to see that the biggest one was moderately confident people. Little surprising actually, given where things are But on the flip side, three in four enterprises said they could not show value to their CFO on their AI spending.

There’s some common traits of those who are confident in their spending. Those companies are metering and attributing workloads between teams, and they’re tracking concrete business metrics against their spending goals. Notably, confidence in spend visibility fell for the larger companies. So bigger company, the less likely they were to be confident in their ability to report visibility on the spending.

And if we correlate that against the value question, we found that 60% of those organizations who told us, "Yeah, we’re moderately confident in our spending," they couldn’t show value. 20- only 25% of those who had high confidence in their visibility were able to show value to their CFO. So a challenge a challenge in value realization and reporting regardless of the level of visibility that you have.

How do you think about going to show value back to your CFO? I think we’ve always invested in technology based on, you know, business value. So I, I think this is not necessarily a new concept in how I think about it. So in some ways, the business KPIs don’t change actually. If, if it is about, you know increasing the top line, if it is about increasing productivity in certain aspects or maybe being able to you know, talk to more clients, whatever the dimension is, I think those KPIs are largely the same.

However, with AI in the mix, I think there is the expectations maybe are a little bit different, right? How do you hit a particular target of the KPI? Maybe there’s an expectation that it will happen more quickly because of AI. We asked what the biggest challenge is in tokenomics. Challenges, this was a free form answer.

They’re grouped together. Not surprisingly, proving value, ROI, 43% of you. That was the biggest group that said that. What was interesting is only 7% reported that pricing complexity or prices being too high or some form of that were the big challenge, right? The... It wasn’t that the AI is costing too much, it’s that they couldn’t prove value on it.

They couldn’t attribute it. They couldn’t measure it. They needed to re-skill. They needed to forecast. They needed governance improvements. So all of these things stacking down to what is really How do we get to showing and proving the value? What are the big challenges that you’re facing right now? Yeah.

07:01Risk and Volatility

So when it comes to financial services, I think it, you know, what keeps us awake a lot of it is about risk management. And I, I will say less about the observability in tokens. It’s the huge risk. I think operational risk is probably something that, that we’Okay... thinking about. You know, if we are rolling out agentic systems at scale obviously we run a lot of mission critical services and a lot of them, you know, doesn’t just impact our organization.

If there’s failures, it could have broader financial markets implications. So, you know, as we’re moving from more of a deterministic model to probabilistic with, you know, AI models, how does that change the failure mode? What could we be introducing that we’re not aware of? And so I think from a risks perspective, I think that is the thing that’s, you know, keeping me awake at night and we have to think through.

But tying this back to tokenomics, look, I think there is a degree of uncertainty around how we manage this, right? If we look at the traditional models, right? You know when, certainly when I first started out in my career most infrastructure spend is CapEx, right? It’s well understood. It’s a large upfront investment.

You understand the life cycle management of the infrastructure. And then over the years we’ve moved to cloud towards the OpEx model. And I think even now that, you know, by now we do have well understood structures and frameworks about how we manage that. And suddenly I think when you add AI into the mix, okay, all of a sudden we have this level of volatility Mm.

08:27Ownership

Mm-hmm... you know, maybe is not well understood how we manage yet. And so, you know, that is a level of uncertainty that we have to manage through as well. 88% of these companies said they had some kind of defined ownership. I think this is probably gonna seem like a high number to a lot of folks, but keep in mind this was very large companies who have very large AI budgets already 12% of those said there was no owner of this, and of the, those 12%, not a single one said that they were able to correlate value and show that to their CFO, right?

So ownership being at least a strong correlation to the ability to prove value. We also saw a correlation of 3.7x higher likelihood to showing value when there was ownership. So no surprises there. Hmm. So I think the survey data shows that it largely sits within technology at the moment.

And I think that’s because because we’re mainly working at the platform implementation- Hmm... level. And really the technology platforms is where we surface a lot of this data. Though having said that, I would say I am partnering very closely with colleagues from other, other disciplines or departments like, like procurement, like finance you know, even legal compliance to kind of really figure out the overall big picture of how this all ties together.

09:44Model Routing

I think we all come to this from a different perspective. And you know, I, I think, you know, it’s only gonna be successful if we, if we take into all of those different, different views. So model routing was another big topic that came up. Model routing we found was being done or evaluated by 86% of the organizations.

I think about half were saying they had, they had implemented it, and another 40 or so percent were saying they were currently evaluating those. High correlation as well with model routing and the ability to show value, a maturity correlation. In the biggest enterprises, we found that they were both building and buying model routers, that they had multiple options running.

So breaking down more of the free form answers, we asked what they were doing, which tools they were using. There was a long tail. I think there was it was about 36, 38 give or take different routing offerings were mentioned in the data. Open Router and LiteLLM were the two that were the highest on that list.

So when we talked before and you had mentioned Hmm. That’s right... and that impact. Fine-tuning, yeah. How, how does that play in here? And what, what is SLM for the crowd? Small language model. So they’re more fine-tuned for a specific domain. In our case it would be financial services use cases.

and typically we would be hosting that in-house. But I think there is gonna be this diversification, right? And I think you know, where possible we would want to have more performant SLMs for, you know, like specific use cases that we have and then lean on LLMs to do more of the kind of broader, more general Mm-hmm

11:15Pricing and Margins

across the bank. We asked is AI, the cost, the spending, the investment, is it impacting your business models? Is it changing the way that you need to price to customers or how you sell to customers? 52% said that they had already changed pricing or were considering changing pricing. I would give an asterisk to this number.

If you recall, the biggest group, the industry, was technology and software, right? Where they’re most likely to be adjusting software per seat, consumption-based licensing based on this. But I think we’re also gonna see for the non-technology and software companies delivering different types of services, a margin management challenge, and this was quoted a lot.

"Although we’re not changing how we’re pricing, we’re really struggling to maintain margins, as we don’t, we can’t predict or control the costs underneath the services we’re delivering to our customers." And a lot of uncertainty. A lot of the questions were like, "We just don’t know." But how does the cost of AI impact your business planning now?

Yeah, so it’s, it’s, you know, less about the pricing per se, because we’re not necessarily selling, selling a product. But I think it does impact how we, how we budget. And, you know, I think more and more we’re thinking about, okay, for all of the business investments, what would also be the token costs that would be required?

12:36Developer Productivity

Mm. Yeah. Mm-hmm. We asked how companies are doing governance specifically of developer productivity, right? Because this is different than maybe mission-critical AI that’s running some system for you. And frankly, there weren’t a lot of good answers. Yeah. Most of the people wrote in some form of, "We don’t have a formal way to properly manage developer productivity spending."

they were really focused around monitoring. Again, just simple visibility, saying, "We’re we’re gonna figure out ROI and returns on that later," right? Is it shipping PRs or code? Is that the thing? Is it tying it to outcomes? What does that look like? It was punted. So maybe this isn’t surprising, this data, but it shows that there’s a real immaturity in some of the biggest companies in the world with some of these companies I know that I’ve spoken to, you know, having massive hundreds or billions of dollars of AI spend and not being able to figure this out.

So speaking of developer productivity- Mm-hmm... What does that look like for you in the bank? Like, what’s, what’s working? How, how do you think about that, and how is it distinct maybe from that mission-critical AI and how you’re, you know, doing governance around that? Yeah. I think mission-critical AI is obviously a lot harder because of the complexity.

So with developer productivity, I think the... You know, it’s easy to say, "Oh, well, we’ll give you a cap, okay? And if you hit that limit, okay, there’s a process where you can get more tokens." So that’s kind of better understood. I think with mission-critical AI it’s not like you can just turn off the tap, right?

Because business stops when we do that. So I think we have to be a lot more thoughtful about the design and actually being able to model a lot of this upfront. Which is interesting, right? Because going back to the CapEx versus OpEx we’ve always said OpEx is kind of elastic. Mm. Right? Mm-hmm.

You can get more when you want it. Mm-hmm. But now we have OpEx in the context of AI. Maybe it, you know, isn’t that. We have to actually go back to almost, like, capacity planning and really understanding the shape of the workloads- Mm-hmm... and what we’re gonna do in order to get a better estimation. Is power user measured by how many tokens they use?

Mm-hmm. Or is it measured by their outputs or their Yeah... or their value? It’s, it’s... Yeah so going back to developer productivity, we all know lines of code is not the right measure. No, not commits, not PRs. Number of tokens is also not the right measure. So I think it’s really down to the work that they’re doing.

14:48Total Cost of AI

What exactly are they building? Okay. You know, what are they shipping- Yeah... into production? That’s really the value there One in five using open-weights models. Think about that shift that’s happening. Think about the number of open-weights models out there, and think about how you’re gonna want, even if you’re running it on your own hardware, which is free, right?

If you have it in-house. No, there’s all these costs associated with that. You’re gonna wanna be able to get that total cost of AI and align it to your frontier model providers. So this is a big challenge waiting to happen that’s only gonna get a lot worse. For you, total cost of AI what does that look like and why does it, why does it matter in your practice?

Yeah. So I would say this definitely doesn’t know about, not just about accounting tokens. I mean, look, in the infrastructure business we’ve always had to consider all of the different layers of the infrastructure platforms. Mm. So I think now we’re just literally adding AI as one aspect of that.

So I think you know, what we’re hearing about a total cost of AI it is important and I think there is this, for us as a bank, there’s a substantial part which is also not AI-related that we have to take into account. Mm. Mm-hmm. And I think we talk, we’re always talking about the total cost of ownership of any application that we run.

16:01The Case for Standards

and whilst, you know, the AI portion is increasing-... there is still a lot that is not AI. Mm. Companies are struggling to measure value, they’re struggling with fragmented visibility, and many of them reported a challenge with lots of different ways that the model providers or the AI providers are putting out data.

Cost data, usage data in different formats, different granularities. Does this sound familiar to the challenges that we faced in cloud over the last few years? So we’re seeing these model providers and the token providers and those hosting those playing the same game that we saw the cloud providers play over the years of getting better and deeper data, and a complete lack of standardization in how that data is reported today.

What frameworks, what, what areas do you think we should be focusing in? Yeah. I mean, look, I already think there’s a lot of interest and passion in the community to work on some of these almost like unanswered questions at this point, right? And I, I think, you know, that’s, that’s great and we should continue.

I, I would like to see some common set of standards- Mm... if you will. Because if I... You know, speaking as a, as a technologist, typically if I’m building some application, I understand what the architectural blueprints, what good looks like. In this space, I just don’t think we have that and I think this is not something any single provider or any single person can come up with.

So, you know, I would really like us to come together as a community and work on some of these kind of common challenges that we have. Amazing. Well, Vanessa, I really appreciate you taking time out of Mm. Thank you... exceptionally busy schedule to come and share your learnings. We hope that you’ll come back next year and give us an update and thank you for your time.

That’s great. Thank you. Thank you for having me. Thank you, Chad. Thank you.