The five-layer Tokenomics stack

The five layers rest on one another, and the order is load bearing. Silicon sets the floor on what a token costs; capacity determines how much of that silicon is usable and how well it is used; the inference stack decides how efficiently each request runs; the model layer fixes what runs and at what precision; routing and governance form the control plane over all of it. Each layer inherits the cost and availability of the ones beneath, so gains compound upward. Cutting across all five are the capabilities that make the stack operable: governance and security, identity and access, observability, and cost management. They are not optimization levers in the way a caching strategy is, but they are why a token can be attributed, a budget enforced, or a quality regression caught. Each is dealt with where it does its work: governance appears at L5 as budgets, quotas, and circuit breakers; observability and cost management underwrite the measurement section below; identity sits beneath both, since a token cannot be attributed to a team, tenant, or feature the stack cannot name. Their absence is what turns an efficient architecture into an unverifiable one.

The bottom-up sequence applies to anyone who chooses how their models are served, not only those who own hardware; on a cloud, selecting GPU, TPU, CPU, or a managed service is itself a lower-layer decision. The exception is the organization that only calls a frontier API, with no access below the inference stack, whose levers concentrate in model choice, routing, and caching at the top.

The consumption spectrum

Token consumption is not flat. The same useful task can consume almost nothing or scale without bound depending on how it is architected, and Big-T notation names the classes that range spans: T(1) where a cache hit avoids generation entirely, T(log n) where deterministic work strips the input before the model sees it, T(n) as the honest linear baseline, T(n·k) where one activity becomes many calls, T(n·k·a) where that fans across an agent tree, and T(∞) where an autonomous loop has no ceiling. The classes, the evidence behind them, and the argument for why a higher class is a decision rather than a failure are set out in the Big-T paper and taken as given here.

What the spectrum does not tell you is where to act. That is what the layers are for, and the two answer different questions: the layers are where optimization is applied, the classes are what that optimization achieves. The distinction matters because the layers do not all work the same way. Silicon and capacity set what a token costs, in dollars and in watts, but they cannot move a workload between classes. A poorly utilized fleet makes every token more expensive than it needs to be at whatever class the workload already occupies.

The upper three layers change how many tokens are consumed, which means they can move a workload down. Caching and reuse in the inference stack turn repeated work into T(1) and T(log n). Right-sizing a model pulls T(n·k) work back toward T(n). Budgets, caps, and circuit breakers in the routing layer are what keep an agent out of T(∞).

The layers are the tools that move a workload into a lower class, and then hold it there.

Set side by side, the layers divide cleanly into those that set the cost of a token and those that move a workload between tiers.

L1Silicon

Silicon is the physical substrate: the accelerator and its generation, along with the memory, interconnect, and system power it ships with. It converts energy into tokens, and it sets the floor on cost per token and energy per token for everything above it. Newer generations push both down along a decay curve, and not all tokens are equal: a low-latency token on a general-purpose GPU can cost more than the same token on purpose-built silicon. What this layer covers is what you select; how that hardware is powered, placed, and used is a capacity decision, so interconnect and power appear here as components and again at L2 as choices.

The measures that matter are cost per token in dollars and in watts, tokens per second per accelerator, price performance in dollars per million tokens, efficiency as TOPS per watt, and the mix of hardware generations across the fleet.

Optimization begins with matching the silicon class to the workload rather than buying one general-purpose part for everything. Latency-critical and real-time work favors inference-specialized silicon; large mixture-of-experts models favor rack-scale systems with high-bandwidth interconnect. Adopting the newest generation pays off when its price performance justifies the move, and steady, high-volume inference is often a candidate for custom accelerators.

L2Capacity

Capacity is the decision layer over the substrate beneath it: how much of it you hold, where it sits, how it is powered and connected, and how much of it does useful work. Reserved or on-demand, one region or another, scaled up or scaled to zero, all of it decided against a traffic shape that rarely matches the commitment. Silicon that cannot be powered, or that sits idle, produces expensive tokens, because idle capacity raises the effective per-token cost of the work that does run.

Utilization is the central measure: productive hours against idle hours, reserved capacity against actual use, power draw and data-center efficiency, regional cost differentials, queue and wait times, and the share of batchable work actually run during demand troughs.

Idle capacity is best filled with batch and asynchronous inference during off-peak hours. Autoscaling and scale-to-zero absorb spiky traffic; a blend of reserved, on-demand, and spot capacity matches the traffic shape; and workloads move to cheaper or greener regions where data residency allows. Disaggregating prefill and decode lets each run on the capacity that suits it, and rack-scale interconnect serves large and mixture-of-experts models.

L3Inference stack

The inference stack is the serving software: the engine, key-value (KV) cache management, batching, and the split between prefill and decode. It is usually the single largest source of optimization, and the first layer where consumption is actively suppressed rather than merely priced.

Useful signals include GPU utilization, throughput in tokens and requests per second, time to first token, inter-token latency, KV cache hit rate and prefix-reuse percentage, batch efficiency, and tail latency at the 50th and 95th percentiles.

The first decision is choosing an engine that fits the workload. From there, prefix caching is enabled and the KV cache is extended beyond GPU memory, tiered across GPU, CPU memory, and NVMe, with cached blocks reused across queries. Speculative decoding adds a small draft model, and continuous batching lets new requests join the running batch as slots free. Disaggregating prefill and decode is decided here and provisioned at L2; the engine supports the split, the capacity layer supplies workers suited to each half.

Cache hit rate is the number that matters most, and it is won largely through prompt and prefix stability. This is the layer that turns repeated work into T(1) and T(log n).

L4Model management and selection

This layer fixes which model runs, at what numeric precision, and with what adapted weights. Running a frontier-scale model for a trivial task is the classic waste; quantization reduces memory and compute with limited quality loss, and distillation and pruning go further.

The right measures are cost per task or outcome rather than cost per token alone, quality per dollar expressed as an evaluation score against spend, VRAM footprint, throughput at a given precision, and the accuracy change after quantization, checked against a held-out set so regressions surface before release.

Models are right-sized to the task, with sub-steps routed to smaller models, and quantized to the precision the hardware supports well. FP8 is near-lossless on current hardware; FP4 formats such as NVFP4 and MXFP4 offer more throughput but need validation and often quantization-aware tuning; INT4 with AWQ or GPTQ suits older and consumer GPUs; and GGUF suits CPU and edge.

Calibrating quantization on real traffic, distilling task-specific small models, and pruning where quality holds complete the toolkit. Adapters such as LoRA are the third option between right-sizing and escalating back to a frontier model: task-specific weights applied over one frozen base, so many variants share a single deployment.

Right-sizing is what pulls multiplicative work back toward linear; precision and adapter strategy set what each token costs once the class is settled.

L5Routing and governance

Routing and governance is the control plane in front of everything else: which model or deployment serves each request, caching at the gateway, budgets and quotas, fallbacks, and the guardrails that bound agent fan-out and retries. It is the only layer that can decide whether a request needs no model at all, and the only one that can stop a workload consuming without limit.

Measurement centers on per-request and per-route cost attribution; cost per feature, team, customer, or tenant; the model-mix distribution; semantic cache hit rate; the share of requests routed to cheaper models; budget burn-down; fan-out width per task; and retry, iteration, and agent-step counts.

Requests are routed by complexity, so simple prompts reach cheap or local models and only hard prompts reach frontier models, with cascading where it helps. Semantic caching sits at the gateway. Per-team and per-tenant token budgets are enforced, with routes demoted as budgets are hit. Agentic behavior is bounded by fan-out limits, recursion and iteration caps, and circuit breakers, while fallbacks and load balancing provide resilience.

This is the primary lever for the amplifying classes. Routing pulls multiplicative work toward linear by sending it somewhere cheaper; caps and circuit breakers are what keep agent-multiplicative work from becoming unbounded. The distinction matters, because the first is an optimization and the second is a containment: a workload with no ceiling has no worst case, and no amount of efficiency in the layers beneath compensates for that.

Measuring across the stack

Measurement here is diagnostic rather than decisional. In classic FinOps the measurement gates a discretionary choice: here is the spend, and the question is whether an optimization is worth acting on. In tokenomics the optimization is built into how the workload runs, so measurement instead reports whether the discipline is holding and where it is degrading.

The signal worth watching is drift up the consumption spectrum without a matching rise in value, a workload that was linear quietly becoming multiplicative or agent-multiplicative. Drift is rarely a decision anyone made; it arrives through a model upgrade that enables extended thinking by default, a feature that adds a tool-use loop, an agent given one more level of delegation. The class changed and nothing in the request looks different. Measurement is the only way that surfaces before the invoice does.

When it does surface, the question is whether the new class is justified by what the workload now returns, and that is a question for whoever owns the workload rather than for the measurement layer itself. The budgets, quotas, and circuit breakers at L5 do not answer it either; they hold a workload inside agreed bounds while the answer is worked out.

Token cost data arrives from three categories of tooling, which is part of why a shared definition of conformant, token-level cost data matters.

Two details deserve standardization. Usage now divides into more than input and output: cached tokens, reasoning tokens, and audio and image tokens each price differently, so token-level data should carry the usage type. Without it, the reasoning multiplier that moves a workload between classes is invisible in the billing record, which is precisely the drift this section is about catching.

The second is granularity. FOCUS data at the token level rather than the invoice level is what makes any of this comparable across providers, models, and teams. Measurement across a stack this heterogeneous depends on the data arriving in a shape that can be compared at all.

Conclusion

The stack and the spectrum turn token cost from a single number into a set of architectural decisions with known levers. Silicon sets the lower limit on what a token costs. Capacity decides how much of that silicon does useful work. The inference stack is where repeated work stops being generated at all. The model layer fixes what runs and at what precision. Routing and governance decide whether a request needs a model in the first place, and whether an agent has a ceiling. The lower two layers set what a token costs; the upper three move a workload between classes, and then hold it there.

The stack says where to act; the spectrum says what the action is worth. The discipline is keeping the two aligned, so that every unit of work runs on the optimal class its value can justify, and every token in that class is as efficient as the layers beneath it can produce it.

None of that is one person's job. The decisions are distributed across the stack by construction, which means the discipline is too, and an organization gets good at it the way it gets good at security: not by appointing someone, but by enough of the people making those decisions understanding what their layer costs.

The five layers are a starting model, not a finished standard. They exist to establish a shared vocabulary to refine and extend; the boundaries, the metrics, and the tooling that fills them will move as the field does. Some of that movement is already visible. As local and on-premise generation gains more attention, tokens may leave the metered economy altogether, which changes the shape of the cost rather than removing it. And value, how token cost feeds pricing, monetization, and product decisions, is what turns efficiency into business outcome. Consumption is treated first because it is where the engineering leverage is most concrete today, not because it is the whole of tokenomics.