# Tokenomics Terminology 827 terms, 94 of them KPIs (also listed in kpis.md). Pinned to mind map v0.0.3. Published by the Tokenomics Foundation as an open, vendor-neutral resource. CC BY 4.0. ## # ### 3FS (Fire-Flyer File System) **Concept** · Tokenomics > Production > Serving > KV Cache > Cache Management > Hierarchical KV cache > KV cache storage tier > 3FS (Fire-Flyer File System) DeepSeek's open-source distributed file system for AI training and inference, built around SSDs and RDMA, used as a KV cache store as well as a training data store. Sources: ## A ### A/B testing **Concept** · Tokenomics > Consumption > Guardrails > Production evals > A/B testing Comparing variants on live traffic, the only way to see effects that offline sets cannot reproduce. ### A2A (Agent to Agent) **Concept** · Tokenomics > Consumption > Optimization > Agentic > Tool and protocol layer > A2A (Agent to Agent) A protocol for communication between separate agent systems, addressing coordination rather than tool access. ### Abuse and automation **Concept** · Tokenomics > Value > Monetization > Margin risk > Abuse and automation Programmatic consumption of a plan priced for human use. ### Abuse and cost attacks **Concept** · Tokenomics > Consumption > Guardrails > Adversarial input > Abuse and cost attacks Traffic designed to consume expensive capacity, where the damage is the bill rather than the output. ### Accelerator metrics **Concept** · Tokenomics > Consumption > Observability > Telemetry sources > Accelerator metrics Device-level utilization, memory, power, and temperature, which reveal whether hardware is earning its cost. ### Accelerator type (GPU, TPU, Trainium, Inferentia, ASIC) **Concept** · Tokenomics > Production > Hardware > Capacity > Compute > Accelerator type (GPU, TPU, Trainium, Inferentia, ASIC) The class of chip used, each with distinct cost and performance tradeoffs. ### Acceptance rate **KPI** · Tokenomics > Production > Serving > Inference Optimization > Speculative decoding > Economics > Acceptance rate The share of drafted tokens accepted, and the mean accepted length per step; the single number that determines whether speculation pays. ### Acceptance rule **Concept** · Tokenomics > Production > Serving > Inference Optimization > Speculative decoding > Mechanics > Acceptance rule The test that decides how many drafted tokens survive; the standard rule is lossless, meaning accepted tokens match the target model's own distribution. ### Access & Delivery **Concept** · Tokenomics > Consumption > Access & Delivery How requests are sent and results returned, which affects price. ### Accuracy vs price tradeoff **Concept** · Tokenomics > Consumption > Optimization > Model Selection > Accuracy vs price tradeoff Weighing quality gains against the higher cost of stronger models. ### Achieved memory bandwidth **KPI** · Tokenomics > Production > Hardware > Performance characterization > Achieved memory bandwidth How much of rated bandwidth a workload actually reaches, the decode-side counterpart to MFU. ### Activation quantization **Concept** · Tokenomics > Production > Training > Quantization > What gets quantized > Activation quantization Reduces precision of intermediate values too, needed to actually use low-precision arithmetic units. ### Active vs total params (MoE) **KPI** · Tokenomics > Production > Model / Architecture > Scale > Active vs total params (MoE) In MoE models only a fraction of parameters run per token, decoupling size from compute cost. ### AdamW **Concept** · Tokenomics > Production > Training > Pretraining > Optimizer > AdamW The default adaptive optimizer, whose two state tensors per parameter dominate optimizer memory. ### Admission control and queueing **Concept** · Tokenomics > Production > Serving > Batching > Admission control and queueing Holding or rejecting work when the engine is saturated, the alternative to letting latency degrade for everyone. ### Adoption and spend growth **KPI** · Tokenomics > Consumption > Demand > Adoption and spend growth How quickly pilots turn into material token budgets, which sets the pressure on every cost control downstream. ### Adversarial input **Concept** · Tokenomics > Consumption > Guardrails > Adversarial input Deliberate attempts to break the intended behaviour of a deployed model. ### Affective facet **Concept** · Tokenomics > Value > Business Outcomes > Value Categorization > Employee Engagement > Affective facet Feelings toward the organization: whether AI change management was done well or people are worried about layoffs. Part of the full decomposition, not carried on the claim. ### Affine quantization **Concept** · Tokenomics > Production > Training > Quantization > Scheme > Affine quantization Uses a scale and zero-point offset to map values, handling asymmetric ranges. ### Agent frameworks **Concept** · Tokenomics > Consumption > Optimization > Agentic > Tool and protocol layer > Agent frameworks Libraries providing the loop, memory, and tool plumbing, which set defaults that quietly determine token spend. ### Agentic **Concept** · Tokenomics > Consumption > Optimization > Agentic Multi-step, tool-using workflows that can multiply token consumption. As seen in: [What Is AI Tokenomics?](https://www.tokeneconomics.com/what-is-ai-tokenomics/), [Big-T Notation](https://www.tokeneconomics.com/projects/big-t-notation/) ### Agentic RAG **Concept** · Tokenomics > Consumption > Optimization > AI memory > Knowledge injection patterns > Agentic RAG An agent plans and refines its own retrieval over several rounds, which raises answer quality and multiplies both retrieval calls and generation cost. ### Aggregators and gateways **Concept** · Tokenomics > Consumption > Access & Delivery > Aggregators and gateways Intermediaries offering many models behind one interface, adding routing and failover at some margin. Where a product only passes traffic through and collects metadata it is a proxy; the Proxy, Router, and Gateway definitions live under Optimization and Routing. ### AI factory **Concept** · Tokenomics > Production > Hardware > Capacity > Facilities > AI factory A facility designed end to end to produce AI rather than house servers: power, cooling, accelerators, and networking engineered as one system whose output is intelligence. As seen in: [What Is AI Tokenomics?](https://www.tokeneconomics.com/what-is-ai-tokenomics/) ### AI memory **Concept** · Tokenomics > Consumption > Optimization > AI memory Giving models access to prior context and external knowledge. ### AIBrix KVCache **Concept** · Tokenomics > Production > Serving > KV Cache > Cache Management > Hierarchical KV cache > KV cache storage tier > AIBrix KVCache A KV cache offloading framework providing memory tiering and cross-engine reuse, with pluggable eviction policies and backend stores. Sources: ### Alerting **Concept** · Tokenomics > Consumption > Observability > Practices > Alerting Notification on threshold breaches, useful only where a named owner can act. ### Algorithm **Concept** · Tokenomics > Production > Tokenization > Algorithm The method used to split text into tokens. ### Algorithms **Concept** · Tokenomics > Production > Training > Quantization > Algorithms Specific quantization techniques. ### Alignment **Concept** · Tokenomics > Production > Training > Post-Training > Alignment Shaping model behavior to human preferences and safety norms. ### Amortization over inference **Concept** · Tokenomics > Production > Training > Training economics > Amortization over inference Training cost spread across the tokens the model eventually serves, which only pays back at volume. ### ANN indexes (HNSW, IVF) **Concept** · Tokenomics > Consumption > Optimization > AI memory > Retrieval mechanics > Vector store > ANN indexes (HNSW, IVF) Approximate nearest neighbour structures that make vector search fast, trading a little recall for large speed gains. ### Any-to-any / omni models **Concept** · Tokenomics > Production > Model / Architecture > Modality > Any-to-any / omni models Single models accepting and producing several modalities, which complicates per-unit pricing. ### Application and gateway logs **Concept** · Tokenomics > Consumption > Observability > Telemetry sources > Application and gateway logs Request-level records including model, tokens, and user, usually the practical basis for attribution. ### Arbitrage **Concept** · Tokenomics > Value > Market Structure > Arbitrage Exploiting price differences between providers, regions, or commitment types for the same effective capability. ### Arena and Elo ratings **KPI** · Tokenomics > Production > Training > Evaluation > Benchmarks > Arena and Elo ratings Rankings derived from pairwise human preference, which capture perceived quality that fixed test sets miss. ### Arithmetic intensity **KPI** · Tokenomics > Production > Hardware > Performance characterization > Arithmetic intensity Operations performed per byte moved, the number that decides which limit a workload hits. ### Attention **Concept** · Tokenomics > Production > Model / Architecture > Attention The mechanism relating tokens to each other, and a major driver of memory and compute. ### Attention backends **Concept** · Tokenomics > Production > Serving > Inference Optimization > Kernels and compilation > Attention backends Interchangeable attention implementations, where the chosen backend constrains which other features can be enabled. ### Attention sinks **Concept** · Tokenomics > Production > Model / Architecture > Context > Long-context methods > Attention sinks Keeping the earliest tokens permanently in cache so quality does not collapse when older context is evicted. ### Attribution boundary **Concept** · Tokenomics > Value > Unit Economics > Measurement pitfalls > Attribution boundary Whether retrieval, storage, guardrails, and human review count toward the unit cost, which is a choice that must be stated to be comparable. ### Attribution grain **Concept** · Tokenomics > Value > Business Outcomes > Value Measurement > Cost boundary > Attribution grain For a claim and a cost to compose into a return, TCA has to land on the same workload boundary as the outcome. The value side defines the grain; the requirement is stated here and the solution is being built separately. As seen in: [AI Value Classification](https://www.tokeneconomics.com/projects/ai-value-classification/) ### Audio / speech **Concept** · Tokenomics > Production > Model / Architecture > Modality > Audio / speech Processes or generates audio, with its own tokenization and latency profile. ### Audio and speech workloads **Concept** · Tokenomics > Consumption > Workloads / Use Cases > Audio and speech workloads Applications built on audio input or output, usually metered per second or minute rather than per token. ### Audio tokens **Concept** · Tokenomics > Production > Tokenization > Multimodal tokenization > Audio tokens Audio is discretized into tokens, typically many per second of sound. ### Augmentation and automation **Concept** · Tokenomics > Value > Business Outcomes > Value Measurement > Augmentation and automation The old single Labor Offset label let teams claim augmentation and defend it with automation math. The split into two categories is the resolution: they need different baselines, human alone versus best-of-either, and saying which one you mean is most of the defense. What freed people move to afterwards stays open. ### Autoscaling **Concept** · Tokenomics > Production > Serving > Scaling > Autoscaling Automatically adding or removing serving capacity with load. ### AWQ **Concept** · Tokenomics > Production > Training > Quantization > Algorithms > AWQ Activation-aware quantization that protects the most important weights. ## B ### Backpressure **Concept** · Tokenomics > Consumption > Access & Delivery > Client-side resilience > Backpressure Signalling upstream to slow down rather than queueing without bound, the difference between degraded and collapsed. ### Bandwidth-adaptive scheduling **Concept** · Tokenomics > Production > Serving > Inference Optimization > Memory hierarchy and offloading > Bandwidth-adaptive scheduling Dividing work between transfer and host execution according to measured bandwidth rather than committing to one fixed strategy. ### Base model **Concept** · Tokenomics > Production > Model / Architecture > Model variants > Base model A pretrained model with no instruction tuning, which completes text rather than following instructions. ### Baseline **Concept** · Tokenomics > Value > Business Outcomes > Value Measurement > Baseline The comparison a value claim rests on, ideally the best available alternative rather than a stale before-AI state. Baselines go stale quickly and created processes may have none. As seen in: [AI Value Classification](https://www.tokeneconomics.com/projects/ai-value-classification/) ### Batch / offline jobs **Concept** · Tokenomics > Consumption > Workloads / Use Cases > Batch / offline jobs Non-interactive bulk processing that can use discounted throughput. ### Batch API (discounted) **Concept** · Tokenomics > Consumption > Access & Delivery > Batch API (discounted) Submitting jobs for asynchronous processing at a lower rate. ### Batch API discounts **Concept** · Tokenomics > Consumption > Purchase Options > Price modifiers > Batch API discounts Lower prices for non-urgent asynchronous jobs. ### Batch size and the latency tradeoff **Concept** · Tokenomics > Production > Serving > Batching > Batch size and the latency tradeoff Larger batches raise tokens per second per accelerator while lengthening the wait for any individual request. ### Batch size sensitivity **Concept** · Tokenomics > Production > Serving > Inference Optimization > Speculative decoding > Economics > Batch size sensitivity Gains are largest at low concurrency where decode leaves the accelerator idle, and shrink as batching already fills it. ### Batching **Concept** · Tokenomics > Production > Serving > Batching Grouping requests to raise GPU utilization. ### Beam search **Concept** · Tokenomics > Consumption > Optimization > Sampling and decoding controls > Beam search Exploring several candidate sequences in parallel and keeping the best, which multiplies compute and is rarely used for chat. ### Behavioral facet **Concept** · Tokenomics > Value > Business Outcomes > Value Categorization > Employee Engagement > Behavioral facet What shows up in the data: retention, productivity, and counterproductive work behaviors including withdrawal and sabotage. The facet the reported claim is built on. ### Benchmark against existing cost **Concept** · Tokenomics > Value > Business Case > Benchmark against existing cost Comparing against the human or software cost the workload replaces, which is usually the number that decides funding. ### Benchmarking **Concept** · Tokenomics > Consumption > Optimization > Model Selection > Benchmarking Testing models on representative tasks to compare quality. ### Benchmarks **Concept** · Tokenomics > Production > Training > Evaluation > Benchmarks Standard test sets used to compare models, useful for triage and weak as a proxy for production behaviour. ### BF16 **Concept** · Tokenomics > Production > Training > Quantization > Precision formats > BF16 16-bit brain floating point, the common training precision. ### Bi-encoder architecture **Concept** · Tokenomics > Consumption > Optimization > AI memory > Retrieval mechanics > Dense retrieval > Bi-encoder architecture Query and document are embedded independently, so document vectors are precomputed and the query costs one short embedding call. This is what makes dense retrieval cheap at query time and what caps its accuracy below a cross-encoder that reads both together. ### Bill shock **Concept** · Tokenomics > Consumption > Demand > Bill shock A gap between expected and actual spend that surfaces late, often driven by rapid usage growth, agent loops, or delayed cost visibility. ### Billing and usage records **Concept** · Tokenomics > Consumption > Observability > Telemetry sources > Billing and usage records The provider's own account of what was consumed, authoritative for spend and slow to arrive. ### bitsandbytes **Concept** · Tokenomics > Production > Training > Quantization > Algorithms > bitsandbytes A library providing 8-bit and 4-bit quantization, widely used as the base for memory-efficient fine-tuning. ### Blended cost **KPI** · Tokenomics > Value > Unit Economics > Margin > Blended cost A weighted average across the model mix, which is what finance sees and what hides the outliers. ### BM25 **Concept** · Tokenomics > Consumption > Optimization > AI memory > Retrieval mechanics > Sparse retrieval > BM25 A probabilistic refinement of TF-IDF that saturates term frequency and normalizes for document length, so a term appearing twenty times does not score ten times higher than one appearing twice. The default lexical scorer across search engines and hybrid stacks, and a baseline dense retrievers do not uniformly beat out of domain. Sources: ### BM25 parameters (k1 and b) **Concept** · Tokenomics > Consumption > Optimization > AI memory > Retrieval mechanics > Sparse retrieval > BM25 parameters (k1 and b) Two constants controlling term frequency saturation and document length normalization. They are corpus-specific and almost always left at their defaults, which makes them a quiet and recoverable source of retrieval quality loss. ### Book **Concept** · Tokenomics > Value > Business Outcomes > Value Measurement > Claim chain > Book Only a dollar-denominated result can reach a destination; a claim that stops at Net or converts into a non-dollar unit is reported and has no destination. Evidence marks say what kind of argument a claim requires: a ledger line, a counterfactual, an expected loss, or attribution. As seen in: [AI Value Classification](https://www.tokeneconomics.com/projects/ai-value-classification/) ### Bounded proof-of-value project **Concept** · Tokenomics > Value > Business Outcomes > Value Measurement > Bounded proof-of-value project A migration, upgrade, or technical debt program with a predefined unit of work and finish line, used to prove AI value. It offers a countable inventory, a known end point, and often a pre-AI effort estimate that serves as the baseline. Published cases: Asana's Enzyme test migration, projected to be five years from finishing at its prior pace, done in about a week and a half of engineering time; Amazon's Java 17 upgrades, credited with $260 million in annual cost savings from hosts actually removed; Google's internal migrations, where success is timed end to end, review and rollout included. Sources: , , ### BPE (Byte-Pair Encoding) **Concept** · Tokenomics > Production > Tokenization > Algorithm > BPE (Byte-Pair Encoding) Merges frequent character pairs into subword tokens, the dominant scheme for modern LLMs. ### Break-even ratio **KPI** · Tokenomics > Consumption > Optimization > Caching > Cache measurement > Break-even ratio The static target set by one price sheet: (write price multiple minus 1) over (1 minus read price multiple). At a typical 1.25x write premium and 0.1x read rate it is about 0.28, and it is zero wherever writes are free; the reuse ratio must clear it. ### Build vs adapt vs prompt **Concept** · Tokenomics > Production > Training > Training economics > Build vs adapt vs prompt The choice between training a model, fine-tuning one, and getting the same result from context engineering. ### Build vs buy **Concept** · Tokenomics > Value > Business Case > Build vs buy Whether to self-host or use a managed provider. As seen in: [Tokenomics Brief #16: Routing Wars](https://www.tokeneconomics.com/insights/podcast-16-of-tokenomics-brief-routing-wars-tokenomics-in-action/) ### Burstiness **Concept** · Tokenomics > Consumption > Workloads / Use Cases > Workload characteristics > Burstiness Peak to average ratio, which decides how much headroom must be paid for. ### Business Case **Concept** · Tokenomics > Value > Business Case The justification for AI spend against its returns. ### Business Outcomes **Concept** · Tokenomics > Value > Business Outcomes What the AI changed and by how much, measured against a stated baseline. Anchors the Value layer: cost and utilization from the other layers only matter through the outcome they produce. ### Byte-level tokenization **Concept** · Tokenomics > Production > Tokenization > Algorithm > Byte-level tokenization Operates on raw bytes so any input is representable without unknown tokens. ## C ### Cache access modes **Concept** · Tokenomics > Consumption > Optimization > Caching > Cache access modes How much cache control and cache observability a consumer gets depends on the door they enter through, from fully hidden to fully exposed. ### Cache build cost **Concept** · Tokenomics > Consumption > Optimization > AI memory > Knowledge injection patterns > CAG (Cache-Augmented Generation) > Cache build cost A one-off prefill over the entire corpus, paid again on every rebuild, which is the fixed cost the per-query saving has to repay. ### Cache compression **Concept** · Tokenomics > Production > Serving > KV Cache > Cache compression Shrinking stored KV data to fit more requests in memory. ### Cache cost efficiency **KPI** · Tokenomics > Consumption > Optimization > Caching > Cache measurement > Cache cost efficiency The fraction of prompt spend caching actually avoided: one minus actual prompt cost over the uncached-equivalent cost, the dollar-shaped readout of the ratios above. As seen in: [Cache Explainer](https://www.tokeneconomics.com/projects/cache-explainer/) ### Cache hit rate **KPI** · Tokenomics > Consumption > Optimization > Caching > Cache measurement > Cache hit rate Cache-read tokens over all input tokens (reads, writes, and uncached): coverage, driven mostly by prompt ordering, which can look healthy while the payback stays under break-even. As seen in: [Cache Explainer](https://www.tokeneconomics.com/projects/cache-explainer/#tcMonitoring) ### Cache hit rate **KPI** · Tokenomics > Value > Unit Economics > Cost drivers > Cache hit rate The share of input served from cache, the dominant driver for prompt-heavy API workloads. ### Cache isolation **Concept** · Tokenomics > Consumption > Optimization > Caching > Provider-side prompt caching > Cache isolation Provider caches are partitioned per organization, workspace, and region, so there is no cross-customer reuse; within a tenant, a fleet of agents on one prefix shares a warm entry, but a parallel fan-out misses together and pays its own writes. ### Cache Keep-Alive **Concept** · Tokenomics > Consumption > Optimization > Caching > Provider-side prompt caching > Cache Keep-Alive Also known as cache warming, keepalive pings, or heartbeat. A periodic minimal request that reads an existing cache entry during an idle period, resetting its time to live (TTL) before it expires. Keep-alive trades a small recurring cache-read cost for avoiding a much larger cache-write cost when work resumes. It pays off only when work is likely to resume before the accumulated pings cost more than the rewrite they prevent, and it saves nothing where usage is not billed per token. Longer retention tiers (for example, 1-hour or 24-hour TTLs) are the main alternative. Sources: ### Cache lifetime **Concept** · Tokenomics > Consumption > Optimization > Caching > Provider-side prompt caching > Cache lifetime Cached prefixes expire after an idle period, and some providers age the clock from the start of the request that touches the entry, which ties the benefit to traffic cadence rather than to prompt design alone. ### Cache Management **Concept** · Tokenomics > Production > Serving > KV Cache > Cache Management Strategies for storing, sharing, and evicting KV cache entries. ### Cache measurement **Concept** · Tokenomics > Consumption > Optimization > Caching > Cache measurement The ratios that decide whether a cache earns its keep, computed from token buckets in usage payloads and compared per workload, never blended. ### Cache sizing **Concept** · Tokenomics > Production > Serving > KV Cache > Cache sizing How much device memory is left for cache after weights, which sets the ceiling on concurrent requests. ### Cache tiers **Concept** · Tokenomics > Production > Serving > KV Cache > Cache Management > Hierarchical KV cache > Cache tiers GPU memory as the fast tier, host memory as the middle tier, and shared storage as the tier any node can reach. Only the shared tier is reusable across instances, and only when every instance points at the same namespace. ### Cache Warmup **Concept** · Tokenomics > Consumption > Optimization > Caching > Provider-side prompt caching > Cache Warmup Also known as cache warming or cache pre-warming. A minimal request that writes a shared prompt prefix (tool definitions, system prompt, and shared context) to a provider's prompt cache before the requests that will reuse it are sent. Use warmup before a predictable burst of traffic or before fanning out parallel work such as subagents. A cache entry becomes readable only after its writing request begins responding, so parallel requests sent sooner each pay for their own write. Warmup pays off only if every follow-up request matches the warmed prefix exactly, including model, tools, and reasoning settings. Sources: ### Cache write vs read pricing **Concept** · Tokenomics > Consumption > Optimization > Caching > Provider-side prompt caching > Cache write vs read pricing Writing to cache usually costs a premium over a normal input token while reading costs a small fraction of one, so caching only pays above a reuse threshold set by the price sheet. ### Cache-aware routing **Concept** · Tokenomics > Production > Serving > Scaling > Load balancing > Cache-aware routing Sending a request to the worker that already holds its prefix, which turns routing into a cost optimization. ### Cached token pricing **Concept** · Tokenomics > Consumption > Purchase Options > Price axes > Cached token pricing Discounted rates for reused, cached prompt tokens. ### Caching **Concept** · Tokenomics > Consumption > Optimization > Caching Reusing prior results to avoid paying for repeated work, measured against a provider-specific break-even rather than a generic hit-rate feeling. Sources: ### CAG (Cache-Augmented Generation) **Concept** · Tokenomics > Consumption > Optimization > AI memory > Knowledge injection patterns > CAG (Cache-Augmented Generation) Preloads the whole knowledge set into the context window once, caches the resulting model state, and answers every later query from that cache with no retrieval step at query time. Sources: ### Calibration data **Concept** · Tokenomics > Production > Training > Quantization > Algorithms > Calibration data The sample inputs used to fit quantization scales, which quietly determines how well the result holds up. ### Candidate depth **Concept** · Tokenomics > Consumption > Optimization > AI memory > Retrieval mechanics > Result fusion > Candidate depth How many results each retriever hands into the fusion step, set as rank\_window\_size in Elasticsearch and the prefetch limit in Qdrant. This is the real cost lever in the branch: retrieval work scales with depth times the number of retrievers, and the fused list sets how many candidates a downstream reranker bills for. Depth set too low caps quality regardless of which fusion method is chosen. ### Capability jumps versus price cuts **Concept** · Tokenomics > Consumption > Purchase Options > Price dynamics > Capability jumps versus price cuts Vendors may hold price and raise capability instead of cutting rates, which shows up as better value rather than lower spend. ### Capability limit vs harness waste **Concept** · Tokenomics > Consumption > Optimization > Agentic > Waste attribution > Capability limit vs harness waste Whether extra tokens buy capability the task needs or are overhead the scaffold could avoid, the line that decides what is fixable. ### Capability tiers **Concept** · Tokenomics > Consumption > Optimization > Model Selection > Capability tiers Grouping models by strength so the cheapest adequate one can be picked. ### Capacity **Concept** · Tokenomics > Production > Hardware > Capacity The pool of physical resources available for training and serving. ### Capacity and commitment **Concept** · Tokenomics > Consumption > Purchase Options > Capacity and commitment Buying throughput ahead of time rather than consumption after the fact. ### Capacity as a traded good **Concept** · Tokenomics > Value > Market Structure > Capacity as a traded good Accelerator time increasingly bought, resold, and brokered rather than only rented from a provider. ### Capacity blocks **Concept** · Tokenomics > Consumption > Purchase Options > Capacity and commitment > Capacity blocks Fixed future windows of guaranteed accelerator time, priced separately from on-demand. ### Capacity blocks **Concept** · Tokenomics > Production > Hardware > Procurement > Capacity blocks Fixed windows of guaranteed accelerator capacity bought in advance, a middle ground between on-demand and long reservations. ### Capacity Gain **KPI** · Tokenomics > Value > Business Outcomes > Value Categorization > Capacity Gain Same people produce more. Individually nice-to-haves, in aggregate often the largest line. For a non-profit or mission-driven organization this is usually where the gain lands: the same staff deliver more of the mission, measured in the organization's own output unit, such as cases handled or grants processed. Whether freed capacity counts as realized benefit is decided at the Net part of the claim chain, not here. As seen in: [AI Value Classification](https://www.tokeneconomics.com/projects/ai-value-classification/) ### Capacity pricing **Concept** · Tokenomics > Consumption > Purchase Options > Price axes > Capacity pricing Paying for dedicated throughput over time instead of for units consumed. ### Capex vs Opex **Concept** · Tokenomics > Production > Hardware > Procurement > Capex vs Opex Whether compute is a capital purchase or an operating expense. ### Carbon intensity **KPI** · Tokenomics > Production > Sustainability > Carbon intensity Emissions per unit of energy, which varies by grid and therefore by region and time of day. ### Cascades / fallback **Concept** · Tokenomics > Consumption > Optimization > Routing > Cascades / fallback Trying a cheap model first and escalating only when a check on the first answer fails, the pattern; confidence-based escalation and output validation are the usual triggers. ### Centralised Cache Server **Concept** · Tokenomics > Production > Serving > KV Cache > Cache Management > Cross-Engine Support > Centralised Cache Server A shared server that holds KV cache for many workers to reuse. ### Chain of thought **Concept** · Tokenomics > Production > Model / Architecture > Reasoning and test-time compute > Chain of thought Prompting or training the model to write intermediate steps, which raises output token counts substantially. ### Chart and screen reading **Concept** · Tokenomics > Consumption > Workloads / Use Cases > Vision workloads > Chart and screen reading Extracts structure from charts, screenshots, and interfaces, the reading step behind computer and browser use. ### Chat / assistant **Concept** · Tokenomics > Consumption > Workloads / Use Cases > Chat / assistant Interactive conversation, typically bursty and latency-sensitive. ### Chat templates **Concept** · Tokenomics > Production > Tokenization > Vocabulary > Chat templates The role and turn markup a model was trained to expect, which adds billable tokens to every message. ### Checkpointing **Concept** · Tokenomics > Production > Training > Operations > Checkpointing Periodically saving model state to survive failures and resume runs. ### Chunked prefill **Concept** · Tokenomics > Production > Serving > Inference Optimization > Chunked prefill Breaks long prompt processing into chunks to smooth latency and batching. ### Chunking **Concept** · Tokenomics > Consumption > Prompt / Context Engineering > Context window management > Chunking Splitting long documents into pieces sized for retrieval and for the window. ### Chunking and embedding cost **Concept** · Tokenomics > Consumption > Optimization > AI memory > Retrieval mechanics > Indexing > Chunking and embedding cost Indexing is its own token workload, paid up front and again on every re-index. ### Circuit breakers **Concept** · Tokenomics > Consumption > Access & Delivery > Client-side resilience > Circuit breakers Stopping calls to a failing dependency for a period, which prevents paying repeatedly for requests that cannot succeed. ### Claim chain **Concept** · Tokenomics > Value > Business Outcomes > Value Measurement > Claim chain The how-to workflow, in four parts rather than layers, since layer is reserved for the technical stack. Observe gathers the inputs, all in one unit. Net turns them into one number in that unit. Convert carries the net number into another unit and can repeat. Book names where a dollar-denominated result lands. Many claims stop after Net, and a claim that stops there is complete. As seen in: [AI Value Classification](https://www.tokeneconomics.com/projects/ai-value-classification/) ### Client-side resilience **Concept** · Tokenomics > Consumption > Access & Delivery > Client-side resilience How callers behave when limits and failures are hit, which decides whether pressure is absorbed or amplified. ### Cluster goodput **KPI** · Tokenomics > Production > Training > Operations > Cluster goodput The share of allocated cluster time that produced usable training progress. ### Code / structured text density **Concept** · Tokenomics > Production > Tokenization > Efficiency > Code / structured text density Code and markup tokenize inefficiently, inflating token counts for technical workloads. ### Code benchmarks **Concept** · Tokenomics > Production > Training > Evaluation > Benchmarks > Code benchmarks Function-level sets such as HumanEval and repository-level sets such as SWE-bench, which measure very different things. ### Code generation **Concept** · Tokenomics > Consumption > Workloads / Use Cases > Code generation Producing and editing code, often long-output and context-heavy. ### Coding agents **Concept** · Tokenomics > Consumption > Workloads / Use Cases > Coding agents Autonomous multi-step code work, among the most token-intensive workloads in production. ### Cognitive facet **Concept** · Tokenomics > Value > Business Outcomes > Value Categorization > Employee Engagement > Cognitive facet Fit and skill match: whether the job is changing toward or away from what the person is good at. Part of the construct's full decomposition, documented for the WG record rather than carried on the claim. ### COGS composition **KPI** · Tokenomics > Value > Unit Economics > Margin > COGS composition How much of the cost is model calls, retrieval, storage, and human review, which shows where optimization would actually land. ### Cold Starts **Concept** · Tokenomics > Production > Serving > Batching > Cold Starts The latency and cost of loading a model before it can serve, worst after idle periods. ### Collective communication **Concept** · Tokenomics > Production > Hardware > Capacity > Networking > Collective communication All-reduce and all-to-all operations that dominate distributed cost when models are split across devices. ### Colocation vs owned **Concept** · Tokenomics > Production > Hardware > Capacity > Facilities > Colocation vs owned Renting space versus building it, which sets the capex and commitment profile. ### CombSUM and CombMNZ **Concept** · Tokenomics > Consumption > Optimization > AI memory > Retrieval mechanics > Result fusion > Score-based fusion > CombSUM and CombMNZ The original score-combination family from the early TREC evaluations, where CombSUM adds normalized scores and CombMNZ multiplies that sum by the number of lists a document appeared in, rewarding agreement across retrievers. Named here because today's fusion parameters are rediscoveries of these rather than new inventions. ### Commercial structures **Concept** · Tokenomics > Consumption > Purchase Options > Commercial structures The contract forms buyers purchase under. ### Commitment (Reserved vs on-demand vs spot) **Concept** · Tokenomics > Production > Hardware > Procurement > Commitment (Reserved vs on-demand vs spot) The pricing commitment chosen, trading flexibility against unit cost. ### Commitment risk **Concept** · Tokenomics > Consumption > Purchase Options > Capacity and commitment > Commitment risk Reserved capacity converts a variable cost into a fixed one, which is a saving only if the forecast holds. ### Compaction economics **Concept** · Tokenomics > Consumption > Optimization > Agentic > Waste attribution > Compaction economics Trimming or summarizing accumulated context cuts per-turn cost but can trigger re-read loops that spend more than they save. ### Compression rate **KPI** · Tokenomics > Production > Tokenization > Efficiency > Compression rate The overall ratio of characters to tokens a tokenizer achieves. ### Compute **Concept** · Tokenomics > Production > Hardware > Capacity > Compute Raw processing capability for model math. ### Compute budget **KPI** · Tokenomics > Production > Training > Pretraining > Compute budget Total accelerator time allocated to a run, fixed early and difficult to revise once underway. ### Compute-bound vs memory-bound **Concept** · Tokenomics > Production > Hardware > Performance characterization > Compute-bound vs memory-bound Prefill is usually limited by arithmetic and decode by memory bandwidth, which is why the two phases behave differently. ### Computer and browser use **Concept** · Tokenomics > Consumption > Optimization > Agentic > Tool and protocol layer > Computer and browser use Agents driving a screen or browser directly, which consumes vision tokens per step and is among the most expensive interaction modes. ### Concentration risk **Concept** · Tokenomics > Value > Market Structure > Concentration risk Dependence on a small number of suppliers at several layers of the stack at once. ### Concurrency **KPI** · Tokenomics > Production > Serving > Performance > Concurrency How many requests are in flight at once, the knob that trades per-request latency against total throughput. ### Concurrency caps **KPI** · Tokenomics > Consumption > Access & Delivery > Rate limits / quotas > Concurrency caps Limits on simultaneous in-flight requests, which cap agent fan-out. ### Concurrency profile **Concept** · Tokenomics > Consumption > Workloads / Use Cases > Workload characteristics > Concurrency profile How many requests run at once, the main input to sizing a self-hosted deployment. ### Condorcet Fuse **Concept** · Tokenomics > Consumption > Optimization > AI memory > Retrieval mechanics > Result fusion > Rank-based fusion > Condorcet Fuse An earlier rank-based method that orders documents by pairwise majority vote across the input rankings. RRF was introduced as a simpler alternative and beat it across the original evaluation, which leaves this mostly of historical interest. ### Confidence-based escalation **Concept** · Tokenomics > Consumption > Optimization > Routing > Confidence-based escalation Using the first model's own uncertainty to decide whether a stronger model is needed, the signal most cascades run on. ### Constitutional AI **Concept** · Tokenomics > Production > Training > Post-Training > Alignment > Constitutional AI Aligns to a written set of principles via model-generated critiques and revisions, cutting human labeling and underlying much RLAIF. ### Consumption **Concept** · Tokenomics > Consumption Owns delivering AI services: how much is consumed, where the waste lives, and how to serve efficiently without degrading outcomes. As seen in: [What Is AI Tokenomics?](https://www.tokeneconomics.com/what-is-ai-tokenomics/), [Personas and Operating Model Map](https://www.tokeneconomics.com/projects/personas-and-operating-model-map/) ### Contamination **Concept** · Tokenomics > Production > Training > Evaluation > Contamination Benchmark data leaking into training, which inflates scores without improving capability. ### Context **Concept** · Tokenomics > Production > Model / Architecture > Context How much input a model can attend to at once. ### Context accumulation **Concept** · Tokenomics > Consumption > Optimization > Agentic > Context accumulation Growing context as an agent gathers state, steadily raising cost. ### Context compression / summarization **Concept** · Tokenomics > Consumption > Optimization > AI memory > Context compression / summarization Condensing history to keep context small and cheap. ### Context failure modes **Concept** · Tokenomics > Consumption > Prompt / Context Engineering > Context failure modes Ways that longer context stops helping and starts hurting. ### Context length tiers **Concept** · Tokenomics > Consumption > Purchase Options > Price modifiers > Context length tiers Higher rates once a request exceeds a length threshold, which can make one long call cost more than several short ones. ### Context resend per step **Concept** · Tokenomics > Consumption > Optimization > Agentic > Token amplification > Context resend per step Each turn resends the accumulated history, so input cost grows with the square of conversation length unless cached. ### Context rot **Concept** · Tokenomics > Consumption > Prompt / Context Engineering > Context failure modes > Context rot Accumulated stale or contradictory context degrading answers over a long session. ### Context window length **KPI** · Tokenomics > Production > Model / Architecture > Context > Context window length The maximum tokens a model can process in one request, bounding prompt and memory cost. As seen in: [What Is AI Tokenomics?](https://www.tokeneconomics.com/what-is-ai-tokenomics/), [Big-T Notation (paper)](https://www.tokeneconomics.com/projects/big-t-notation-paper/) ### Context window management **Concept** · Tokenomics > Consumption > Prompt / Context Engineering > Context window management Deciding what to include or drop to stay within budget and window limits. ### Contextual retrieval **Concept** · Tokenomics > Consumption > Optimization > AI memory > Retrieval mechanics > Indexing > Contextual retrieval Prepending a short generated summary of surrounding context to each chunk before indexing, which improves recall and adds a one-off generation cost per chunk. ### Continued pretraining **Concept** · Tokenomics > Production > Training > Post-Training > Fine Tuning > Continued pretraining Further general training on domain data before any task-specific step. ### Continuous batching **Concept** · Tokenomics > Production > Serving > Batching > Continuous batching Dynamically adds and removes requests mid-flight to maximize throughput. ### Convert **Concept** · Tokenomics > Value > Business Outcomes > Value Measurement > Claim chain > Convert The optional, repeatable hop from a net observation into hours, headcount, or money. Conversion operates on the net and never on the gross; each hop carries its own confidence and confidence compounds downward; the claim names its hop count, and a claim beyond the hop threshold the organization sets is reported, not booked. Defensibility lives here, not at the destination. As seen in: [AI Value Classification](https://www.tokeneconomics.com/projects/ai-value-classification/) ### Cooling **Concept** · Tokenomics > Production > Hardware > Capacity > Power > Cooling Heat removal systems required to keep dense accelerators running. As seen in: [The Seven-Layer Tokenomics Stack (paper) (L1)](https://www.tokeneconomics.com/projects/the-seven-layer-tokenomics-stack-paper/) ### Coordination overhead **Concept** · Tokenomics > Consumption > Optimization > Agentic > Sub-agents / orchestration > Coordination overhead Tokens spent on briefing, handoffs, and duplicated context beyond the work performed by the agents themselves. ### Corpus fit **Concept** · Tokenomics > Consumption > Optimization > AI memory > Knowledge injection patterns > CAG (Cache-Augmented Generation) > Corpus fit Viable only when the knowledge set fits inside the context window, which makes context length the gating constraint rather than index size. ### Cost Avoidance **KPI** · Tokenomics > Value > Business Outcomes > Value Categorization > Cost Avoidance Spend that does not occur. Spend that went down is visible on an invoice, spend never incurred is a counterfactual. Spans most of the range by itself. As seen in: [AI Value Classification](https://www.tokeneconomics.com/projects/ai-value-classification/) ### Cost boundary **Concept** · Tokenomics > Value > Business Outcomes > Value Measurement > Cost boundary A value claim nets the costs the AI creates inside the workload: review, triage, rework, supervision, escalation. It does not net the costs the AI consumes, which are TCA under the Business Case and compose later. The test is the baseline's own unit: if the baseline was 45 minutes of human review, triage time is human review time and nets before conversion; the GPU bill is not, so it belongs to TCA. As seen in: [AI Value Classification](https://www.tokeneconomics.com/projects/ai-value-classification/) ### Cost denominators **Concept** · Tokenomics > Value > Unit Economics > Cost denominators What the spend is divided by, which is the choice that determines what the metric can tell you. ### Cost drivers **Concept** · Tokenomics > Value > Unit Economics > Cost drivers The operational variables that move the unit number. ### Cost inflation exposure **Concept** · Tokenomics > Value > Monetization > Margin risk > Cost inflation exposure Committed customer pricing meeting rising provider or infrastructure cost. ### Cost of evaluation **Concept** · Tokenomics > Production > Training > Evaluation > Cost of evaluation Evaluation is itself a token workload, and thorough evaluation of a frontier model is not cheap. ### Cost pass-through vs absorption **Concept** · Tokenomics > Value > Monetization > Metering design > Cost pass-through vs absorption Whether upstream price changes reach the customer, which decides who carries provider pricing risk. ### Cost per agent run **KPI** · Tokenomics > Value > Unit Economics > Cost denominators > Cost per agent run Spend for one complete autonomous task, the natural unit once workflows stop being single calls. ### Cost per outcome **KPI** · Tokenomics > Value > Business Outcomes > Outcome KPIs > Cost per outcome Total cost divided by completed business results that succeeded, not by every attempt. The natural bridge from unit economics into business outcomes; attempts that failed belong in cost per task. ### Cost per request **KPI** · Tokenomics > Value > Unit Economics > Cost denominators > Cost per request Spend per API call, which hides wide variation in request size. ### Cost per session **KPI** · Tokenomics > Value > Business Outcomes > Outcome KPIs > Cost per session Spend across a full user interaction, the first denominator that reflects conversion rather than infrastructure. ### Cost per task **KPI** · Tokenomics > Consumption > Optimization > Model Selection > Cost per task Total cost divided by every attempt at a real task, successes and failures alike, so retries are priced in rather than averaged away. Not just per-token price; defining a task comparable across teams remains open. ### Cost per token **KPI** · Tokenomics > Value > Unit Economics > Cost denominators > Cost per token The rawest measure, useful for comparing infrastructure and nearly useless for comparing products. As seen in: [What Is AI Tokenomics?](https://www.tokeneconomics.com/what-is-ai-tokenomics/), [The Seven-Layer Tokenomics Stack (L1)](https://www.tokeneconomics.com/projects/the-seven-layer-tokenomics-stack/) ### Cost per user **KPI** · Tokenomics > Value > Business Outcomes > Outcome KPIs > Cost per user Spend per active user over a period, the denominator that most directly connects usage to business metrics. ### Cost-based routing **Concept** · Tokenomics > Consumption > Optimization > Routing > Cost-based routing Choosing among eligible models by current price and expected quality, the comparison that cost per task is meant to settle rather than the per-token price sheet. ### CPU and GPU co-execution **Concept** · Tokenomics > Production > Serving > Inference Optimization > Memory hierarchy and offloading > CPU and GPU co-execution Running some experts or layers on the host processor rather than moving their weights, worthwhile when host compute is cheaper than the transfer. ### Credit abstraction **Concept** · Tokenomics > Value > Monetization > Metering design > Credit abstraction Exposing an internal unit rather than raw tokens, which preserves pricing freedom and obscures cost comparison. ### Credits / prepaid **Concept** · Tokenomics > Value > Monetization > Credits / prepaid Selling usage credits customers draw down. ### Credits and prepayment **Concept** · Tokenomics > Consumption > Purchase Options > Commercial structures > Credits and prepayment Money committed up front, often with expiry, which creates use-it-or-lose-it pressure. ### Cross-Engine Support **Concept** · Tokenomics > Production > Serving > KV Cache > Cache Management > Cross-Engine Support Sharing cached state across different inference engines or nodes. ### CTranslate2 **Concept** · Tokenomics > Production > Serving > Inference Engines > Local / edge runtimes > CTranslate2 An efficient inference engine for transformer models on CPU and GPU. ### CUDA **Concept** · Tokenomics > Production > Hardware > Software stack > CUDA NVIDIA's parallel computing platform, the default target for nearly all inference software. ### CUDA graphs **Concept** · Tokenomics > Production > Serving > Inference Optimization > Kernels and compilation > CUDA graphs Replaying a captured sequence of GPU operations to remove per-step launch overhead, which matters most at small batch sizes. ## D ### Data curation **Concept** · Tokenomics > Production > Training > Pretraining > Data curation Selecting and cleaning the training corpus, a major quality and cost lever. ### Data licensing **Concept** · Tokenomics > Production > Training > Pretraining > Data licensing Securing legal rights to training data. ### Data mixing and curriculum **Concept** · Tokenomics > Production > Training > Pretraining > Data mixing and curriculum The proportions and ordering of data sources, which shape capability without changing model size. ### Data parallelism **Concept** · Tokenomics > Production > Training > Distributed Training > Data parallelism Replicates the model across devices, each processing different data. ### Data protection **Concept** · Tokenomics > Consumption > Guardrails > Data protection Controls on what sensitive material reaches a model or a provider. ### Data retention and training use **Concept** · Tokenomics > Consumption > Guardrails > Data protection > Data retention and training use Whether a provider stores prompts or trains on them, a contract term rather than a technical control. ### Datacenter engines **Concept** · Tokenomics > Production > Serving > Inference Engines > Datacenter engines High-throughput engines for server-class GPUs. ### DeepSpeed-MII **Concept** · Tokenomics > Production > Serving > Inference Engines > Datacenter engines > DeepSpeed-MII Microsoft's low-latency inference engine built on DeepSpeed. ### Defined before optimization **Concept** · Tokenomics > Value > Business Outcomes > Value Measurement > Quality Floor > Defined before optimization Routing to cheaper models is impossible until acceptable has a definition, so the floor must exist before cost work begins. ### Delimiters and formatting **Concept** · Tokenomics > Consumption > Prompt / Context Engineering > Prompt design (few-shot, system prompts) > Delimiters and formatting Markup that separates instructions from data, which reduces confusion and injection risk. ### Demand **Concept** · Tokenomics > Consumption > Demand Understanding, measuring, and forecasting the drivers of AI consumption. ### Demand elasticity **Concept** · Tokenomics > Consumption > Demand > Demand elasticity How consumption changes as token prices change, determining whether lower unit prices reduce total spend or stimulate more usage. ### Denominator drift **Concept** · Tokenomics > Value > Unit Economics > Measurement pitfalls > Denominator drift A unit metric improving because the unit changed rather than because anything got cheaper. ### Dense retrieval **Concept** · Tokenomics > Consumption > Optimization > AI memory > Retrieval mechanics > Dense retrieval Matching by embedding similarity, commonly called vector or semantic search, which captures meaning and misses exact identifiers. Every query costs an embedding call, and every document costs one at index time and again on every re-index. ### Dense transformer **Concept** · Tokenomics > Production > Model / Architecture > Family > Dense transformer A standard transformer where every parameter is active on every token. ### Deprecation and repricing **Concept** · Tokenomics > Consumption > Purchase Options > Price dynamics > Deprecation and repricing Existing models being retired or repriced, which can raise the cost of an unchanged workload. ### Depreciation schedules **Concept** · Tokenomics > Production > Hardware > Lifecycle (Intersections with HAM) > Depreciation schedules How accelerator value is written down over its useful life. ### Depth, width, and heads **Concept** · Tokenomics > Production > Model / Architecture > Scale > Depth, width, and heads Layer count, hidden size, and head count, which together set memory footprint and how the model shards across devices. ### Destinations **Concept** · Tokenomics > Value > Business Outcomes > Value Measurement > Claim chain > Book > Destinations The five places a booked number lands: new revenue won, revenue retained, spend removed, spend avoided, capital freed. The financial statements are a closed list, so the set is closed; one win is booked once, in one destination, by one team. ### Detokenization and streaming **Concept** · Tokenomics > Production > Tokenization > Detokenization and streaming Turning generated token ids back into text as output is produced. ### Device memory capacity **KPI** · Tokenomics > Production > Hardware > Capacity > Memory > Device memory capacity How much fits on one accelerator, which decides whether a model needs sharding at all. ### Diffusion language models **Concept** · Tokenomics > Production > Model / Architecture > Family > Diffusion language models Models that refine a whole sequence in parallel steps rather than emitting one token at a time, with a different latency and cost profile. ### Direct API **Concept** · Tokenomics > Consumption > Optimization > Caching > Cache access modes > Direct API Full control of prompt structure and breakpoint placement, with every cache bucket returned on every response and billed line by line: full visibility and full responsibility. ### Disaggregated Caching **Concept** · Tokenomics > Production > Serving > KV Cache > Cache Management > Disaggregated Caching Storing the KV cache separately from compute nodes so it can be shared and scaled. ### Disclosure and reporting **Concept** · Tokenomics > Production > Sustainability > Disclosure and reporting Obligations to report environmental impact, which turn an operational number into a compliance one. ### Distillation **Concept** · Tokenomics > Production > Training > Post-Training > Distillation Training a smaller model to mimic a larger one, lowering serving cost. ### Distributed Training **Concept** · Tokenomics > Production > Training > Distributed Training Splitting training across many devices to handle large models. ### Distribution-Based Score Fusion (DBSF) **Concept** · Tokenomics > Consumption > Optimization > AI memory > Retrieval mechanics > Result fusion > Score-based fusion > Distribution-Based Score Fusion (DBSF) Normalizes each list against its own mean and score spread rather than its endpoints, then sums the rescaled scores, so the size of a lead survives into the final ranking. Qdrant's alternative to RRF, with no tuning parameters of its own. Sources: ### DoRA **Concept** · Tokenomics > Production > Training > Post-Training > Fine Tuning > PEFT / Adapters > DoRA A LoRA refinement that decomposes weights into magnitude and direction for better accuracy. ### DPO **Concept** · Tokenomics > Production > Training > Post-Training > Alignment > DPO Direct preference optimization, aligning models without a separate reward model. ### Draft precision **Concept** · Tokenomics > Production > Serving > Inference Optimization > Speculative decoding > Tuning knobs > Draft precision Quantizing the drafter, for example to FP8, to cut its latency and memory at some cost to acceptance. ### Draft tree width and depth **Concept** · Tokenomics > Production > Serving > Inference Optimization > Speculative decoding > Tuning knobs > Draft tree width and depth The branching budget for tree based drafters, traded against verification batch width. ### Drafter designs **Concept** · Tokenomics > Production > Serving > Inference Optimization > Speculative decoding > Drafter designs Where proposed tokens come from, the main axis of difference between methods. ### Drafter lifecycle cost **Concept** · Tokenomics > Production > Serving > Inference Optimization > Speculative decoding > Economics > Drafter lifecycle cost Trained drafters such as EAGLE heads and MTP modules must be produced, evaluated, and retrained per target model and per quantization, an ongoing cost that sits outside serving. ### Drafter overhead **Concept** · Tokenomics > Production > Serving > Inference Optimization > Speculative decoding > Economics > Drafter overhead Draft weights, draft KV cache, and verification of extra positions all take memory that would otherwise serve more concurrent requests. ### Drafter to target size ratio **KPI** · Tokenomics > Production > Serving > Inference Optimization > Speculative decoding > Tuning knobs > Drafter to target size ratio The cost of proposing relative to the cost of verifying, the first lever on net speedup. ### Drafting **Concept** · Tokenomics > Production > Serving > Inference Optimization > Speculative decoding > Mechanics > Drafting A cheap proposer generates a candidate continuation of k tokens. ### Driver and toolkit compatibility **Concept** · Tokenomics > Production > Hardware > Software stack > Driver and toolkit compatibility Version alignment between driver, toolkit, and framework, a routine source of deployment failure that costs engineer time rather than compute. ### Dynamic speculation **Concept** · Tokenomics > Production > Serving > Inference Optimization > Speculative decoding > Tuning knobs > Dynamic speculation Varying speculation length at runtime with load or drafter confidence, so speculation backs off under peak traffic. ### Dynamo KV Block Manager (KVBM) **Concept** · Tokenomics > Production > Serving > KV Cache > Cache Management > Hierarchical KV cache > Hierarchical cache managers > Dynamo KV Block Manager (KVBM) NVIDIA Dynamo's memory manager for KV cache, which separates block management from the inference runtime and tiers blocks across GPU memory, host memory, local SSD, and remote storage over NIXL. Sources: ## E ### EAGLE **Concept** · Tokenomics > Production > Serving > Inference Optimization > Speculative decoding > Drafter designs > Feature-level drafting (EAGLE family) > EAGLE Runs draft autoregression at the feature level, reusing top layer features from the target model. ### EAGLE-2 **Concept** · Tokenomics > Production > Serving > Inference Optimization > Speculative decoding > Drafter designs > Feature-level drafting (EAGLE family) > EAGLE-2 Adds context aware dynamic draft trees, so tree shape adapts to how confident the drafter is. ### EAGLE-3 **Concept** · Tokenomics > Production > Serving > Inference Optimization > Speculative decoding > Drafter designs > Feature-level drafting (EAGLE family) > EAGLE-3 Drops the feature prediction constraint in favour of direct token prediction and fuses low, mid, and high layer features via training-time test, which lets draft quality keep improving as draft training data scales. Sources: ### Economics **Concept** · Tokenomics > Production > Serving > Inference Optimization > Speculative decoding > Economics Why speculation matters to cost and not only to latency. ### Edge and on-device **Concept** · Tokenomics > Consumption > Access & Delivery > Edge and on-device Running small models locally, which removes per-token cost and network latency at the price of capability. ### Effect on billing **Concept** · Tokenomics > Production > Serving > Inference Optimization > Speculative decoding > Economics > Effect on billing Speculation does not change how many tokens a buyer is billed for; it changes what those tokens cost to produce, so the benefit lands with whoever owns the serving hardware. ### Effective vs advertised context **Concept** · Tokenomics > Production > Model / Architecture > Context > Effective vs advertised context Models often degrade well before their stated limit, so the usable window is an empirical question rather than a spec. ### Efficiency **Concept** · Tokenomics > Production > Tokenization > Efficiency How compactly a tokenizer encodes text, which directly sets token counts and cost. ### Efficiency ratios **Concept** · Tokenomics > Value > Unit Economics > Efficiency ratios Production-side measures of how much output a resource yields. ### Efficiency signals **KPI** · Tokenomics > Consumption > Observability > Signals > Efficiency signals Cache hit rate, acceptance rate, and batch occupancy, the numbers that explain why cost per token is what it is. ### Elastic memory allocation **Concept** · Tokenomics > Production > Serving > Inference Optimization > Memory hierarchy and offloading > Elastic memory allocation Resizing the split between weight cache and KV cache while the engine runs, which avoids a restart when available memory changes. ### Embedding caching **Concept** · Tokenomics > Consumption > Optimization > Caching > Embedding caching Storing computed embeddings so unchanged documents are not re-embedded on every index run. ### Embedding dimensions **Concept** · Tokenomics > Consumption > Optimization > AI memory > Retrieval mechanics > Dense retrieval > Embedding dimensions Vector width, which sets storage and search cost across the whole corpus and is fixed at index time. ### Embedding model **Concept** · Tokenomics > Production > Model / Architecture > Model variants > Embedding model Produces vectors rather than text, priced per input token with no output cost, and the workhorse of retrieval. ### Embedding model versioning **Concept** · Tokenomics > Consumption > Optimization > AI memory > Retrieval mechanics > Dense retrieval > Embedding model versioning Changing the embedding model invalidates every vector in the index, so an upgrade means re-embedding the whole corpus. That re-indexing bill is the switching cost of dense retrieval and it is rarely priced in at selection time. ### Embeddings and indexing **Concept** · Tokenomics > Consumption > Workloads / Use Cases > Embeddings and indexing Producing vectors rather than text, priced differently and dominated by corpus size. ### Embodied impact **Concept** · Tokenomics > Production > Sustainability > Embodied impact The footprint of manufacturing and disposing of hardware, separate from the cost of running it. ### Employee Engagement **Concept** · Tokenomics > Value > Business Outcomes > Value Categorization > Employee Engagement Candidate tenth category, renamed from Job Satisfaction to name the actual I-O psychology construct. An AI program changes engagement along three facets, of which the claim carries only the behavioral top line: retention rates, productivity, and counterproductive work behaviors such as withdrawal or sabotage, measured with engagement surveys and incident reports. No forced conversion: the defensible move is to report it, not book it. As seen in: [AI Value Classification](https://www.tokeneconomics.com/projects/ai-value-classification/) ### End-to-end latency **KPI** · Tokenomics > Production > Serving > Performance > End-to-end latency Total time to complete a request, the sum of queueing, prefill, and all decode steps. ### Energy **Concept** · Tokenomics > Production > Hardware > Capacity > Power > Energy Electricity consumed, a growing share of total token cost. ### Energy per token **KPI** · Tokenomics > Production > Sustainability > Energy per token Electricity consumed to produce a unit of output, the bridge between infrastructure efficiency and environmental reporting. ### Engine metrics **Concept** · Tokenomics > Consumption > Observability > Telemetry sources > Engine metrics Counters emitted by the inference server, the only source that knows about cache hits, queue depth, and acceptance rates. ### Engine support **Concept** · Tokenomics > Production > Serving > Inference Optimization > Speculative decoding > Engine support Which serving stacks expose which methods, the practical constraint on what can be adopted. ### Error and retry rates **KPI** · Tokenomics > Consumption > Observability > Signals > Error and retry rates Failed and repeated work, which shows up as cost long before it shows up as a quality complaint. ### Evaluation **Concept** · Tokenomics > Production > Training > Evaluation Measuring whether a trained model is actually good enough to deploy. ### Eviction policies **Concept** · Tokenomics > Production > Serving > KV Cache > Eviction policies Rules for which cache entries to drop when memory fills. ### Execution topology **Concept** · Tokenomics > Production > Serving > Inference Optimization > Speculative decoding > Execution topology How drafting and verification are placed and overlapped across hardware; the newer frontier, since drafter quality alone is hitting diminishing returns. ### ExLlamaV2 **Concept** · Tokenomics > Production > Serving > Inference Engines > Local / edge runtimes > ExLlamaV2 A fast runtime for quantized models on consumer GPUs. ### Expert parallelism **Concept** · Tokenomics > Production > Model / Architecture > Family > Mixture of Experts (MoE) > Expert parallelism Spreading experts across devices, which turns routing into network traffic and makes interconnect a cost factor. ### Expert residency and caching **Concept** · Tokenomics > Production > Serving > Inference Optimization > Memory hierarchy and offloading > Expert residency and caching Which MoE experts stay resident on the accelerator, with the rest fetched or computed elsewhere as routing demands them. ### Exponential backoff **Concept** · Tokenomics > Consumption > Access & Delivery > Client-side resilience > Exponential backoff Increasing the wait between retries, the standard response to rate limiting and the one clients most often get wrong. ### Extraction / classification **Concept** · Tokenomics > Consumption > Workloads / Use Cases > Extraction / classification Pulling structured data or labels from text, usually short outputs. ## F ### Facilities **Concept** · Tokenomics > Production > Hardware > Capacity > Facilities The physical datacenter housing the hardware. ### Failed runs and restarts **Concept** · Tokenomics > Production > Training > Operations > Failed runs and restarts Work lost to divergence or hardware failure, a real line item that rarely appears in published training costs. ### Failure / RMA rates **KPI** · Tokenomics > Production > Hardware > Lifecycle (Intersections with HAM) > Failure / RMA rates How often units fail and must be returned, reducing effective capacity. ### Family **Concept** · Tokenomics > Production > Model / Architecture > Family The high-level architectural class of the model. ### Feature-level drafting (EAGLE family) **Concept** · Tokenomics > Production > Serving > Inference Optimization > Speculative decoding > Drafter designs > Feature-level drafting (EAGLE family) Drafting autoregressively over the target model's hidden features rather than over tokens alone, which raises acceptance for a small drafter. ### Few-shot examples **Concept** · Tokenomics > Consumption > Prompt / Context Engineering > Prompt design (few-shot, system prompts) > Few-shot examples Worked examples included in context, which raise quality and input token count together. ### Fine Tuning **Concept** · Tokenomics > Production > Training > Post-Training > Fine Tuning Adapting a pretrained model to a specific task or domain. ### Fireworks / Together / Baseten **Concept** · Tokenomics > Production > Serving > Inference Engines > Packaged / managed > Fireworks / Together / Baseten Managed inference providers that host open models behind an API. ### First-party agent tools **Concept** · Tokenomics > Consumption > Optimization > Caching > Cache access modes > First-party agent tools The provider's own coding or coworking harness manages breakpoints automatically and exposes session-level token dashboards, but its dollar figures are computed locally at list rates, so they show shape rather than the real bill. ### Fixed cost amortization **Concept** · Tokenomics > Value > Unit Economics > Measurement pitfalls > Fixed cost amortization Training, evaluation, and platform cost spread over volume, which makes early unit costs look worse than the steady state. ### FlashAttention **Concept** · Tokenomics > Production > Model / Architecture > Attention > FlashAttention A memory-efficient attention kernel that speeds training and inference without changing results. ### FlashInfer **Concept** · Tokenomics > Production > Serving > Inference Optimization > Kernels and compilation > Attention backends > FlashInfer A kernel library providing attention implementations tuned for serving workloads. ### Floor versus work quality **Concept** · Tokenomics > Value > Business Outcomes > Value Measurement > Quality Floor > Floor versus work quality The floor is a binary pass signal for the AI output; the Product Quality category means the quality of the work the business ships. The rename from Output Quality exists to keep the two from talking past each other. ### FOCUS **Concept** · Tokenomics > Consumption > Observability > Standards / Open Specifications > FOCUS The open specification for normalizing billing and usage data across providers, which makes cross-vendor comparison possible. As seen in: [What FOCUS 1.5 Does for AI Cost](https://www.tokeneconomics.com/projects/what-1-5-does-for-ai-cost-and-what-it-does-not/) ### FP8 **Concept** · Tokenomics > Production > Training > Quantization > Precision formats > FP8 8-bit floating point, increasingly used to speed inference and training. ### FR-Spec **Concept** · Tokenomics > Production > Serving > Inference Optimization > Speculative decoding > Drafter designs > Feature-level drafting (EAGLE family) > FR-Spec Frequency-ranked speculative sampling, limiting the drafting head to a high-frequency vocab subset to cut draft cost on large vocabularies without changing output. ### FreeToken **Concept** · Tokenomics > Production > Serving > Inference Engines > Local / edge runtimes > FreeToken An edge-native MoE engine treating GPU, CPU, host memory, and interconnect as one pool to serve models larger than accelerator memory. ### Friendli **Concept** · Tokenomics > Production > Serving > Inference Engines > Packaged / managed > Friendli A managed inference service focused on high efficiency. ### Full fine-tune **Concept** · Tokenomics > Production > Training > Post-Training > Fine Tuning > Full fine-tune Updating all weights, the most capable but most expensive adaptation. ### Function calling / tool use **Concept** · Tokenomics > Consumption > Optimization > Agentic > Tool and protocol layer > Function calling / tool use The model emitting a structured call the application executes, the mechanism underneath every agent framework. ### Fused kernels **Concept** · Tokenomics > Production > Serving > Inference Optimization > Kernels and compilation > Fused kernels Combining several operations into one to avoid repeated trips to memory. ### Fusion across query variants **Concept** · Tokenomics > Consumption > Optimization > AI memory > Retrieval mechanics > Result fusion > Fusion across query variants Applying the same merge machinery to lists from several rewritten versions of one question rather than from several retrievers. It raises recall on short or ambiguous queries and turns one user question into several searches plus a generation, which makes it a token amplifier rather than a saving. ### Fusion choice and evaluation **Concept** · Tokenomics > Consumption > Optimization > AI memory > Retrieval mechanics > Result fusion > Fusion choice and evaluation Which method wins is a property of the corpus and the query mix rather than a general fact, so the choice can only be settled against a set of queries with known-good answers. Building and maintaining that labeled set is the actual cost here, paid once in human time rather than per query in tokens. ## G ### Gateway **Concept** · Tokenomics > Consumption > Optimization > Routing > Gateway The superset that houses proxy and router functions and adds organization-level policy (which teams may use which models), full observability, and resiliency such as multi-provider failover. You add routing to a gateway, not a gateway to a router. ### Generational uplift **Concept** · Tokenomics > Production > Hardware > Capacity > Compute > Generational uplift Each accelerator generation changes performance per dollar and per watt, which resets cost models on a hardware cycle. ### GGUF **Concept** · Tokenomics > Production > Training > Quantization > Algorithms > GGUF A file format and quantization scheme popular for running models locally. ### Golden datasets **Concept** · Tokenomics > Consumption > Guardrails > Production evals > Golden datasets A curated set of cases with known good answers, the fixed reference any change is measured against. ### Goodput **KPI** · Tokenomics > Production > Serving > Performance > Goodput Throughput counting only work that met its latency target and was actually used, the honest version of the metric. ### GPTQ **Concept** · Tokenomics > Production > Training > Quantization > Algorithms > GPTQ A one-shot post-training method that quantizes weights layer by layer with low error. ### GPU Capacity **Concept** · Tokenomics > Production > Hardware > Capacity > Compute > GPU Capacity The number and type of GPUs available, the central constraint on throughput. ### GPU quotas / allocation **KPI** · Tokenomics > Production > Hardware > Procurement > GPU quotas / allocation Limits vendors place on how many accelerators a customer can access. ### GPU sharing (MIG, time-slicing) **Concept** · Tokenomics > Production > Hardware > Capacity > Compute > GPU sharing (MIG, time-slicing) Partitioning one accelerator across workloads, either as isolated instances or by interleaving, to raise utilization on small models. ### Gradient checkpointing and accumulation **Concept** · Tokenomics > Production > Training > Distributed Training > Gradient checkpointing and accumulation Trades recomputation and extra steps for lower peak memory, letting bigger batches fit on the same hardware. ### Grammar-constrained decoding **Concept** · Tokenomics > Consumption > Prompt / Context Engineering > Structured outputs (JSON mode, grammars) > Grammar-constrained decoding Restricting each next token to those a formal grammar allows, which makes malformed output impossible rather than unlikely. ### Granularity **Concept** · Tokenomics > Production > Training > Quantization > Scheme > Granularity Whether scales are shared per tensor, per channel, or per small block, trading accuracy against overhead. ### Graph compilation **Concept** · Tokenomics > Production > Serving > Inference Optimization > Kernels and compilation > Graph compilation Ahead-of-time compilation of the model graph, trading startup time for steady-state speed. ### GraphRAG **Concept** · Tokenomics > Consumption > Optimization > AI memory > Knowledge injection patterns > GraphRAG Builds a knowledge graph and community summaries over the corpus so questions spanning many documents can be answered, at a much higher one-off indexing cost. ### Greedy decoding **Concept** · Tokenomics > Consumption > Optimization > Sampling and decoding controls > Greedy decoding Always taking the highest probability token, deterministic and prone to repetition. ### Grid and interconnection constraints **Concept** · Tokenomics > Production > Hardware > Capacity > Power > Grid and interconnection constraints Time to secure power at a site, often the true lead time behind apparent chip scarcity. ### Gross & contribution margin **KPI** · Tokenomics > Value > Unit Economics > Margin > Gross & contribution margin The profit left after token and serving costs. ### Groundedness and citation checking **Concept** · Tokenomics > Consumption > Guardrails > Groundedness and citation checking Verifying that claims trace to retrieved sources, which costs additional calls but converts an unmeasurable risk into a measurable one. ### Grouped-Query / Multi-Query Attention **Concept** · Tokenomics > Production > Model / Architecture > Attention > Grouped-Query / Multi-Query Attention Shares key and value heads to shrink the KV cache and speed decoding. ### GRPO **Concept** · Tokenomics > Production > Training > Post-Training > Alignment > GRPO A group-relative policy optimization variant that drops the separate value model, widely used for reasoning training. ### Guardrail and classifier models **Concept** · Tokenomics > Production > Model / Architecture > Model variants > Guardrail and classifier models Small models that screen input or output, adding a fixed per-request cost to every call they protect. ### Guardrails **Concept** · Tokenomics > Consumption > Guardrails Controls that keep output safe and correct, some of which cost extra tokens. Consumption owns enforcement and the measurement machinery; what counts as acceptable is the Quality Floor, owned by Value. As seen in: [The Seven-Layer Tokenomics Stack (paper) (L5)](https://www.tokeneconomics.com/projects/the-seven-layer-tokenomics-stack-paper/) ## H ### Hallucination control **Concept** · Tokenomics > Consumption > Guardrails > Hallucination control Techniques to reduce confident but wrong outputs. ### Hardware **Concept** · Tokenomics > Production > Hardware The physical infrastructure that produces tokens. ### Hardware cost pass-through **Concept** · Tokenomics > Consumption > Purchase Options > Price dynamics > Hardware cost pass-through Changes in accelerator, power, and facility cost eventually reaching published rates. ### Harm from misses **Concept** · Tokenomics > Value > Business Outcomes > Value Measurement > Claim chain > Net > Harm from misses The Quality Floor makes bad output not count; it does not make bad output cost nothing. An incorrect resolution or a compliance miss is net negative, not merely zero, and the damage is subtracted here in the baseline's unit. Most claims carry zero on this line; risk-bearing workloads do not. As seen in: [AI Value Classification](https://www.tokeneconomics.com/projects/ai-value-classification/) ### Harness overhead **Concept** · Tokenomics > Consumption > Optimization > Agentic > Waste attribution > Harness overhead Tokens the scaffold consumes before task work begins, including system prompts, tool schemas, and coordination instructions. ### Harness sensitivity **Concept** · Tokenomics > Consumption > Workloads / Use Cases > Coding agents > Harness sensitivity How the surrounding agent scaffold changes the same model's cost and success rate, making the harness a first-class cost variable. ### HASS **Concept** · Tokenomics > Production > Serving > Inference Optimization > Speculative decoding > Drafter designs > Feature-level drafting (EAGLE family) > HASS Harmonized Speculative Sampling, an EAGLE-based drafter that aligns training with multi-step inference to raise acceptance length without changing output. ### Heads and objectives **Concept** · Tokenomics > Production > Model / Architecture > Heads and objectives What the model is trained to predict, which determines what can be reused at serving time. ### Health checks and draining **Concept** · Tokenomics > Production > Serving > Reliability > Health checks and draining Detecting bad workers and removing them without dropping in-flight requests. ### Held-out and internal evals **Concept** · Tokenomics > Production > Training > Evaluation > Held-out and internal evals Task-specific test sets built from the actual workload, the only evidence that transfers to a deployment decision. ### HiCache **Concept** · Tokenomics > Production > Serving > KV Cache > Cache Management > Hierarchical KV cache > Hierarchical cache managers > HiCache SGLang's hierarchical cache, extending RadixAttention across GPU, host, and an optional external storage tier, with the external tier exposed through one pluggable interface. Sources: ### Hidden prompt content **Concept** · Tokenomics > Consumption > Optimization > Caching > Provider-side prompt caching > Hidden prompt content A hosted harness inserts its own system content ahead of the user prompt, occupying the head of the prefix and often excluded from the minimum cacheable length, while reasoning and tool-output tokens cache inside earlier turns and bill as input when read back. ### Hierarchical cache managers **Concept** · Tokenomics > Production > Serving > KV Cache > Cache Management > Hierarchical KV cache > Hierarchical cache managers The layer that decides which tier a block lives on and moves it between them. Distinct from the storage tier itself: a manager is chosen with the engine, a store is chosen with the infrastructure. ### Hierarchical KV cache **Concept** · Tokenomics > Production > Serving > KV Cache > Cache Management > Hierarchical KV cache Placing KV cache across tiers, typically GPU memory, host memory, and shared storage, so cache survives beyond what fits on the accelerator. The trade is storage and network bandwidth against recomputed prefill, which makes the fetch-or-recompute decision the economic core of the branch. Distinct from weight offloading under Memory hierarchy and offloading, which moves model weights rather than cache. Sources: ### Hierarchical summarization indexes (RAPTOR) **Concept** · Tokenomics > Consumption > Optimization > AI memory > Retrieval mechanics > Indexing > Hierarchical summarization indexes (RAPTOR) Recursively clustering and summarizing the corpus into a tree, so queries can be answered at the right level of abstraction. ### HuggingFace Tokenizers **Concept** · Tokenomics > Production > Tokenization > Implementations > HuggingFace Tokenizers A widely used Rust-backed tokenizer library with many pretrained vocabularies. ### Human review **Concept** · Tokenomics > Consumption > Guardrails > Human review People checking model output, usually the most expensive component per item and the benchmark others are measured against. ### Hybrid CAG and RAG **Concept** · Tokenomics > Consumption > Optimization > AI memory > Knowledge injection patterns > Hybrid CAG and RAG Caches a stable core of knowledge and retrieves only for queries falling outside it, keeping the latency benefit while bounding staleness. ### Hybrid pricing **Concept** · Tokenomics > Value > Monetization > Hybrid pricing A base subscription plus metered overage, the common compromise between predictable revenue and cost coverage. ### Hybrid search **Concept** · Tokenomics > Consumption > Optimization > AI memory > Retrieval mechanics > Hybrid search Running dense and sparse retrieval over the same query and merging the two result lists, usually the strongest default because their failure modes differ. The merge itself is a separate choice, covered under Result fusion. ### HyDE **Concept** · Tokenomics > Consumption > Optimization > AI memory > Retrieval mechanics > Query preparation > HyDE Generating a hypothetical answer and retrieving against its embedding, so the search vector matches document style rather than question style. ### Hyperparameter tuning **Concept** · Tokenomics > Production > Training > Operations > Hyperparameter tuning Searching over training settings to improve results. ## I ### Idempotency keys **Concept** · Tokenomics > Consumption > Access & Delivery > Client-side resilience > Idempotency keys Identifiers that let a retried request be recognised as the same call, avoiding duplicate work and duplicate billing. ### Idle and reserved cost **Concept** · Tokenomics > Value > Unit Economics > Measurement pitfalls > Idle and reserved cost Capacity paid for but unused has to land somewhere, and burying it inflates the apparent cost of active work. ### Image captioning **Concept** · Tokenomics > Consumption > Workloads / Use Cases > Vision workloads > Image captioning A text description of an image, short on output with cost dominated by input tiles. ### Image patches / tiles **Concept** · Tokenomics > Production > Tokenization > Multimodal tokenization > Image patches / tiles Images are split into patches or tiles, each costing a set number of tokens. ### Implementations **Concept** · Tokenomics > Production > Tokenization > Implementations Concrete libraries that perform tokenization. ### Independent draft model **Concept** · Tokenomics > Production > Serving > Inference Optimization > Speculative decoding > Drafter designs > Independent draft model A separate small model from the same family, simple to deploy but costs extra weights, extra memory, and its own autoregressive latency. ### Indexing **Concept** · Tokenomics > Consumption > Optimization > AI memory > Retrieval mechanics > Indexing Work done once over the corpus and repeated on every rebuild, paid in full before a single query arrives. ### Inference Engines **Concept** · Tokenomics > Production > Serving > Inference Engines Software that runs models to serve tokens. As seen in: [The Seven-Layer Tokenomics Stack (paper) (L3)](https://www.tokeneconomics.com/projects/the-seven-layer-tokenomics-stack-paper/) ### Inference Optimization **Concept** · Tokenomics > Production > Serving > Inference Optimization Techniques that lower the cost and latency of generating tokens. ### Infrastructure price inflation **Concept** · Tokenomics > Value > Market Structure > Infrastructure price inflation Rising cost of the underlying compute and facilities, which does not always appear in per-token rates but eventually constrains them. ### Input embeddings and LM head **Concept** · Tokenomics > Production > Model / Architecture > Heads and objectives > Input embeddings and LM head The input table mapping token ids to vectors and the output projection back to vocabulary, sized vocab times hidden dimension and sometimes sharing weights to save memory. ### Input to output ratio **KPI** · Tokenomics > Consumption > Workloads / Use Cases > Workload characteristics > Input to output ratio Whether a workload is dominated by reading or writing, which decides whether caching or output control is the lever. ### Input to output ratio **KPI** · Tokenomics > Value > Unit Economics > Cost drivers > Input to output ratio Which side of the meter dominates, which determines whether prompt or generation is worth optimizing. ### Input vs output token pricing **Concept** · Tokenomics > Consumption > Purchase Options > Price axes > Input vs output token pricing Input and output tokens are billed at different rates, usually output higher. ### Instruction-tuned model **Concept** · Tokenomics > Production > Model / Architecture > Model variants > Instruction-tuned model A base model post-trained to follow instructions and hold a conversation, the default for application use. ### INT4 **Concept** · Tokenomics > Production > Training > Quantization > Precision formats > INT4 4-bit integer precision, maximizing memory savings at some quality cost. ### INT8 **Concept** · Tokenomics > Production > Training > Quantization > Precision formats > INT8 8-bit integer precision, a widely supported inference quantization. ### Intangible Outcomes **Concept** · Tokenomics > Value > Business Outcomes > Intangible Outcomes Candidate eleventh category alongside the ten: real effects that resist any defensible number, such as learning, option value, and better decisions. The category exists so these can be named and reported rather than dropped or inflated. Same rule as Employee Engagement: report them, do not book them, and never let an uncounted line turn into an uncounted budget. ### Interconnect (NVLink, InfiniBand) **Concept** · Tokenomics > Production > Hardware > Capacity > Networking > Interconnect (NVLink, InfiniBand) High-speed links between accelerators that enable efficient model parallelism. ### Interconnect bandwidth as the constraint **Concept** · Tokenomics > Production > Serving > Inference Optimization > Memory hierarchy and offloading > Interconnect bandwidth as the constraint Host link throughput setting the ceiling once weights no longer fit, the binding limit in offloaded serving. ### Internal product workload **Concept** · Tokenomics > Value > Business Outcomes > Value Measurement > Product versus internal posture > Internal product workload AI-enabled features built and packaged like a product for internal users. The goal is productivity, so it is measured as Internal AI no matter how well it is packaged. ### Inverted index **Concept** · Tokenomics > Consumption > Optimization > AI memory > Retrieval mechanics > Sparse retrieval > Inverted index The term-to-document map sparse retrieval runs on. Cheap to build and update relative to a vector index, and it stores no floating-point vectors, so its footprint over the same corpus is a fraction of a dense index. ## J ### Jailbreaking **Concept** · Tokenomics > Consumption > Guardrails > Adversarial input > Jailbreaking Prompts crafted to bypass a model's trained restrictions. ### JSON mode **Concept** · Tokenomics > Consumption > Prompt / Context Engineering > Structured outputs (JSON mode, grammars) > JSON mode Forcing syntactically valid JSON output, the most common structured mode. ## K ### K-quants **Concept** · Tokenomics > Production > Training > Quantization > Algorithms > K-quants Mixed-precision block quantization variants within the GGUF family, offering a spread of size and quality points. ### Kernels and compilation **Concept** · Tokenomics > Production > Serving > Inference Optimization > Kernels and compilation The low-level code that actually executes the model, often worth more than any algorithmic change. ### Knowledge and reasoning benchmarks **Concept** · Tokenomics > Production > Training > Evaluation > Benchmarks > Knowledge and reasoning benchmarks Standard multiple-choice and problem-solving sets such as MMLU and GPQA, used for coarse capability comparison. ### Knowledge in weights **Concept** · Tokenomics > Consumption > Optimization > AI memory > Knowledge injection patterns > Knowledge in weights Training knowledge into the model through fine-tuning or continued pretraining, which removes per-query context cost entirely and replaces it with retraining cost on every update. ### Knowledge injection patterns **Concept** · Tokenomics > Consumption > Optimization > AI memory > Knowledge injection patterns How knowledge reaches the model at query time, the architectural choice that sets both latency and the shape of the per-query bill. ### KServe **Concept** · Tokenomics > Production > Serving > Inference Engines > Serving orchestration > KServe A Kubernetes-native platform for serving models. ### KV Cache **Concept** · Tokenomics > Production > Serving > KV Cache Stored attention keys and values that let the model skip recomputing past tokens, the dominant inference memory cost. As seen in: [What Is AI Tokenomics?](https://www.tokeneconomics.com/what-is-ai-tokenomics/), [Cache Explainer](https://www.tokeneconomics.com/projects/cache-explainer/#tcTheOneThing) Sources: ### KV cache quantization **Concept** · Tokenomics > Production > Training > Quantization > What gets quantized > KV cache quantization Stores cached keys and values at lower precision, which buys concurrency rather than speed. ### KV cache storage tier **Concept** · Tokenomics > Production > Serving > KV Cache > Cache Management > Hierarchical KV cache > KV cache storage tier A shared, persistent storage layer that holds KV cache beyond device and host memory so any node can retrieve precomputed state instead of re-running prefill, turning cache from a per-node optimization into a fleet-wide stored asset with its own capacity, retention, and breakeven economics. ## L ### Labor Augmentation **KPI** · Tokenomics > Value > Business Outcomes > Value Categorization > Labor Augmentation AI does part of the task and the human stays in the loop. The gain shows up as throughput or hours per unit before it shows up as headcount, measured against the human alone. As seen in: [AI Value Classification](https://www.tokeneconomics.com/projects/ai-value-classification/) ### Labor Automation **KPI** · Tokenomics > Value > Business Outcomes > Value Categorization > Labor Automation AI does the task end to end. Trivial to compute and hardest to defend: needs the best-of-either baseline and a cleared quality gate, and raises who owns the decision once the human is out of the loop. As seen in: [AI Value Classification](https://www.tokeneconomics.com/projects/ai-value-classification/) ### Language disparity (non-English cost premium) **Concept** · Tokenomics > Production > Tokenization > Efficiency > Language disparity (non-English cost premium) Non-English text often uses more tokens per word, making it more expensive to process. ### Late chunking **Concept** · Tokenomics > Consumption > Optimization > AI memory > Retrieval mechanics > Indexing > Late chunking Embedding the full document first and pooling per chunk afterwards, so chunk vectors retain document context. ### Latency percentiles **KPI** · Tokenomics > Production > Serving > Performance > Latency percentiles Median and tail figures such as p50, p95, and p99, which describe the distribution that an average conceals. ### Latency sensitivity **Concept** · Tokenomics > Consumption > Workloads / Use Cases > Workload characteristics > Latency sensitivity How much a delay costs, which sets whether batch pricing is available at all. ### Latency versus throughput **Concept** · Tokenomics > Production > Serving > Inference Optimization > Speculative decoding > Economics > Latency versus throughput Speculation spends extra compute to shorten time per output token, so it improves interactivity first and cost per token only when the hardware was underused. ### Lead times **KPI** · Tokenomics > Production > Hardware > Procurement > Lead times The wait between ordering and receiving scarce accelerators. ### Learned sparse retrieval (SPLADE) **Concept** · Tokenomics > Consumption > Optimization > AI memory > Retrieval mechanics > Sparse retrieval > Learned sparse retrieval (SPLADE) Uses a language model to predict term weights across the whole vocabulary, expanding documents with terms they do not literally contain while still indexing and searching as sparse vectors. It buys semantic matching without abandoning the inverted index, and pays for it with a model pass over every document at index time. Sources: ### License terms **Concept** · Tokenomics > Production > Model / Architecture > Openness > License terms The legal conditions governing commercial use, redistribution, and fine-tuning. ### Lifecycle (Intersections with HAM) **Concept** · Tokenomics > Production > Hardware > Lifecycle (Intersections with HAM) Managing hardware from acquisition through retirement, overlapping with hardware asset management. ### Limits as product tiers **Concept** · Tokenomics > Value > Monetization > Metering design > Limits as product tiers Using rate and volume caps to differentiate plans, which turns a cost control into a packaging decision. ### Linear and sub-quadratic attention **Concept** · Tokenomics > Production > Model / Architecture > Attention > Linear and sub-quadratic attention Reformulations that avoid the full pairwise comparison, trading exactness for cheaper scaling with length. ### Live and streaming video **Concept** · Tokenomics > Consumption > Workloads / Use Cases > Video workloads > Live and streaming video Continuous frame ingestion under a latency budget, where cost accrues per minute of stream rather than per request. ### llama.cpp (Ollama, LM Studio) **Concept** · Tokenomics > Production > Serving > Inference Engines > Local / edge runtimes > llama.cpp (Ollama, LM Studio) A CPU and GPU runtime for running quantized models locally. ### LLM as judge **Concept** · Tokenomics > Consumption > Guardrails > Production evals > LLM as judge Using a model to grade output, which is itself a token workload that scales with traffic. ### llm-d **Concept** · Tokenomics > Production > Serving > Inference Engines > Serving orchestration > llm-d A Kubernetes-native stack for distributed LLM serving, including cache-aware routing across replicas. ### LMCache **Concept** · Tokenomics > Production > Serving > KV Cache > Cache Management > Hierarchical KV cache > Hierarchical cache managers > LMCache An external KV cache layer that stores and reuses cache across engine instances and storage tiers. Sources: ### LMDeploy **Concept** · Tokenomics > Production > Serving > Inference Engines > Datacenter engines > LMDeploy A toolkit for compressing and serving LLMs efficiently. ### Load balancing **Concept** · Tokenomics > Production > Model / Architecture > Family > Mixture of Experts (MoE) > Load balancing Auxiliary objectives or bias terms that stop a few experts absorbing most traffic and stranding the rest. ### Load balancing **Concept** · Tokenomics > Production > Serving > Scaling > Load balancing Distributing requests across workers to avoid hotspots. ### Local / edge runtimes **Concept** · Tokenomics > Production > Serving > Inference Engines > Local / edge runtimes Engines for running models on personal or edge hardware. ### Logit bias **Concept** · Tokenomics > Consumption > Optimization > Sampling and decoding controls > Logit bias Directly raising or lowering the odds of specific tokens, a blunt way to force or forbid particular output. ### Long-context methods **Concept** · Tokenomics > Production > Model / Architecture > Context > Long-context methods Techniques that push usable context beyond the trained window, usually at added compute cost. ### Long-context probes **Concept** · Tokenomics > Production > Training > Evaluation > Benchmarks > Long-context probes Tests such as needle in a haystack retrieval and RULER that measure usable context rather than advertised context. ### Long-context stuffing **Concept** · Tokenomics > Consumption > Optimization > AI memory > Knowledge injection patterns > Long-context stuffing Placing source material directly in the prompt with no caching or retrieval layer, the simplest pattern and the most expensive per query at volume. ### Lookahead / Jacobi decoding **Concept** · Tokenomics > Production > Serving > Inference Optimization > Speculative decoding > Drafter designs > Lookahead / Jacobi decoding Generates and verifies n-grams from parallel Jacobi iterations, with no separate draft model and no extra training. ### LoRA (Low-Rank Adaptation) **Concept** · Tokenomics > Production > Training > Post-Training > Fine Tuning > PEFT / Adapters > LoRA (Low-Rank Adaptation) Trains small low-rank matrices, cutting fine-tuning cost dramatically. ### Lost in the middle **Concept** · Tokenomics > Consumption > Prompt / Context Engineering > Context failure modes > Lost in the middle Information placed mid-context is used less reliably than material at the start or end, so more context is not automatically better. ## M ### Margin **Concept** · Tokenomics > Value > Unit Economics > Margin What is left after the cost of producing the tokens. ### Margin risk **Concept** · Tokenomics > Value > Monetization > Margin risk Ways a pricing model stops covering its costs. ### Market Structure **Concept** · Tokenomics > Value > Market Structure The supply landscape that sets what any buyer can actually get and at what price. ### Markup on token cost **Concept** · Tokenomics > Value > Monetization > Markup on token cost Reselling tokens above their raw cost as the margin. ### Matryoshka embeddings **Concept** · Tokenomics > Consumption > Optimization > AI memory > Retrieval mechanics > Dense retrieval > Matryoshka embeddings Embeddings trained so a truncated prefix stays usable, letting dimensions be cut later to reclaim storage and speed. ### Maturity staging **Concept** · Tokenomics > Value > Business Outcomes > Value Measurement > Maturity staging What a claim looks like at each stage of running the chain. Crawl: a net delta in native units with a stated baseline and floor, stopping at Net. Walk: adds the conversion chain and a named destination, running through Book. Run: composes the claim against TCA for a return, which needs cost at the same attribution grain. As seen in: [AI Value Classification](https://www.tokeneconomics.com/projects/ai-value-classification/) ### MCP (Model Context Protocol) **Concept** · Tokenomics > Consumption > Optimization > Agentic > Tool and protocol layer > MCP (Model Context Protocol) An open protocol for connecting models to tools and data through one interface, replacing bespoke integration per tool. ### Measurement pitfalls **Concept** · Tokenomics > Value > Unit Economics > Measurement pitfalls Ways a unit economics figure can be technically correct and still mislead. ### Mechanics **Concept** · Tokenomics > Production > Serving > Inference Optimization > Speculative decoding > Mechanics The draft then verify loop shared by every speculative method. ### Memory **Concept** · Tokenomics > Production > Hardware > Capacity > Memory On-device and system memory that bounds model and batch size. ### Memory bandwidth (HBM) **KPI** · Tokenomics > Production > Hardware > Capacity > Memory > Memory bandwidth (HBM) High-bandwidth memory speed, often the true bottleneck for inference. ### Memory hierarchy and offloading **Concept** · Tokenomics > Production > Serving > Inference Optimization > Memory hierarchy and offloading Serving a model whose weights exceed accelerator memory by placing state across GPU, host memory, and storage. ### Metal / MPS **Concept** · Tokenomics > Production > Hardware > Software stack > Metal / MPS Apple's GPU stack, the Apple Silicon analog to CUDA and ROCm, and the layer MLX and llama.cpp target on Mac. ### Metering design **Concept** · Tokenomics > Value > Monetization > Metering design Choosing what the customer sees on their bill. ### Method **Concept** · Tokenomics > Production > Training > Quantization > Method When quantization is applied relative to training. ### MFU (Model FLOPs Utilization) **KPI** · Tokenomics > Production > Hardware > Performance characterization > MFU (Model FLOPs Utilization) Achieved arithmetic as a share of the hardware's rated peak, the standard measure of whether expensive silicon is earning its keep. ### Microscaling formats (MXFP4, NVFP4) **Concept** · Tokenomics > Production > Training > Quantization > Precision formats > Microscaling formats (MXFP4, NVFP4) Four-bit formats that attach a shared scale to small blocks of values, giving low precision more usable range. ### Migration cost **Concept** · Tokenomics > Consumption > Optimization > Model Selection > Migration cost Prompts, evals, and guardrails are tuned per model, so switching is rarely as cheap as the price difference suggests. ### Min-p sampling **Concept** · Tokenomics > Consumption > Optimization > Sampling and decoding controls > Min-p sampling Setting a floor relative to the top token's probability, which adapts the candidate pool to model confidence. ### Minimum cacheable prefix **Concept** · Tokenomics > Consumption > Optimization > Caching > Provider-side prompt caching > Minimum cacheable prefix Each model sets a minimum prefix length, from hundreds to thousands of tokens, below which caching silently does nothing instead of returning an error. ### Mirror-SD **Concept** · Tokenomics > Production > Serving > Inference Optimization > Speculative decoding > Execution topology > Mirror-SD Runs draft and target as two overlapping pipelines across heterogeneous accelerators, the draft proposing continuations while the target proposes correction paths, preserving the standard acceptance rule. Sources: ### Misroutes and routing quality **Concept** · Tokenomics > Consumption > Optimization > Routing > Misroutes and routing quality Requests a classifier sends to a model that cannot handle them, answered confidently and incorrectly. Damage per misroute runs many multiples of the request cost, and uninstrumented misroutes surface months later through customer harm rather than the bill, so per-tier quality monitoring and a log of which model served every request are prerequisites to trusting a cost-based router. ### Mixture of Experts **Concept** · Tokenomics > Production > Training > Distributed Training > Mixture of Experts Distributes expert subnetworks across devices, active only when routed to. ### Mixture of Experts (MoE) **Concept** · Tokenomics > Production > Model / Architecture > Family > Mixture of Experts (MoE) Routes each token to a subset of expert subnetworks, cutting active compute per token. ### MLC-LLM **Concept** · Tokenomics > Production > Serving > Inference Engines > Local / edge runtimes > MLC-LLM A compiler-based runtime that deploys models across many devices. ### MLX / mlx-lm **Concept** · Tokenomics > Production > Serving > Inference Engines > Local / edge runtimes > MLX / mlx-lm Apple's array framework for Apple Silicon, supporting local inference and LoRA fine-tuning on Mac over unified memory with 4- and 8-bit quantization. ### Modality **Concept** · Tokenomics > Production > Model / Architecture > Modality The kinds of input and output a model supports. ### Model / Architecture **Concept** · Tokenomics > Production > Model / Architecture The model's design, which sets the cost floor before any training or serving. ### Model card **Concept** · Tokenomics > Production > Model / Architecture > Openness > Model card Published documentation of a model's intended use, training approach, and limitations. ### Model deprecation **Concept** · Tokenomics > Consumption > Access & Delivery > Reliability and continuity > Model deprecation Providers retire model versions on their own schedule, forcing migration work that was not planned or budgeted. ### Model mix **KPI** · Tokenomics > Value > Unit Economics > Cost drivers > Model mix The distribution of traffic across price tiers, usually more movable than any single price. ### Model multiplexing **Concept** · Tokenomics > Production > Serving > Scaling > Model multiplexing Hosting several models or adapters on one deployment, including LoRA swapping and sleeping idle models to reclaim memory. ### Model obsolescence **Concept** · Tokenomics > Production > Training > Training economics > Model obsolescence A trained model depreciates as better base models arrive, shortening the window to recover its cost. ### Model Selection **Concept** · Tokenomics > Consumption > Optimization > Model Selection Choosing the right model for each task to balance quality and cost, the portfolio decision made at design time; Routing applies the same criteria per request at run time. As seen in: [The Seven-Layer Tokenomics Stack (paper) (L4)](https://www.tokeneconomics.com/projects/the-seven-layer-tokenomics-stack-paper/) ### Model tiering **Concept** · Tokenomics > Consumption > Optimization > Agentic > Sub-agents / orchestration > Model tiering Planning on a stronger model while delegating execution to cheaper ones, reserving the most expensive tokens for judgment. ### Model tiers (nano / small / frontier) **Concept** · Tokenomics > Production > Model / Architecture > Scale > Model tiers (nano / small / frontier) Capability tiers that let teams match model cost to task difficulty. ### Model variants **Concept** · Tokenomics > Production > Model / Architecture > Model variants The distinct kinds of model a stack uses, which are priced and evaluated separately. ### Model version binding **Concept** · Tokenomics > Consumption > Optimization > Caching > Provider-side prompt caching > Model version binding A cache entry belongs to one exact model version, so escalating to another model or a provider shipping an update invalidates prefixes fleet-wide with no change on the consumer side. ### Moderation / safety **Concept** · Tokenomics > Consumption > Guardrails > Moderation / safety Screening inputs and outputs for harmful content. ### Monetization **Concept** · Tokenomics > Value > Monetization How the value of tokens is captured as revenue. ### Mooncake Store **Concept** · Tokenomics > Production > Serving > KV Cache > Cache Management > Hierarchical KV cache > KV cache storage tier > Mooncake Store The distributed KV cache store inside Mooncake, the serving platform behind Kimi, which pools spare CPU, DRAM, and SSD capacity across prefill and decode clusters. Sources: , ### Multi-Head Attention **Concept** · Tokenomics > Production > Model / Architecture > Attention > Multi-Head Attention The standard attention with separate key and value heads, the most memory-hungry variant. ### Multi-head drafting **Concept** · Tokenomics > Production > Serving > Inference Optimization > Speculative decoding > Drafter designs > Multi-head drafting Extra prediction heads bolted onto the target model emit several future positions in parallel; Medusa and Hydra are the reference designs. ### Multi-head Latent Attention (MLA) **Concept** · Tokenomics > Production > Model / Architecture > Attention > Multi-head Latent Attention (MLA) Compresses keys and values into a low-rank latent space, cutting KV cache size sharply at some added compute. ### Multi-provider failover **Concept** · Tokenomics > Consumption > Access & Delivery > Reliability and continuity > Multi-provider failover Keeping a second provider viable, which costs duplicated evaluation and prompt tuning. ### Multi-query fan-out **Concept** · Tokenomics > Consumption > Optimization > AI memory > Retrieval mechanics > Result fusion > Fusion across query variants > Multi-query fan-out Generating N reformulations of the query and retrieving for each. Cost scales linearly with N across search and embedding, and the generation producing the variants is paid on every query, so fan-out width is a direct multiplier on per-query cost. ### Multi-tenancy **Concept** · Tokenomics > Production > Serving > Scaling > Multi-tenancy Serving many users or models on shared hardware to raise utilization. ### Multi-Token Prediction (MTP) **Concept** · Tokenomics > Production > Serving > Inference Optimization > Speculative decoding > Drafter designs > Multi-Token Prediction (MTP) Extra prediction modules trained into the model itself, originally as a training objective, then reused at serving time as the drafter; DeepSeek-V3 popularised the pattern. Sources: ### Multi-token prediction heads **Concept** · Tokenomics > Production > Model / Architecture > Heads and objectives > Multi-token prediction heads Extra heads trained to predict several positions ahead, useful during training and reusable as a drafter at inference. ### Multi-vector retrieval (ColBERT) **Concept** · Tokenomics > Consumption > Optimization > AI memory > Retrieval mechanics > Dense retrieval > Multi-vector retrieval (ColBERT) Storing per-token vectors and scoring by late interaction, which raises accuracy and sharply raises index size. ### Multimodal tokenization **Concept** · Tokenomics > Production > Tokenization > Multimodal tokenization How non-text inputs are converted into tokens. ### Multiple samples (n, best-of) **Concept** · Tokenomics > Consumption > Optimization > Sampling and decoding controls > Multiple samples (n, best-of) Generating several candidates per request, a direct multiplier on output cost. ### Muon **Concept** · Tokenomics > Production > Training > Pretraining > Optimizer > Muon Momentum-orthogonalizing optimizer for 2D hidden weights, reaching target loss in fewer steps and cutting compute at scale. ## N ### Native spend controls **Concept** · Tokenomics > Consumption > Workloads / Use Cases > Coding agents > Native spend controls Usage dashboards, budgets, and limits exposed inside an agent product rather than only through the underlying billing layer. ### NCCL and collective libraries **Concept** · Tokenomics > Production > Hardware > Software stack > NCCL and collective libraries Libraries implementing multi-device communication, which dominate performance once a model is sharded. ### Negotiated and private pricing **Concept** · Tokenomics > Consumption > Purchase Options > Commercial structures > Negotiated and private pricing Discounted terms at volume, which make public rates a poor basis for benchmarking. ### Neoclouds and GPU specialists **Concept** · Tokenomics > Production > Hardware > Procurement > Neoclouds and GPU specialists Providers built specifically around accelerator rental, often priced and contracted differently from general-purpose clouds. ### Net **Concept** · Tokenomics > Value > Business Outcomes > Value Measurement > Claim chain > Net Gross delta times volume, less outputs that missed the floor, less cost created, less harm from misses, equals net benefit. One number, in one unit, over one period, with one confidence mark that carries the weakest provenance among the inputs. ### Network Bandwidth **KPI** · Tokenomics > Production > Hardware > Capacity > Networking > Network Bandwidth Throughput between nodes, critical for distributed training and serving. ### Networking **Concept** · Tokenomics > Production > Hardware > Capacity > Networking Data movement within and between machines. ### New Capability **KPI** · Tokenomics > Value > Business Outcomes > Value Categorization > New Capability Something not previously possible, with no prior state to measure against. The claim is about viability rather than delta: name the alternative you chose not to build. As seen in: [AI Value Classification](https://www.tokeneconomics.com/projects/ai-value-classification/) ### Next-token prediction **Concept** · Tokenomics > Production > Model / Architecture > Heads and objectives > Next-token prediction The standard objective, predicting one token ahead, which is what makes decoding serial. ### NIXL **Concept** · Tokenomics > Production > Serving > KV Cache > Cache Management > Hierarchical KV cache > KV cache storage tier > NIXL A transfer library with a unified API over memory and storage tiers, covering plugins such as 3FS, GPUDirect Storage, and S3-compatible object storage. The access layer the stores are reached through rather than a store in its own right. ### No-baseline problem **Concept** · Tokenomics > Value > Business Outcomes > Value Measurement > No-baseline problem Workloads where AI created the process so no counterfactual exists. The hardest measurement case, handled with bounded estimates rather than solved. ### NVIDIA Dynamo **Concept** · Tokenomics > Production > Serving > Inference Engines > Serving orchestration > NVIDIA Dynamo A framework for distributed inference across nodes, including disaggregated prefill and decode. ### NVIDIA NIM **Concept** · Tokenomics > Production > Serving > Inference Engines > Packaged / managed > NVIDIA NIM NVIDIA's packaged inference microservices that wrap optimized engines. ## O ### Object detection / grounding **Concept** · Tokenomics > Consumption > Workloads / Use Cases > Vision workloads > Object detection / grounding Locates or points to image regions, returning coordinates rather than prose. ### Observability **Concept** · Tokenomics > Consumption > Observability Visibility into the usage, cost, and quality of AI systems. ### Observe **Concept** · Tokenomics > Value > Business Outcomes > Value Measurement > Claim chain > Observe The inputs, each denominated in the baseline's unit: scope, baseline, observed state, Quality Floor, cost created, and provenance. Cost created is the review, triage, rework, supervision, and escalation the AI adds inside the workload, expressed in the baseline's unit. As seen in: [AI Value Classification](https://www.tokeneconomics.com/projects/ai-value-classification/) ### OCR / document parsing **Concept** · Tokenomics > Consumption > Workloads / Use Cases > Vision workloads > OCR / document parsing Reads text and layout from images and PDFs, billed as input tiles plus transcribed output tokens. ### Offline vs online evaluation **Concept** · Tokenomics > Consumption > Guardrails > Production evals > Offline vs online evaluation Testing against fixed sets before release versus sampling live traffic afterward. ### Open source **Concept** · Tokenomics > Production > Model / Architecture > Openness > Open source Weights plus training code and often data recipe, a stricter standard than open weights alone. ### Open vs proprietary **Concept** · Tokenomics > Consumption > Optimization > Model Selection > Open vs proprietary Whether to self-host open weights or buy an API, a decision about control and fixed versus variable cost. ### Open weights **Concept** · Tokenomics > Production > Model / Architecture > Openness > Open weights Weights are downloadable, enabling self-hosting and full cost control. ### Openness **Concept** · Tokenomics > Production > Model / Architecture > Openness How available the model's weights and usage rights are. ### OpenTelemetry **Concept** · Tokenomics > Consumption > Observability > Standards / Open Specifications > OpenTelemetry The vendor-neutral standard for traces and metrics, including conventions for generative AI calls. ### Operations **Concept** · Tokenomics > Production > Training > Operations The practical running of training jobs. ### Opportunity cost **Concept** · Tokenomics > Value > Business Case > Opportunity cost What the same capacity, capital, or team could have produced elsewhere. ### Optimization **Concept** · Tokenomics > Consumption > Optimization Levers that reduce the number or price of tokens consumed. ### Optimizer **Concept** · Tokenomics > Production > Training > Pretraining > Optimizer Turns gradients into weight updates, with stored state that is a large share of training memory and a choice that sets convergence speed. ### Out-of-domain degradation **Concept** · Tokenomics > Consumption > Optimization > AI memory > Retrieval mechanics > Dense retrieval > Out-of-domain degradation Embedding models trained on general web text lose accuracy on specialist corpora, the failure that hybrid search exists to cover. The BEIR benchmark is the standard evidence. Sources: ### Out-of-memory failures **Concept** · Tokenomics > Production > Serving > Reliability > Out-of-memory failures The most common serving failure, usually caused by cache growth rather than by weights. ### Outcome KPIs **Concept** · Tokenomics > Value > Business Outcomes > Outcome KPIs The outcome-facing readouts of unit economics, where cost per unit stops describing production efficiency and starts describing business results. ### Outcome pricing **Concept** · Tokenomics > Consumption > Purchase Options > Price axes > Outcome pricing Charging per completed task rather than per token, which shifts efficiency risk to the seller. ### Outcome reward model (ORM) **Concept** · Tokenomics > Production > Training > Post-Training > Alignment > Reward models > Outcome reward model (ORM) Scores only the final answer, cheap to label but blind to where a reasoning trace failed. ### Outcome-based pricing **Concept** · Tokenomics > Value > Monetization > Outcome-based pricing Charging for completed work, which requires confident unit economics because the efficiency risk sits with the seller. ### Output length control (max tokens, stop sequences) **Concept** · Tokenomics > Consumption > Prompt / Context Engineering > Output length control (max tokens, stop sequences) Capping generation length to control output cost. ### Output validation **Concept** · Tokenomics > Consumption > Guardrails > Output validation Checking responses against schemas or rules before use. ### Over-refusals and caveats **Concept** · Tokenomics > Consumption > Guardrails > Over-refusals and caveats Over-refusals and appended caveats burn output tokens and trigger retries, a safety and quality tradeoff with direct token cost. ### Overage terms **Concept** · Tokenomics > Consumption > Purchase Options > Commercial structures > Overage terms What happens past a committed volume, frequently the least examined term in the contract. ### Overflow handling **Concept** · Tokenomics > Consumption > Prompt / Context Engineering > Context window management > Overflow handling What happens when assembled context exceeds the limit, which is a correctness issue as much as a cost one. ## P ### Packaged / managed **Concept** · Tokenomics > Production > Serving > Inference Engines > Packaged / managed Prebuilt or hosted serving offerings that trade cost for convenience. ### PagedAttention **Concept** · Tokenomics > Production > Serving > KV Cache > PagedAttention Pages the KV cache into fixed-size blocks to cut fragmentation and raise GPU utilization. Sources: ### Parallel / diffusion decoding **Concept** · Tokenomics > Production > Model / Architecture > Heads and objectives > Parallel / diffusion decoding Refines the whole sequence over parallel denoising steps rather than one token at a time, which gives diffusion models a latency profile unlike autoregressive decoding. ### Parallelism (tensor / pipeline / model sharding) **Concept** · Tokenomics > Production > Serving > Inference Optimization > Parallelism (tensor / pipeline / model sharding) Splits a model across devices so it fits and serves faster. ### Parameter count **KPI** · Tokenomics > Production > Model / Architecture > Scale > Parameter count The total number of weights, correlating with capability and serving cost. ### Partial token handling **Concept** · Tokenomics > Production > Tokenization > Detokenization and streaming > Partial token handling A single character can span several tokens, so streamed output must buffer until a boundary is safe to emit. ### pass@k **KPI** · Tokenomics > Production > Training > Evaluation > Benchmarks > pass@k The share of problems solved within k attempts, which makes the cost of retries explicit in the metric itself. ### Payback period **KPI** · Tokenomics > Value > Business Case > Payback period How long until savings or revenue cover the investment. ### Peer-to-Peer (P2P) Cache Transfer **Concept** · Tokenomics > Production > Serving > KV Cache > Cache Management > Cross-Engine Support > Peer-to-Peer (P2P) Cache Transfer Moving cache directly between workers without a central store. ### PEFT / Adapters **Concept** · Tokenomics > Production > Training > Post-Training > Fine Tuning > PEFT / Adapters Parameter-efficient methods that train small add-on weights instead of the full model. ### Per-query cost shape **Concept** · Tokenomics > Consumption > Optimization > AI memory > Knowledge injection patterns > CAG (Cache-Augmented Generation) > Per-query cost shape Removes retrieval and reranking calls but presents the full corpus as input on every query, so the economics depend entirely on cached input being priced below normal input. ### Per-request pricing **Concept** · Tokenomics > Consumption > Purchase Options > Price axes > Per-request pricing Charging by call rather than by token, common for embeddings and small utility models. ### Per-turn and per-session budgets **KPI** · Tokenomics > Consumption > Prompt / Context Engineering > Token budgeting / counting > Per-turn and per-session budgets Limits applied at the level a user or agent actually operates, rather than per request. ### Per-unit media pricing **Concept** · Tokenomics > Consumption > Purchase Options > Price axes > Per-unit media pricing Images, audio, and video priced by tile, second, or frame rather than by token. ### Per-workload measurement **Concept** · Tokenomics > Consumption > Optimization > Caching > Cache measurement > Per-workload measurement One high-volume cached workload will mask a dozen broken ones in a blended number, so every cache ratio is tracked per workload and re-checked when model versions or price sheets change. ### Performance **Concept** · Tokenomics > Production > Serving > Performance The latency and throughput metrics that define serving quality and cost. ### Performance characterization **Concept** · Tokenomics > Production > Hardware > Performance characterization How to tell whether hardware is actually being used, and what is limiting it. ### Perplexity **KPI** · Tokenomics > Production > Training > Quantization > Quality impact > Perplexity How well a model predicts held-out text, the standard quick proxy for quantization damage and a weak predictor of task performance. ### Persistent Caching **Concept** · Tokenomics > Production > Serving > KV Cache > Cache Management > Persistent Caching Keeping KV cache across requests so repeated prefixes are not recomputed. ### PII detection and redaction **Concept** · Tokenomics > Consumption > Guardrails > Data protection > PII detection and redaction Finding and removing personal data before it is sent, usually an extra model or classifier call per request. ### Pipeline parallelism **Concept** · Tokenomics > Production > Training > Distributed Training > Pipeline parallelism Splits the model by layers into stages across devices. ### Positional encoding (RoPE, ALiBi) **Concept** · Tokenomics > Production > Model / Architecture > Context > Positional encoding (RoPE, ALiBi) Methods for encoding token position that affect how far context can be extended. ### Post-Training **Concept** · Tokenomics > Production > Training > Post-Training Steps after pretraining that adapt and align a model. ### Power **Concept** · Tokenomics > Production > Hardware > Capacity > Power Electrical supply and heat management for the fleet. ### PPO **Concept** · Tokenomics > Production > Training > Post-Training > Alignment > PPO The policy optimization algorithm long used to run the reinforcement learning step in RLHF. ### Practices **Concept** · Tokenomics > Consumption > Observability > Practices Turning telemetry into something anyone acts on. ### Pre-flight counting **Concept** · Tokenomics > Consumption > Prompt / Context Engineering > Token budgeting / counting > Pre-flight counting Measuring a request before sending it, the only way to enforce a budget rather than discover it afterward. ### Pre-tokenization rules **Concept** · Tokenomics > Production > Tokenization > Algorithm > Pre-tokenization rules The regex or splitting pass applied before merging, which decides how whitespace, digits, and punctuation group together. ### Precision formats **Concept** · Tokenomics > Production > Training > Quantization > Precision formats The numeric formats weights and activations use. ### Precision throughput **Concept** · Tokenomics > Production > Hardware > Capacity > Compute > Precision throughput Rated performance differs sharply by numeric format, so quoted peak figures only apply at a stated precision. ### Precomputed document caches **Concept** · Tokenomics > Consumption > Optimization > AI memory > Knowledge injection patterns > Precomputed document caches Building and storing reusable cached state per document chunk so retrieved passages skip prefill, a middle path that keeps retrieval but removes most of its compute cost. ### Predictability **Concept** · Tokenomics > Consumption > Workloads / Use Cases > Workload characteristics > Predictability How reliably future volume can be forecast, the precondition for any capacity commitment. ### Preemption and eviction **Concept** · Tokenomics > Production > Serving > Reliability > Preemption and eviction Pausing or dropping running requests under memory pressure, which wastes already-spent compute. ### Prefetch policy **Concept** · Tokenomics > Production > Serving > KV Cache > Cache Management > Hierarchical KV cache > Prefetch policy Pulling cache up from a slower tier before it is needed, which hides transfer latency and wastes bandwidth when the prediction is wrong. ### Prefill / Decode disaggregation **Concept** · Tokenomics > Production > Serving > Inference Optimization > Prefill / Decode disaggregation Separates the compute-heavy prefill phase from the memory-bound decode phase for efficiency. The compute-side half of a KVCache-centric architecture; the cache-side half is Hierarchical KV cache under KV Cache. ### Prefix caching **Concept** · Tokenomics > Production > Serving > KV Cache > Prefix caching Reusing the cached computation of shared prompt prefixes across requests. As seen in: [Cache Explainer](https://www.tokeneconomics.com/projects/cache-explainer/) ### Prefix stability **Concept** · Tokenomics > Consumption > Optimization > Caching > Provider-side prompt caching > Prefix stability Cached reuse requires an exact shared prefix starting at token zero, so anything that varies must sit at the end of the prompt and serialization must be frozen. ### Pretraining **Concept** · Tokenomics > Production > Training > Pretraining The large-scale initial training that gives a model its base capabilities. ### Price axes **Concept** · Tokenomics > Consumption > Purchase Options > Price axes What the meter actually counts, which differs by provider and by modality. ### Price by model tier **Concept** · Tokenomics > Consumption > Purchase Options > Price modifiers > Price by model tier Different prices for models of different capability. ### Price dynamics **Concept** · Tokenomics > Consumption > Purchase Options > Price dynamics How prices move over time and why. ### Price modifiers **Concept** · Tokenomics > Consumption > Purchase Options > Price modifiers Adjustments applied on top of the base rate. ### Price per unit of capability **KPI** · Tokenomics > Consumption > Purchase Options > Price dynamics > Price per unit of capability Cost to reach a given quality level, which has fallen even where headline rates have not. ### Priority and latency tiers **Concept** · Tokenomics > Consumption > Purchase Options > Price modifiers > Priority and latency tiers Paying more for faster or more reliable service, or less for best-effort. ### Process reward model (PRM) **Concept** · Tokenomics > Production > Training > Post-Training > Alignment > Reward models > Process reward model (PRM) Scores each reasoning step, a denser signal at much higher labeling and inference cost. ### Procurement **Concept** · Tokenomics > Production > Hardware > Procurement How compute capacity is acquired. ### Product Quality **KPI** · Tokenomics > Value > Business Outcomes > Value Categorization > Product Quality Fewer defects, less rework, better product. Quality of the work the business ships, deliberately renamed from Output Quality to keep it from being confused with the Quality Floor signal. Measurable only once the workload has a defined floor and instrumentation. As seen in: [AI Value Classification](https://www.tokeneconomics.com/projects/ai-value-classification/) ### Product versus internal posture **Concept** · Tokenomics > Value > Business Outcomes > Value Measurement > Product versus internal posture Restored as the what of common AI task types: Product AI is customer-facing and judged on growth and margin, Internal AI is operational and judged on cost and efficiency. The posture picks the scoreboard but never changes the measurement math, and packaging never decides posture, consumption does. ### Production **Concept** · Tokenomics > Production The supply side of AI. Energy, capital resources, and the economics of building and running AI in production. How capacity is planned, costed, and brought online. As seen in: [What Is AI Tokenomics?](https://www.tokeneconomics.com/what-is-ai-tokenomics/), [Personas and Operating Model Map](https://www.tokeneconomics.com/projects/personas-and-operating-model-map/) ### Production evals **Concept** · Tokenomics > Consumption > Guardrails > Production evals Ongoing quality measurement of live traffic. ### Prompt / Context Engineering **Concept** · Tokenomics > Consumption > Prompt / Context Engineering Per-request techniques that control how many tokens a call uses. ### Prompt / Prefix tuning **Concept** · Tokenomics > Production > Training > Post-Training > Fine Tuning > Prompt / Prefix tuning Learns soft prompt vectors while freezing the model, the lightest adaptation. ### Prompt compression **Concept** · Tokenomics > Consumption > Prompt / Context Engineering > Prompt compression Shortening prompts while preserving meaning to cut input cost. ### Prompt design (few-shot, system prompts) **Concept** · Tokenomics > Consumption > Prompt / Context Engineering > Prompt design (few-shot, system prompts) Structuring prompts and examples to get good results with fewer tokens. ### Prompt injection **Concept** · Tokenomics > Consumption > Prompt / Context Engineering > Context failure modes > Prompt injection Untrusted content in context being treated as instruction, a security problem with direct cost consequences when it triggers loops. ### Prompt lookup / n-gram **Concept** · Tokenomics > Production > Serving > Inference Optimization > Speculative decoding > Drafter designs > Retrieval and n-gram drafting > Prompt lookup / n-gram Matches the recent suffix against the prompt and generation history, then proposes what followed last time. ### Prompt ordering for cacheability **Concept** · Tokenomics > Consumption > Prompt / Context Engineering > Prompt design (few-shot, system prompts) > Prompt ordering for cacheability Stable content first, growing content next, per-request content last, since cached reuse stops at the first token that differs from the previous request. ### Prompt versioning **Concept** · Tokenomics > Consumption > Guardrails > Production evals > Prompt versioning Tracking prompts as versioned artifacts, without which a quality or cost change cannot be traced to its cause. ### Proprietary / API-only **Concept** · Tokenomics > Production > Model / Architecture > Openness > Proprietary / API-only Accessible only through a vendor API, trading control for convenience. ### Provider differences **Concept** · Tokenomics > Consumption > Optimization > Caching > Provider-side prompt caching > Provider differences Published write charges, read discounts, minimum lengths, and lifetimes differ sharply between frontier providers, and the terms can change without notice (a major provider introduced cache write charges in mid-2026), which is why shared cache metrics must stay agnostic to any vendor implementation. Sources: , ### Provider landscape **Concept** · Tokenomics > Value > Market Structure > Provider landscape Model developers, hyperscalers, specialist accelerator clouds, and aggregators, each with a different cost basis. ### Provider SLAs **KPI** · Tokenomics > Consumption > Access & Delivery > Reliability and continuity > Provider SLAs Contractual availability and support commitments, usually weaker than the equivalent for traditional cloud services. ### Provider-side prompt caching **Concept** · Tokenomics > Consumption > Optimization > Caching > Provider-side prompt caching Vendor caching of repeated prompt prefixes at a discounted rate, billed through separate cache write and cache read token buckets on every response. ### Proxy **Concept** · Tokenomics > Consumption > Optimization > Routing > Proxy A transparent pass-through on the request path: it forwards every call unchanged, collects metadata, and enforces network-level concerns such as rate limits and retries. It makes no model decision; the caller's chosen model is the model that answers. As seen in: [The Seven-Layer Tokenomics Stack (paper) (L5)](https://www.tokeneconomics.com/projects/the-seven-layer-tokenomics-stack-paper/) ### PTQ (Post-Training Quantization) **Concept** · Tokenomics > Production > Training > Quantization > Method > PTQ (Post-Training Quantization) Quantizes an already-trained model, fast but sometimes lossy. ### Public rate cards **Concept** · Tokenomics > Consumption > Purchase Options > Commercial structures > Public rate cards Published per-token pricing, the reference point for everything else. ### PUE (Power Usage Effectiveness) **KPI** · Tokenomics > Production > Hardware > Capacity > Power > PUE (Power Usage Effectiveness) The ratio of total facility power to IT power, measuring datacenter efficiency. ### Purchase Options **Concept** · Tokenomics > Consumption > Purchase Options How usage is bought from providers: the rate card, its modifiers, and the commercial structures around it. Purchasing mechanics rather than value measurement. ## Q ### QAT (Quantization-Aware Training) **Concept** · Tokenomics > Production > Training > Quantization > Method > QAT (Quantization-Aware Training) Trains with quantization simulated, preserving more accuracy. ### QLoRA (Quantized Low-Rank Adaptation) **Concept** · Tokenomics > Production > Training > Post-Training > Fine Tuning > PEFT / Adapters > QLoRA (Quantized Low-Rank Adaptation) Combines LoRA with a quantized base model to fine-tune on modest hardware. ### Quadratic cost of attention **Concept** · Tokenomics > Production > Model / Architecture > Context > Quadratic cost of attention Prefill compute grows with the square of input length, which is why long prompts cost more than their token count suggests. ### Quality / latency SLIs **KPI** · Tokenomics > Consumption > Observability > Signals > Quality / latency SLIs Indicators tracking response quality and speed. ### Quality Floor **Concept** · Tokenomics > Value > Business Outcomes > Value Measurement > Quality Floor The pass bar for acceptable output on a specific workload: quality is the gate, not a multiplier. Value only counts when output clears the floor, cheap output that misses it is waste, and full utilization of a model that misses it is waste rather than efficiency. As seen in: [AI Value Classification](https://www.tokeneconomics.com/projects/ai-value-classification/) ### Quality impact **Concept** · Tokenomics > Production > Training > Quantization > Quality impact How to tell what a precision reduction actually cost. ### Quantization **Concept** · Tokenomics > Production > Training > Quantization Reducing the numeric precision of weights and activations to shrink cost. As seen in: [What Is AI Tokenomics?](https://www.tokeneconomics.com/what-is-ai-tokenomics/), [The Seven-Layer Tokenomics Stack (L4)](https://www.tokeneconomics.com/projects/the-seven-layer-tokenomics-stack/) ### Quantized inference **Concept** · Tokenomics > Production > Serving > Inference Optimization > Quantized inference Serving a reduced-precision model to cut memory and raise throughput, the most widely used single optimization. ### Quantized KV cache **Concept** · Tokenomics > Production > Serving > KV Cache > Quantized KV cache Storing KV cache at lower precision to reduce its memory footprint. ### Quantized matmul kernels **Concept** · Tokenomics > Production > Serving > Inference Optimization > Kernels and compilation > Quantized matmul kernels Specialised kernels such as Marlin that make low-precision weights actually faster rather than merely smaller. ### Query preparation **Concept** · Tokenomics > Consumption > Optimization > AI memory > Retrieval mechanics > Query preparation Work done on the question before any search runs, cheap per query and often more effective than tuning the index. ### Query rewriting and expansion **Concept** · Tokenomics > Consumption > Optimization > AI memory > Retrieval mechanics > Query preparation > Query rewriting and expansion Reformulating the user question before search, which costs a small generation and often beats tuning the index. Generating several reformulations instead of one turns this into multi-query fan-out, covered under Result fusion. ## R ### Rack density **KPI** · Tokenomics > Production > Hardware > Capacity > Facilities > Rack density How much compute is packed per rack, affecting power and cooling design. ### Rack power density **KPI** · Tokenomics > Production > Hardware > Capacity > Power > Rack power density Kilowatts per rack, which decides whether existing facilities can host current accelerators at all. ### RadixAttention **Concept** · Tokenomics > Production > Serving > KV Cache > Prefix caching > RadixAttention Automatic reuse of shared prefixes via a radix tree over cached sequences, so prefix sharing needs no explicit cache management. ### RAG (Retrieval-Augmented Generation) **Concept** · Tokenomics > Consumption > Optimization > AI memory > Knowledge injection patterns > RAG (Retrieval-Augmented Generation) Fetching relevant documents to ground responses and cut hallucination. ### RAG / search **Concept** · Tokenomics > Consumption > Workloads / Use Cases > RAG / search Retrieval-grounded question answering over a corpus. ### RAG-Fusion **Concept** · Tokenomics > Consumption > Optimization > AI memory > Retrieval mechanics > Result fusion > Fusion across query variants > RAG-Fusion The published name for multi-query fan-out merged with RRF. Reported to give more comprehensive answers because the variants approach the question from several angles, and to drift off topic when the generated variants stray from the original intent. Sources: ### Rank constant k **Concept** · Tokenomics > Consumption > Optimization > AI memory > Retrieval mechanics > Result fusion > Rank-based fusion > Rank constant k The constant added to the rank before inversion, fixed at 60 in the original pilot and reused as the conventional default since, exposed by Elasticsearch as rank\_constant. A larger k flattens the gap between top and lower ranks, a smaller one sharpens it. ### Rank-based fusion **Concept** · Tokenomics > Consumption > Optimization > AI memory > Retrieval mechanics > Result fusion > Rank-based fusion Merges by position only, discarding the underlying scores. It needs no calibration between retrievers, so it works across scoring schemes that are not comparable, and it costs nothing beyond a sort. ### Rate limits / quotas **Concept** · Tokenomics > Consumption > Access & Delivery > Rate limits / quotas Provider caps on request and token throughput. ### Ray Serve **Concept** · Tokenomics > Production > Serving > Inference Engines > Serving orchestration > Ray Serve A scalable model-serving framework built on Ray. ### Re-baselining **Concept** · Tokenomics > Value > Business Outcomes > Value Measurement > Quality Floor > Re-baselining A floor calibrated for one model generation goes stale for the next, so it must be re-established as models evolve. ### ReAct pattern **Concept** · Tokenomics > Consumption > Optimization > Agentic > Tool and protocol layer > ReAct pattern Interleaving reasoning and action steps, the loop shape most agent implementations follow. ### Reasoning and test-time compute **Concept** · Tokenomics > Production > Model / Architecture > Reasoning and test-time compute Spending extra tokens at inference to raise answer quality, the newest and least predictable cost driver. ### Reasoning effort levels **Concept** · Tokenomics > Production > Model / Architecture > Reasoning and test-time compute > Reasoning effort levels Provider controls that trade answer quality against how many reasoning tokens are spent. ### Reasoning tokens **Concept** · Tokenomics > Production > Model / Architecture > Reasoning and test-time compute > Reasoning tokens Intermediate tokens a reasoning model produces before its answer, usually billed even when not shown to the user. As seen in: [What Is AI Tokenomics?](https://www.tokeneconomics.com/what-is-ai-tokenomics/), [Big-T Notation (paper)](https://www.tokeneconomics.com/projects/big-t-notation-paper/), [What FOCUS 1.5 Does for AI Cost](https://www.tokeneconomics.com/projects/what-1-5-does-for-ai-cost-and-what-it-does-not/) ### Reciprocal Rank Fusion (RRF) **Concept** · Tokenomics > Consumption > Optimization > AI memory > Retrieval mechanics > Result fusion > Rank-based fusion > Reciprocal Rank Fusion (RRF) Sums 1/(k + rank) across every list a document appears in. It combines positions rather than scores, needs no training and no model call, and ships in Elasticsearch, Qdrant, and Weaviate, where it is Qdrant's default. It is the baseline any reranker model, which bills per candidate, has to beat. Sources: , ### Reconciliation **Concept** · Tokenomics > Consumption > Observability > Practices > Reconciliation Checking engine and application counts against the invoice, which is where silent measurement errors surface. ### Red teaming **Concept** · Tokenomics > Consumption > Guardrails > Adversarial input > Red teaming Structured adversarial testing before and after release, which finds failure modes normal evaluation does not. ### Redeployment **Concept** · Tokenomics > Value > Business Outcomes > Value Measurement > Claim chain > Net > Redeployment Freed capacity that was not redeployed nets to zero realized benefit. It is tracked as an FTE estimate, not claimed, until the capacity is spent on work that produces an outcome. This applies to every labor category, not only capacity gain. As seen in: [AI Value Classification](https://www.tokeneconomics.com/projects/ai-value-classification/) ### Reflection loops **Concept** · Tokenomics > Consumption > Optimization > Agentic > Token amplification > Reflection loops Steps where the model critiques and revises its own work, doubling or more the tokens per unit of output. ### Refresh cycles **Concept** · Tokenomics > Production > Hardware > Lifecycle (Intersections with HAM) > Refresh cycles The cadence of replacing hardware with newer generations. ### Region and data residency **Concept** · Tokenomics > Consumption > Access & Delivery > Reliability and continuity > Region and data residency Where inference physically runs, which constrains provider choice and often price. ### Regional pricing **Concept** · Tokenomics > Consumption > Purchase Options > Price modifiers > Regional pricing Rates that vary by where inference runs, reflecting local capacity and power cost. ### Regression detection **Concept** · Tokenomics > Consumption > Guardrails > Production evals > Regression detection Catching quality drops after a prompt or model change, the control that makes cost optimization safe to attempt. ### Reinforcement learning with verifiable rewards **Concept** · Tokenomics > Production > Training > Post-Training > Alignment > Reinforcement learning with verifiable rewards Training against automatically checkable outcomes such as passing tests, which removes the human labelling bottleneck. ### Relative Score Fusion **Concept** · Tokenomics > Consumption > Optimization > AI memory > Retrieval mechanics > Result fusion > Score-based fusion > Relative Score Fusion Min-max normalizes each list so its top score becomes 1 and its bottom becomes 0, keeping the spacing in between, then adds the weighted results. Weaviate's default since v1.24, where alpha sets the split between the vector and keyword contributions. Sources: ### Release-level throughput **KPI** · Tokenomics > Value > Business Outcomes > Release-level throughput The rate at which finished, production-quality changes reach customers, counted at release or later. Because a delivery system moves only as fast as its constraint, speedups at other stages move this number far less than they move stage metrics: in a matched study of over 500,000 GitHub developers (Demirer, Musolff, Yang, NBER w35275), autonomous coding gains of roughly 240% in coding activity shrank to about 30% at releases, with no measurable rise in product usage across four app marketplaces. Sources: , ### Reliability **Concept** · Tokenomics > Production > Serving > Reliability Keeping a serving deployment healthy under real traffic. ### Reliability and continuity **Concept** · Tokenomics > Consumption > Access & Delivery > Reliability and continuity Whether the capacity will be there when needed. ### Repetition and frequency penalties **Concept** · Tokenomics > Consumption > Optimization > Sampling and decoding controls > Repetition and frequency penalties Downweighting tokens already produced, which suppresses loops that would otherwise run to the token limit and bill for it. ### Replica sizing **Concept** · Tokenomics > Production > Serving > Scaling > Replica sizing Choosing how many accelerators back one model instance, which sets both the granularity of scaling and the parallelism overhead. ### Requests and tokens per minute **KPI** · Tokenomics > Consumption > Access & Delivery > Rate limits / quotas > Requests and tokens per minute The two limits that bind in practice, often before any budget does. ### Requests per second (QPS) **KPI** · Tokenomics > Production > Serving > Performance > Requests per second (QPS) Completed requests per second, the capacity unit that translates most directly into replica sizing. ### Reranker model **Concept** · Tokenomics > Production > Model / Architecture > Model variants > Reranker model Scores query and document pairs directly, more accurate than vector similarity and more expensive per candidate. ### Reranking **Concept** · Tokenomics > Consumption > Optimization > AI memory > Retrieval mechanics > Reranking A second scoring pass over retrieved candidates using a model that reads query and document together, which raises precision and lets fewer documents be sent to the model. Distinct from result fusion, which reorders existing lists arithmetically with no model call and no per-candidate cost. ### Reserved and unused slots **Concept** · Tokenomics > Production > Tokenization > Vocabulary > Reserved and unused slots Vocabulary entries held back for later use, often repurposed for control tokens or tool syntax. ### Reserved capacity (PTUs / provisioned throughput) **Concept** · Tokenomics > Consumption > Purchase Options > Capacity and commitment > Reserved capacity (PTUs / provisioned throughput) Prepaid dedicated throughput for predictable cost and latency. ### Resolution and detail settings **Concept** · Tokenomics > Production > Tokenization > Multimodal tokenization > Resolution and detail settings Requested image detail changes how many tiles are produced, making resolution a direct cost control. ### Result fusion **Concept** · Tokenomics > Consumption > Optimization > AI memory > Retrieval mechanics > Result fusion How several ranked result lists become one ordering, the step that actually implements hybrid search and multi-query retrieval. Fusion only reorders the union of what the retrievers returned, so a document none of them found cannot appear at all, and the method chosen changes ranking quality without changing token cost. ### Retries (token cost) **Concept** · Tokenomics > Consumption > Guardrails > Retries (token cost) Re-running failed calls, a hidden and often overlooked cost. ### Retries and self-correction **Concept** · Tokenomics > Consumption > Optimization > Agentic > Token amplification > Retries and self-correction Failed steps are re-attempted with the failure added to context, which compounds cost on exactly the hardest tasks. ### Retrieval and n-gram drafting **Concept** · Tokenomics > Production > Serving > Inference Optimization > Speculative decoding > Drafter designs > Retrieval and n-gram drafting Model free drafting that copies candidate spans from text already in context, cheap to run and strong on repetitive or edit heavy work. ### Retrieval from an external corpus **Concept** · Tokenomics > Production > Serving > Inference Optimization > Speculative decoding > Drafter designs > Retrieval and n-gram drafting > Retrieval from an external corpus Drafts from a datastore rather than only the current context, trading memory and lookup cost for higher acceptance on familiar material. ### Retrieval mechanics **Concept** · Tokenomics > Consumption > Optimization > AI memory > Retrieval mechanics The retrieval pipeline beneath any knowledge injection pattern, ordered by stage: what is built at index time, how the query is prepared, how candidates are found, how lists are merged and reordered, and what it costs when the result is wrong. ### Retrieval quality and wasted context **Concept** · Tokenomics > Consumption > Optimization > AI memory > Retrieval mechanics > Retrieval quality and wasted context Irrelevant retrieved passages are billed as input tokens and can actively degrade the answer. ### Retry and failure rate **KPI** · Tokenomics > Value > Unit Economics > Cost drivers > Retry and failure rate Work paid for twice, which links quality engineering directly to unit cost. ### Reuse ratio **KPI** · Tokenomics > Consumption > Optimization > Caching > Cache measurement > Reuse ratio Tokens read from cache divided by tokens written to cache over the same window: pulls off the shelf over pulls paid to put there, the payback measure for the cache. ### Revenue Enablement **KPI** · Tokenomics > Value > Business Outcomes > Value Categorization > Revenue Enablement Revenue that would not exist without the AI. The number already sits in a ledger, so the argument is attribution rather than measurement. As seen in: [AI Value Classification](https://www.tokeneconomics.com/projects/ai-value-classification/) ### Reward models **Concept** · Tokenomics > Production > Training > Post-Training > Alignment > Reward models Models trained to score outputs, used as the objective that policy optimization maximizes. ### Risk adjustment **Concept** · Tokenomics > Value > Business Case > Risk adjustment Discounting expected return for model, vendor, and regulatory uncertainty. ### Risk Reduction **KPI** · Tokenomics > Value > Business Outcomes > Value Categorization > Risk Reduction Exposure, liability, or security posture materially improved. The value is a loss that did not occur, so the claim rests on an expected-loss estimate and should say so plainly. As seen in: [AI Value Classification](https://www.tokeneconomics.com/projects/ai-value-classification/) ### RLAIF **Concept** · Tokenomics > Production > Training > Post-Training > Alignment > RLAIF Uses AI-generated feedback in place of human labels to cut alignment cost. ### RLHF **Concept** · Tokenomics > Production > Training > Post-Training > Alignment > RLHF Reinforcement learning from human feedback, the standard preference-alignment method. ### ROCm **Concept** · Tokenomics > Production > Hardware > Software stack > ROCm AMD's equivalent stack, the main route to running the same workloads on non-NVIDIA accelerators. ### ROI **KPI** · Tokenomics > Value > Business Case > ROI The return generated relative to AI investment. ### Role ladder **Concept** · Tokenomics > Value > Business Outcomes > Value Measurement > Claim chain > Role ladder Each scope from IC to C-Suite answers the value question at a different range and has its own natural unit, hours per task at the bottom, P and L lines and market position at the top. Estimates add up within a scope, not across scopes. ### Rollback **Concept** · Tokenomics > Production > Serving > Inference Optimization > Speculative decoding > Mechanics > Rollback Discarding drafted tokens after the first rejection and resuming from the last verified position. ### Roofline analysis **Concept** · Tokenomics > Production > Hardware > Performance characterization > Roofline analysis Plotting achieved performance against arithmetic intensity to see how much headroom a kernel has. ### RoPE scaling and YaRN **Concept** · Tokenomics > Production > Model / Architecture > Context > Long-context methods > RoPE scaling and YaRN Rescaling rotary position frequencies so a model handles sequences longer than it was trained on. ### Router **Concept** · Tokenomics > Consumption > Optimization > Routing > Router The decision layer: picks which model or serving path handles each request from request properties or rules, such as query complexity, cache continuity, or right-sizing to capability tiers. It puts traffic on the rails rather than holding it. As seen in: [The Seven-Layer Tokenomics Stack (paper) (L5)](https://www.tokeneconomics.com/projects/the-seven-layer-tokenomics-stack-paper/) ### Router / gating **Concept** · Tokenomics > Production > Model / Architecture > Family > Mixture of Experts (MoE) > Router / gating The learned function that picks which experts see each token. ### Router overhead **Concept** · Tokenomics > Consumption > Optimization > Routing > Router overhead Classification adds its own latency and tokens, and spreading traffic across models splits one warm cache into several cold ones, which can erase the saving on short requests. ### Routing **Concept** · Tokenomics > Consumption > Optimization > Routing Directing each request to the model or serving path that meets the task's quality bar at the lowest cost, the decision layer over a mixed portfolio of hosted and open-weight models. A router answers the cost question against a quality bar set elsewhere; it does not define the bar. As seen in: [What Is AI Tokenomics?](https://www.tokeneconomics.com/what-is-ai-tokenomics/), [The Seven-Layer Tokenomics Stack (L5)](https://www.tokeneconomics.com/projects/the-seven-layer-tokenomics-stack/), [Cache Explainer](https://www.tokeneconomics.com/projects/cache-explainer/#tcModelSwitching), [Tokenomics Brief #16: Routing Wars](https://www.tokeneconomics.com/insights/podcast-16-of-tokenomics-brief-routing-wars-tokenomics-in-action/) ### Routing and caching interaction **Concept** · Tokenomics > Consumption > Optimization > Routing > Routing and caching interaction Provider caches are machine-local, so which machine a request reaches decides whether a cached prefix exists there; affinity fields tie a session to a machine, and routing across models with different tokenizers or hardware invalidates cache continuity entirely. A router that ignores cache placement silently converts cache hits into writes. Related: prefix stability and model version binding under Caching. ### Routing ownership **Concept** · Tokenomics > Consumption > Optimization > Routing > Routing ownership Who owns the router matters when the routing layer comes from a company that also sells models, since the picker can favor its own inventory. A material share of buyers build the layer in-house, which makes build versus buy read as a trust decision, not only a tooling gap. The routing decision can also carry a governance role (access policy, budgets, monitoring) depending on the router; where that role lives, in the router, a gateway, or a separate governance platform in front of everything, is unsettled. Related: aggregators and gateways under Access and Delivery. ## S ### Sampling and decoding controls **Concept** · Tokenomics > Consumption > Optimization > Sampling and decoding controls Request-level settings that change both output character and cost. ### Sampling and retention cost **Concept** · Tokenomics > Consumption > Observability > Practices > Sampling and retention cost Observability is itself an expense, and full-fidelity tracing of a high-volume token workload is not free. ### Saturation point **KPI** · Tokenomics > Production > Serving > Performance > Saturation point The load beyond which added traffic degrades latency without raising throughput. ### Scale **Concept** · Tokenomics > Production > Model / Architecture > Scale The size of the model, a primary driver of both quality and cost. ### Scale to zero **Concept** · Tokenomics > Production > Serving > Scaling > Scale to zero Releasing capacity entirely when idle, trading cold start latency for eliminated idle cost. ### Scale-up vs scale-out **Concept** · Tokenomics > Production > Hardware > Capacity > Networking > Scale-up vs scale-out Whether capacity grows by making one node bigger or by adding nodes, which changes the cost of parallelism. ### Scaling **Concept** · Tokenomics > Production > Serving > Scaling Adjusting serving capacity to match demand. ### Scaling laws **Concept** · Tokenomics > Production > Training > Pretraining > Scaling laws Empirical relationships predicting quality from model size, data, and compute. ### Scarcity premiums **Concept** · Tokenomics > Consumption > Purchase Options > Price dynamics > Scarcity premiums Prices reflecting availability rather than production cost during periods of constrained supply. ### Scheduling policy **Concept** · Tokenomics > Production > Serving > Batching > Scheduling policy The rules deciding which waiting requests join the next step, which sets the fairness and tail latency profile. ### Scheme **Concept** · Tokenomics > Production > Training > Quantization > Scheme How numeric ranges are mapped to lower precision. ### Score normalization **Concept** · Tokenomics > Consumption > Optimization > AI memory > Retrieval mechanics > Result fusion > Score-based fusion > Score normalization Rescaling each retriever's scores onto a shared range before they are combined, the prerequisite for any score-based method. Dense similarity is bounded while BM25 is not, so raw scores are not comparable and a fixed weight over them is dominated by whichever retriever has the larger magnitude on that query. ### Score-based fusion **Concept** · Tokenomics > Consumption > Optimization > AI memory > Retrieval mechanics > Result fusion > Score-based fusion Keeps the magnitude of each retriever's scores and combines the numbers directly. It preserves the information that one result beat the next by a wide margin rather than a hair, at the cost of depending on score distributions that shift per query, per corpus, and per model version. ### Seat-based **Concept** · Tokenomics > Value > Monetization > Seat-based Charging per user regardless of usage. ### Second-order / Shampoo **Concept** · Tokenomics > Production > Training > Pretraining > Optimizer > Second-order / Shampoo Uses curvature for faster convergence per step, at higher per-step compute and memory. ### Secondary market resale **Concept** · Tokenomics > Production > Hardware > Lifecycle (Intersections with HAM) > Secondary market resale Recovering value by selling used accelerators. ### Seeds and determinism **Concept** · Tokenomics > Consumption > Optimization > Sampling and decoding controls > Seeds and determinism Reproducibility controls, which matter for evaluation and for caching. ### Self-consistency **Concept** · Tokenomics > Consumption > Optimization > Sampling and decoding controls > Self-consistency Sampling several reasoning paths and taking the majority answer, buying accuracy with a fixed cost multiple. ### Self-correcting retrieval **Concept** · Tokenomics > Consumption > Optimization > AI memory > Knowledge injection patterns > Self-correcting retrieval The model judges whether retrieved passages are adequate and retrieves again or falls back to another source, trading extra calls for fewer wrong answers. ### Self-drafting / early exit **Concept** · Tokenomics > Production > Serving > Inference Optimization > Speculative decoding > Drafter designs > Self-drafting / early exit The target model drafts for itself by exiting at an intermediate layer or skipping layers, avoiding a second set of weights. ### Self-hosting **Concept** · Tokenomics > Consumption > Access & Delivery > Self-hosting Running open-weight models on owned or rented hardware, converting per-token cost into capacity cost. ### Semantic caching **Concept** · Tokenomics > Consumption > Optimization > Caching > Semantic caching Serving cached answers for semantically similar queries. ### Semantic routing **Concept** · Tokenomics > Consumption > Optimization > Routing > Semantic routing Routing by request meaning or category to the best-suited model. ### SentencePiece **Concept** · Tokenomics > Production > Tokenization > Implementations > SentencePiece Google's tokenizer library implementing byte-pair and Unigram models. ### SentencePiece / Unigram **Concept** · Tokenomics > Production > Tokenization > Algorithm > SentencePiece / Unigram A language-agnostic tokenizer that trains directly on raw text, common in multilingual models. ### Sequence / context parallelism **Concept** · Tokenomics > Production > Training > Distributed Training > Sequence / context parallelism Splits a single long sequence across devices so context length is not bounded by one device's memory. ### Serial draft then verify **Concept** · Tokenomics > Production > Serving > Inference Optimization > Speculative decoding > Execution topology > Serial draft then verify The classic arrangement, where the drafter and target take turns on the same device; larger drafts raise acceptance but add latency on the critical path. ### Serving **Concept** · Tokenomics > Production > Serving Running trained models to produce tokens for users. Also called inference, and the two words are used interchangeably: model building is training, everything after it that answers users is serving. ### Serving orchestration **Concept** · Tokenomics > Production > Serving > Inference Engines > Serving orchestration Infrastructure that manages models and traffic at scale. ### Session cost variance **Concept** · Tokenomics > Consumption > Workloads / Use Cases > Coding agents > Session cost variance The spread in spend across runs of the same task, wide enough to make per-task budgeting unreliable. ### SGLang **Concept** · Tokenomics > Production > Serving > Inference Engines > Datacenter engines > SGLang An engine optimized for structured generation and fast prefix reuse. ### SGLang **Concept** · Tokenomics > Production > Serving > Inference Optimization > Speculative decoding > Engine support > SGLang Ships EAGLE family speculative decoding alongside its structured generation and prefix reuse features. ### Shared experts **Concept** · Tokenomics > Production > Model / Architecture > Family > Mixture of Experts (MoE) > Shared experts Experts every token passes through, used to hold general capability outside the routed pool. ### Short vs long-term memory **Concept** · Tokenomics > Consumption > Optimization > AI memory > Short vs long-term memory Distinguishing per-session context from persistent recall. ### Signals **Concept** · Tokenomics > Consumption > Observability > Signals What is worth measuring once the data is available. ### Similarity metric **Concept** · Tokenomics > Consumption > Optimization > AI memory > Retrieval mechanics > Dense retrieval > Similarity metric Cosine, dot product, or Euclidean distance. The correct choice is set by how the embedding model was trained rather than by preference, and using the wrong one degrades retrieval silently rather than failing. ### Single vs multi-agent tradeoff **Concept** · Tokenomics > Consumption > Optimization > Agentic > Sub-agents / orchestration > Single vs multi-agent tradeoff Comparing one agent with many at the same total token budget separates gains from coordination from gains due simply to extra spend. ### Siting and latency **Concept** · Tokenomics > Production > Hardware > Capacity > Facilities > Siting and latency Where capacity physically sits, which decides achievable response times and which data residency rules apply. ### Sliding-window / sparse attention **Concept** · Tokenomics > Production > Model / Architecture > Attention > Sliding-window / sparse attention Limits attention to nearby or selected tokens to reduce cost on long inputs. ### SLOs **KPI** · Tokenomics > Production > Serving > Performance > SLOs Service level objectives that set target latency and availability. ### Small vs large routing **Concept** · Tokenomics > Consumption > Optimization > Routing > Small vs large routing Sending easy requests to small models and hard ones to large models. ### SmoothQuant **Concept** · Tokenomics > Production > Training > Quantization > Algorithms > SmoothQuant Shifts difficulty from activations into weights so both can be quantized together. ### Software stack **Concept** · Tokenomics > Production > Hardware > Software stack The vendor libraries and toolchains between a model and the silicon, which decide portability as much as performance. ### Sourcing model (Cloud vs on-prem vs hybrid) **Concept** · Tokenomics > Production > Hardware > Procurement > Sourcing model (Cloud vs on-prem vs hybrid) The basic decision of renting, owning, or mixing infrastructure. ### Space **Concept** · Tokenomics > Production > Hardware > Capacity > Facilities > Space Floor and rack space available for equipment. ### Sparse retrieval **Concept** · Tokenomics > Consumption > Optimization > AI memory > Retrieval mechanics > Sparse retrieval Scoring on term overlap, also called keyword or lexical search, which handles names, codes, identifiers, and rare terms that embeddings blur. It runs on an inverted index with no model call at query time, so marginal cost per query is near zero once the index is built. ### Special tokens **Concept** · Tokenomics > Production > Tokenization > Vocabulary > Special tokens Reserved tokens such as end-of-sequence, padding, and chat roles that carry structural meaning. ### Speculation length **Concept** · Tokenomics > Production > Serving > Inference Optimization > Speculative decoding > Tuning knobs > Speculation length How many tokens to propose per step; too few wastes the parallel verify, too many wastes drafting on tokens that get rejected. ### Speculative decoding **Concept** · Tokenomics > Production > Serving > Inference Optimization > Speculative decoding Proposes several tokens ahead with a cheap drafter, then verifies them in a single target model pass, cutting per token latency without changing the output distribution. Sources: ### Speculative Speculative Decoding (SSD) **Concept** · Tokenomics > Production > Serving > Inference Optimization > Speculative decoding > Execution topology > Speculative Speculative Decoding (SSD) The drafter predicts likely verification outcomes and prepares speculations for them while verification is still running, so drafting overhead disappears when the guess is right; Saguaro is the optimised algorithm. Sources: ### Speech to text (ASR) **Concept** · Tokenomics > Production > Model / Architecture > Modality > Audio / speech > Speech to text (ASR) Automatic speech recognition, usually priced per second or minute of audio rather than per token. ### Speed to Market **KPI** · Tokenomics > Value > Business Outcomes > Value Categorization > Speed to Market Faster delivery of changes. Cycle time is usually available, what a week of earlier delivery is worth usually is not. Resolves into revenue pulled forward or spend avoided. For software, release-level throughput and lead time are the observable forms of this category. As seen in: [AI Value Classification](https://www.tokeneconomics.com/projects/ai-value-classification/) ### Spend predictability **Concept** · Tokenomics > Consumption > Demand > Spend predictability How reliably future token spend can be forecast from past usage, workload plans, and changes in agent behaviour. ### Spot and preemptible **Concept** · Tokenomics > Consumption > Purchase Options > Capacity and commitment > Spot and preemptible Discounted capacity that can be reclaimed, viable only for interruption-tolerant work. ### Stage metrics **Concept** · Tokenomics > Value > Business Outcomes > Release-level throughput > Stage metrics Pull request counts, review wait, queue depth, and CI time locate the constraint but do not show value. Speeding up authoring moves the constraint downstream: in one studied enterprise of 802 developers (He et al., arXiv 2607.01904), a mandate to double merged pull requests met its target while review load roughly doubled and end-to-end cycle time rose. Sources: , ### Staleness and invalidation **Concept** · Tokenomics > Consumption > Optimization > AI memory > Knowledge injection patterns > CAG (Cache-Augmented Generation) > Staleness and invalidation The cache is precomputed, so any document change requires rebuilding it, making update frequency the deciding factor against retrieval. ### Standards / Open Specifications **Concept** · Tokenomics > Consumption > Observability > Standards / Open Specifications Shared schemas that make measurement portable between tools and vendors. ### State-space / hybrid (Mamba, etc.) **Concept** · Tokenomics > Production > Model / Architecture > Family > State-space / hybrid (Mamba, etc.) Non-attention or hybrid architectures that scale more cheaply with sequence length. ### Stated stopping point **Concept** · Tokenomics > Value > Business Outcomes > Value Measurement > Claim chain > Convert > Stated stopping point The conversion is available, not mandatory: teams managing in time may stop at the delta and report hours or FTE equivalents. The chain requires the claim to name where it stopped, so a freed hour is never quietly counted as a booked dollar. Whether conversion to dollars is optional or mandatory is still open in the working group. ### Static batching **Concept** · Tokenomics > Production > Serving > Batching > Static batching Fixed batches formed before processing, simple but less efficient. ### Step and turn budgets **KPI** · Tokenomics > Consumption > Optimization > Agentic > Step and turn budgets Hard caps on loop iterations, the main defence against an agent spending without bound. ### Stop sequence matching **Concept** · Tokenomics > Production > Tokenization > Detokenization and streaming > Stop sequence matching Stop strings are matched on text rather than tokens, so generation can overshoot slightly before halting. ### Storage **Concept** · Tokenomics > Production > Hardware > Capacity > Memory > Storage Disk capacity for weights, datasets, and checkpoints. ### Streaming vs non-streaming **Concept** · Tokenomics > Consumption > Access & Delivery > Streaming vs non-streaming Whether tokens stream as generated or return all at once. ### Structured output enforcement **Concept** · Tokenomics > Production > Serving > Inference Optimization > Structured output enforcement Constraining decoding to a grammar or schema, which improves reliability and can reduce wasted retries. ### Structured outputs (JSON mode, grammars) **Concept** · Tokenomics > Consumption > Prompt / Context Engineering > Structured outputs (JSON mode, grammars) Constraining output format to reduce retries and wasted tokens. ### Sub-agent fan-out **Concept** · Tokenomics > Consumption > Optimization > Agentic > Token amplification > Sub-agent fan-out Delegating to parallel sub-agents multiplies concurrent spend and is the fastest way to an unexpected bill. ### Sub-agents / orchestration **Concept** · Tokenomics > Consumption > Optimization > Agentic > Sub-agents / orchestration Coordinating multiple agents, each adding token overhead. ### Subscription / tiers **Concept** · Tokenomics > Value > Monetization > Subscription / tiers Recurring plans with bundled usage limits. ### Subscription feedback asymmetry **Concept** · Tokenomics > Consumption > Optimization > Caching > Cache access modes > Subscription feedback asymmetry On flat plans cache misses surface as rate limits rather than line items, so the feedback channel is throttling without explanation and cache economics must be inferred rather than observed. ### Subscription vs metered access **Concept** · Tokenomics > Consumption > Workloads / Use Cases > Coding agents > Subscription vs metered access Whether agent usage is bought as a flat plan or paid per token, which decides who absorbs usage variance. ### Suffix decoding **Concept** · Tokenomics > Production > Serving > Inference Optimization > Speculative decoding > Drafter designs > Retrieval and n-gram drafting > Suffix decoding Builds a suffix tree over prior text and drafts from the best matching branch. ### Summarization **Concept** · Tokenomics > Consumption > Workloads / Use Cases > Summarization Condensing long inputs, input-heavy and output-light. ### Summarization and rolling memory **Concept** · Tokenomics > Consumption > Prompt / Context Engineering > Context window management > Summarization and rolling memory Replacing old turns with a compact summary, trading a one-off generation cost for lower ongoing input cost. ### Supervised fine-tuning (SFT) **Concept** · Tokenomics > Production > Training > Post-Training > Fine Tuning > Supervised fine-tuning (SFT) Trains a base model on curated prompt-response pairs, the supervised step before preference alignment that yields the instruction-tuned model. ### Supply chain **Concept** · Tokenomics > Value > Market Structure > Supply chain Silicon, memory, power, and facilities, each with its own lead time and constraint. ### Sustainability **Concept** · Tokenomics > Production > Sustainability The energy and environmental accounting of token production, increasingly a reported figure rather than an internal one. ### Symmetric quantization **Concept** · Tokenomics > Production > Training > Quantization > Scheme > Symmetric quantization Maps values around zero without an offset, simpler and faster. ### Sync vs async **Concept** · Tokenomics > Consumption > Access & Delivery > Sync vs async Whether the caller waits or is notified when results are ready. ### Synchronous API **Concept** · Tokenomics > Consumption > Access & Delivery > Synchronous API Standard request-response calls at full price. ### Synthetic data **Concept** · Tokenomics > Production > Training > Pretraining > Synthetic data Model-generated data used to augment or replace scarce real data. ### Synthetic data and evaluation **Concept** · Tokenomics > Consumption > Workloads / Use Cases > Synthetic data and evaluation Using models to generate training data or to grade other models, an internal workload easily larger than production traffic. ### System card **Concept** · Tokenomics > Production > Model / Architecture > Openness > System card A fuller disclosure covering evaluations and safety testing for a released system, increasingly the reference for procurement review. ### System prompt **Concept** · Tokenomics > Consumption > Prompt / Context Engineering > Prompt design (few-shot, system prompts) > System prompt Standing instructions sent with every request, a fixed token cost paid on every call unless a provider cache serves it at the cached read rate. ## T ### Tag propagation **Concept** · Tokenomics > Consumption > Observability > Practices > Tag propagation Carrying identity from the caller through gateways to the billing record, without which attribution breaks at the first hop. ### Tail latency **KPI** · Tokenomics > Production > Serving > Performance > Tail latency Behaviour at the slow end of the distribution, which governs user experience far more than the average does. ### Task-level regression **Concept** · Tokenomics > Production > Training > Quantization > Quality impact > Task-level regression Measuring the quantized model on the actual workload, the only check that catches damage perplexity misses. ### Task-specific criteria **Concept** · Tokenomics > Value > Business Outcomes > Value Measurement > Quality Floor > Task-specific criteria Code generation and creative summarization have different correctness criteria, so the floor is defined per workload. Leaderboard scores are a starting point for model selection, not a substitute for measuring on your own workload. ### TCA **KPI** · Tokenomics > Value > Business Case > TCA The Total Cost of AI across tokens, hardware, power, licensing, and the labor to build and operate the AI platform. A value claim does not net these; see Cost boundary under Value Measurement for the boundary and the grain at which a claim and a cost compose into a return. As seen in: [Total Cost of AI (TCA)](https://www.tokeneconomics.com/insights/total-cost-of-ai-tca/) ### Telemetry sources **Concept** · Tokenomics > Consumption > Observability > Telemetry sources Where the numbers come from, each with different coverage and trust. ### Temperature and top-p **Concept** · Tokenomics > Consumption > Optimization > Sampling and decoding controls > Temperature and top-p Controls on randomness, which affect consistency and therefore retry rates. ### Tensor parallelism **Concept** · Tokenomics > Production > Training > Distributed Training > Tensor parallelism Splits individual layers across devices to fit large models. ### TensorRT-LLM **Concept** · Tokenomics > Production > Serving > Inference Engines > Datacenter engines > TensorRT-LLM NVIDIA's optimized inference library for its GPUs. ### TensorRT-LLM **Concept** · Tokenomics > Production > Serving > Inference Optimization > Speculative decoding > Engine support > TensorRT-LLM NVIDIA's engine, with native speculative decoding support across draft model, EAGLE, and MTP style paths. ### Test-time scaling **Concept** · Tokenomics > Production > Model / Architecture > Reasoning and test-time compute > Test-time scaling Improving results by spending more inference compute rather than training a larger model, moving cost from capex to per-request spend. ### Text **Concept** · Tokenomics > Production > Model / Architecture > Modality > Text Text-only input and output, the baseline modality. ### Text to speech (TTS) **Concept** · Tokenomics > Production > Model / Architecture > Modality > Audio / speech > Text to speech (TTS) Speech synthesis, priced per character or per second of generated audio. ### TF-IDF **Concept** · Tokenomics > Consumption > Optimization > AI memory > Retrieval mechanics > Sparse retrieval > TF-IDF Weights a term by how often it appears in a document against how rare it is across the corpus. The original lexical weighting scheme, superseded in practice by BM25 but still the mental model most people carry for keyword search. ### TFLOPS **KPI** · Tokenomics > Production > Hardware > Capacity > Compute > TFLOPS Rated floating point operations per second, quoted per precision format and only reachable under ideal conditions. ### TGI **Concept** · Tokenomics > Production > Serving > Inference Engines > Datacenter engines > TGI Hugging Face's Text Generation Inference server. ### Thermal design power (TDP) **KPI** · Tokenomics > Production > Hardware > Capacity > Compute > Thermal design power (TDP) The power draw a device is designed to sustain, which sets the facility requirement per accelerator. ### Third-party tools **Concept** · Tokenomics > Consumption > Optimization > Caching > Cache access modes > Third-party tools Products that rehost frontier models behind a proxy assemble the prompt themselves and bill in credits or requests, netting cache savings into a pooled price so they never appear on the customer's side of the glass. ### Throughput vs latency **Concept** · Tokenomics > Production > Serving > Performance > Throughput vs latency The core serving tradeoff between total tokens per second and per-request speed. ### Tier progression **Concept** · Tokenomics > Consumption > Access & Delivery > Rate limits / quotas > Tier progression Limits that rise with spend history, making early scaling a function of account age as well as need. ### Tier-aware routing **Concept** · Tokenomics > Production > Serving > KV Cache > Cache Management > Hierarchical KV cache > Tier-aware routing A router that knows which tier a prefix lives on, so a worker holding the prefix in host memory or shared storage is not scored as cold. Without it, a fetchable cache hit is treated as a full recomputation. Related: cache-aware routing under Routing. ### Tiered / volume pricing **Concept** · Tokenomics > Consumption > Purchase Options > Price modifiers > Tiered / volume pricing Unit prices that fall with committed volume. ### tiktoken **Concept** · Tokenomics > Production > Tokenization > Implementations > tiktoken OpenAI's fast byte-pair-encoding tokenizer library. ### Token amplification **Concept** · Tokenomics > Consumption > Optimization > Agentic > Token amplification The way agent loops turn one task into many model calls and tokens. ### Token bucket limiting **Concept** · Tokenomics > Consumption > Access & Delivery > Client-side resilience > Token bucket limiting Client-side pacing against a refilling allowance, which smooths bursts before the provider rejects them. ### Token budgeting / counting **Concept** · Tokenomics > Consumption > Prompt / Context Engineering > Token budgeting / counting Tracking token use per call to forecast and cap spend. ### Tokenization **Concept** · Tokenomics > Production > Tokenization How raw text and media are converted into the tokens that get billed and processed. ### Tokenizer parity across providers **Concept** · Tokenomics > Production > Tokenization > Efficiency > Tokenizer parity across providers The same text costs different token counts on different models, so quoted prices per token are not directly comparable. ### Tokenizer-free approaches **Concept** · Tokenomics > Production > Tokenization > Algorithm > Tokenizer-free approaches Research directions that learn dynamic byte groupings instead of a fixed vocabulary, removing the tokenizer as a cost variable. ### Tokenomics **Concept** · Tokenomics AI Tokenomics (i.e., "AI Economics") is the discipline of converting energy and capital into AI capabilities and efficiently consuming that intelligence across the organization, to realize measurable business value. (Draft v0.5.2, September 2026) As seen in: [What Is AI Tokenomics?](https://www.tokeneconomics.com/what-is-ai-tokenomics/) ### Tokens per dollar **KPI** · Tokenomics > Value > Unit Economics > Efficiency ratios > Tokens per dollar A throughput-per-cost efficiency metric and core optimization target. ### Tokens per GPU-hour **KPI** · Tokenomics > Value > Unit Economics > Efficiency ratios > Tokens per GPU-hour How many tokens a GPU produces per hour, a utilization measure. ### Tokens per second per accelerator **KPI** · Tokenomics > Value > Unit Economics > Efficiency ratios > Tokens per second per accelerator Instantaneous production rate, which combined with hourly cost yields cost per token directly. ### Tokens per watt **KPI** · Tokenomics > Value > Unit Economics > Efficiency ratios > Tokens per watt Token output per unit of energy, an efficiency and sustainability measure. ### Tokens-per-word ratio **KPI** · Tokenomics > Production > Tokenization > Efficiency > Tokens-per-word ratio How many tokens an average word consumes, a direct multiplier on cost. ### Tool and function schemas **Concept** · Tokenomics > Consumption > Prompt / Context Engineering > Structured outputs (JSON mode, grammars) > Tool and function schemas Declared call signatures the model selects from, which occupy input tokens on every request that offers them. ### Tool and protocol layer **Concept** · Tokenomics > Consumption > Optimization > Agentic > Tool and protocol layer How an agent reaches the outside world, and the standards forming around it. ### Tool call round trips **Concept** · Tokenomics > Consumption > Optimization > Agentic > Token amplification > Tool call round trips Every tool invocation costs a generation to call it and another pass to interpret the result. ### Tool-call overhead **Concept** · Tokenomics > Consumption > Optimization > Agentic > Tool-call overhead The extra tokens spent describing tools and parsing their results. ### Top-k routing **Concept** · Tokenomics > Production > Model / Architecture > Family > Mixture of Experts (MoE) > Top-k routing How many experts each token is sent to, setting active compute per token. ### Top-k sampling **Concept** · Tokenomics > Consumption > Optimization > Sampling and decoding controls > Top-k sampling Restricting choice to the k most likely tokens before sampling. ### TPOT / ITL (inter-token latency) **KPI** · Tokenomics > Production > Serving > Performance > TPOT / ITL (inter-token latency) The time between successive output tokens. ### Traces **Concept** · Tokenomics > Consumption > Observability > Telemetry sources > Traces Per-request spans across services, the only way to attribute cost inside a multi-step agent run. ### Training **Concept** · Tokenomics > Production > Training The process of creating and refining a model. ### Training compute cost **KPI** · Tokenomics > Production > Training > Operations > Training compute cost The total accelerator spend to train a model. ### Training economics **Concept** · Tokenomics > Production > Training > Training economics How training spend relates to what it eventually returns. ### Training token budget **KPI** · Tokenomics > Production > Training > Pretraining > Training token budget How many tokens the model sees during training, traded against parameter count under a fixed compute budget. ### Trajectory observability **Concept** · Tokenomics > Consumption > Optimization > Agentic > Trajectory observability Tracing every model call inside a run, without which agent cost cannot be attributed to a cause. ### Transcription and diarization **Concept** · Tokenomics > Consumption > Workloads / Use Cases > Audio and speech workloads > Transcription and diarization Turns recorded speech into labelled text with speakers identified, priced by length of audio rather than by prompt size. ### Translation **Concept** · Tokenomics > Consumption > Workloads / Use Cases > Translation Converting between languages, where tokenizer efficiency varies sharply by language pair. ### Tree drafting and tree attention **Concept** · Tokenomics > Production > Serving > Inference Optimization > Speculative decoding > Mechanics > Tree drafting and tree attention Proposing a branching tree of candidates rather than one chain, raising the odds that some path is accepted. ### Tree of Thoughts **Concept** · Tokenomics > Production > Model / Architecture > Reasoning and test-time compute > Tree of Thoughts Explores and prunes branching reasoning paths, multiplying token spend for a chance at harder problems. ### Triton (kernel language) **Concept** · Tokenomics > Production > Hardware > Software stack > Triton (kernel language) A Python-like language for writing GPU kernels, distinct from the inference server of the same name. ### Triton Inference Server **Concept** · Tokenomics > Production > Serving > Inference Engines > Serving orchestration > Triton Inference Server NVIDIA's server for deploying and managing many models. ### Truncation and sliding windows **Concept** · Tokenomics > Consumption > Prompt / Context Engineering > Context window management > Truncation and sliding windows Dropping the oldest turns once the window fills, the simplest and most lossy strategy. ### TTFT (Time To First Token) **KPI** · Tokenomics > Production > Serving > Performance > TTFT (Time To First Token) How long until the first output token appears, driving perceived responsiveness. ### Tuning knobs **Concept** · Tokenomics > Production > Serving > Inference Optimization > Speculative decoding > Tuning knobs The settings that decide whether speculation pays off in a given deployment. ## U ### Unbounded usage **Concept** · Tokenomics > Value > Monetization > Margin risk > Unbounded usage Flat-price plans meeting agentic workloads, where a small number of users can consume far beyond the average. ### Unified and shared memory **Concept** · Tokenomics > Production > Hardware > Capacity > Memory > Unified and shared memory Architectures where host and accelerator share an address space, changing what fits on smaller systems. ### Unit Economics **Concept** · Tokenomics > Value > Unit Economics The per-unit costs and efficiency ratios behind token spend. As seen in: [Enterprise Tokenomics in Practice at MetLife](https://www.tokeneconomics.com/insights/enterprise-tokenomics-in-practice-at-metlife/) ### Unit of measure **KPI** · Tokenomics > Value > Business Outcomes > Value Measurement > Unit of measure Dollars, hours, or FTE equivalents, chosen per audience. Dollars read to executives, hours and FTE equivalents read to operational leads, and some outcomes deliberately stay unconverted. ### Usage & cost dashboards **Concept** · Tokenomics > Consumption > Observability > Signals > Usage & cost dashboards Live views of token consumption and spend. ### Usage-based / metered **Concept** · Tokenomics > Value > Monetization > Usage-based / metered Charging customers in proportion to tokens or calls used. ### Utilization **KPI** · Tokenomics > Value > Unit Economics > Cost drivers > Utilization The share of paid capacity doing useful work, the dominant driver for self-hosted serving. ### Utilization breakeven **KPI** · Tokenomics > Consumption > Purchase Options > Capacity and commitment > Utilization breakeven The usage level at which a commitment beats on-demand, the calculation that should precede any reservation. ### Utilization tracking **KPI** · Tokenomics > Consumption > Observability > Signals > Utilization tracking Measuring how fully hardware is used, the key to cost efficiency. ### Utilization tracking **KPI** · Tokenomics > Production > Hardware > Lifecycle (Intersections with HAM) > Utilization tracking Measuring how fully hardware is used, the key to cost efficiency. ## V ### Value **Concept** · Tokenomics > Value Owns the last part of Tokenomics, measuring the business value AI produces so investment is judged on outcomes rather than cost alone. Cost and utilization arrive from Production and Consumption as supporting scaffolding, not as the claim itself. As seen in: [What Is AI Tokenomics?](https://www.tokeneconomics.com/what-is-ai-tokenomics/), [Personas and Operating Model Map](https://www.tokeneconomics.com/projects/personas-and-operating-model-map/), [AI Value Classification](https://www.tokeneconomics.com/projects/ai-value-classification/) ### Value case gate **Concept** · Tokenomics > Value > Business Case > Value case gate A measurable value case before a new workload goes live: what the process costs today in money or time, which specific financial outcome the AI improves, and how that outcome will be measured in dollars. A business case that cannot express its expected value in financial terms before launch is an experiment, and should be funded with explicit scope limits and a defined point at which it produces evidence or stops. ### Value Categorization **Concept** · Tokenomics > Value > Business Outcomes > Value Categorization What the AI produces, framed for conversation rather than fixed. The fusion draft groups the ten categories by how the number is produced: direct dollar wins, labor categories estimated in hours or FTE, and reported outcomes. The groups differ in method, not worth, and no category outranks another. As seen in: [AI Value Classification](https://www.tokeneconomics.com/projects/ai-value-classification/) ### Value claim components **Concept** · Tokenomics > Value > Business Outcomes > Value Measurement > Value claim components A full claim names the workload or task type, the outcome measured, the unit, the baseline, and the quality the result was judged at. ### Value Measurement **Concept** · Tokenomics > Value > Business Outcomes > Value Measurement How a value claim is constructed and defended. The categories name what changed; these nodes are the how. ### Value per token (quality, task completion) **KPI** · Tokenomics > Value > Business Case > Value per token (quality, task completion) The business value a token delivers, not just its cost. ### Vector store **Concept** · Tokenomics > Consumption > Optimization > AI memory > Retrieval mechanics > Vector store The database that holds embeddings and serves the index, distinct from the search algorithm running inside it. Cost is driven by whether the index stays resident in memory, how metadata filtering interacts with the search, and whether it is hosted or embedded. ### Vendor lock-in **Concept** · Tokenomics > Production > Hardware > Procurement > Vendor lock-in Dependence on one provider's hardware, software, or pricing. ### Verification **Concept** · Tokenomics > Production > Serving > Inference Optimization > Speculative decoding > Mechanics > Verification The target model scores all candidate positions in one forward pass, so accepted tokens cost roughly one decode step. ### Version and weight rollout **Concept** · Tokenomics > Production > Serving > Reliability > Version and weight rollout Changing model versions without downtime, and the cost of running two versions during a cutover. ### Video **Concept** · Tokenomics > Production > Model / Architecture > Modality > Video Handles moving images, the most token-intensive input a model can accept. ### Video tokens **Concept** · Tokenomics > Production > Tokenization > Multimodal tokenization > Video tokens Video is tokenized per frame or clip, the most token-intensive modality. ### Video understanding **Concept** · Tokenomics > Consumption > Workloads / Use Cases > Video workloads > Video understanding Summarizes or answers questions over recorded video, where frame rate and clip length drive cost more than prompt length does. ### Video workloads **Concept** · Tokenomics > Consumption > Workloads / Use Cases > Video workloads Applications built on moving images, the heaviest input per minute of source content. ### Vision / multimodal **Concept** · Tokenomics > Production > Model / Architecture > Modality > Vision / multimodal Handles images alongside text, adding vision token costs. ### Vision workloads **Concept** · Tokenomics > Consumption > Workloads / Use Cases > Vision workloads Applications built on image input, billed as tiles plus output tokens. ### Visual question answering (VQA) **Concept** · Tokenomics > Consumption > Workloads / Use Cases > Vision workloads > Visual question answering (VQA) Answers questions grounded in an image, combining tile input with reasoning-length output. ### vLLM **Concept** · Tokenomics > Production > Serving > Inference Engines > Datacenter engines > vLLM A high-throughput open-source engine known for PagedAttention. ### vLLM **Concept** · Tokenomics > Production > Serving > Inference Optimization > Speculative decoding > Engine support > vLLM Exposes speculation as a configured method, covering draft models, EAGLE and EAGLE-3, MTP, MLP and parallel draft models, n-gram, and suffix decoding. Sources: ### Vocab / embedding tradeoff **Concept** · Tokenomics > Production > Tokenization > Vocabulary > Vocab / embedding tradeoff Larger vocabularies shorten sequences but enlarge the embedding table and memory footprint. ### Vocab size **KPI** · Tokenomics > Production > Tokenization > Vocabulary > Vocab size The number of distinct tokens, trading model size against how efficiently text compresses. ### Vocabulary **Concept** · Tokenomics > Production > Tokenization > Vocabulary The fixed set of tokens a model recognizes. ### Vocabulary and tokenizer alignment **Concept** · Tokenomics > Production > Serving > Inference Optimization > Speculative decoding > Tuning knobs > Vocabulary and tokenizer alignment Drafter and target must share a tokenizer and vocabulary, or need explicit mapping, which constrains which pairs can be used. ### Voice agents **Concept** · Tokenomics > Consumption > Workloads / Use Cases > Audio and speech workloads > Voice agents Interactive speech to speech assistants, where continuous audio and a strict latency budget make cost per conversation minute the unit that matters. ## W ### Waste attribution **Concept** · Tokenomics > Consumption > Optimization > Agentic > Waste attribution Splitting an agent run's spend into what the task inherently required and what the loop added, so optimization targets the removable part. ### Wasted draft compute **Concept** · Tokenomics > Production > Serving > Inference Optimization > Speculative decoding > Economics > Wasted draft compute Rejected tokens consume real compute and energy, so goodput rather than raw throughput is the honest measure. ### Water use **KPI** · Tokenomics > Production > Sustainability > Water use Consumption associated with cooling, a growing constraint on where capacity can be sited. ### Watermarking / provenance **Concept** · Tokenomics > Consumption > Guardrails > Watermarking / provenance Marks or certifies output for origin tracing under schemes such as SynthID and C2PA, an emerging compliance cost that adds per-request processing. ### Web UI and chat apps **Concept** · Tokenomics > Consumption > Optimization > Caching > Cache access modes > Web UI and chat apps The provider's own chat surface assembles the prompt entirely and reports no token buckets at all; caching is folded into plan economics and stays invisible to the user by design. ### Weight offloading **Concept** · Tokenomics > Production > Serving > Inference Optimization > Memory hierarchy and offloading > Weight offloading Holding part of the weights in host memory or on disk and moving them in as needed, trading interconnect bandwidth for capacity. ### Weight-only **Concept** · Tokenomics > Production > Training > Quantization > What gets quantized > Weight-only Compresses stored weights while computing at higher precision, the most common inference setup. ### Weighted RRF **Concept** · Tokenomics > Consumption > Optimization > AI memory > Retrieval mechanics > Result fusion > Rank-based fusion > Weighted RRF Multiplying each list's reciprocal-rank contribution by a weight before summing, which lets one retriever count for more without reintroducing score comparison. Qdrant accepts a weight pair alongside k on its RRF query. ### Weighted score combination **Concept** · Tokenomics > Consumption > Optimization > AI memory > Retrieval mechanics > Result fusion > Score-based fusion > Weighted score combination Multiplying each normalized list by a weight and summing, the general form that relative score fusion and Weaviate's alpha are instances of. Elasticsearch implements it as the linear retriever, with a per-retriever weight and normalizer. The weights are workload-specific, which makes this the branch carrying ongoing tuning cost. Sources: ### WEKA Augmented Memory Grid **Concept** · Tokenomics > Production > Serving > KV Cache > Cache Management > Hierarchical KV cache > KV cache storage tier > WEKA Augmented Memory Grid A storage vendor's persistent KV cache tier on NVMe, served at near-memory latency and marketed as a token warehouse. Sources: ### What gets quantized **Concept** · Tokenomics > Production > Training > Quantization > What gets quantized Different parts of the model can be reduced independently. ### WordPiece **Concept** · Tokenomics > Production > Tokenization > Algorithm > WordPiece A subword scheme used by BERT-family models that picks merges by likelihood rather than raw frequency. ### Workflow grain **Concept** · Tokenomics > Value > Business Case > Workloads / Use Cases > Workflow grain Value compounds across a whole workflow: the model, the retrieved data, the infrastructure, the engineer who designed the workflow, and the employee who used it. No measurement isolates the share belonging to the AI alone, so claims scope to the workflow outcome, not to individual calls. ### Workload characteristics **Concept** · Tokenomics > Consumption > Workloads / Use Cases > Workload characteristics The properties that determine which optimizations and commitments make sense. ### Workloads / Use Cases **Concept** · Tokenomics > Consumption > Workloads / Use Cases The application types that consume AI capacity, each with a distinct cost profile. ### Workloads / Use Cases **Concept** · Tokenomics > Value > Business Case > Workloads / Use Cases The unit at which value is attributed: a named workload with an owner, best framed as a workflow or business process rather than a single-shot, single-user task. The same unit anchors the cost side under Consumption; here it is the boundary the outcome is measured at, and a use case is its business-facing view. ### Write policy **Concept** · Tokenomics > Production > Serving > KV Cache > Cache Management > Hierarchical KV cache > Write policy Whether cache entries are written down to slower tiers as they are created or only on eviction, which sets how much bandwidth is spent on entries that are never read back. ## Z ### ZeRO and FSDP **Concept** · Tokenomics > Production > Training > Distributed Training > ZeRO and FSDP Shards optimizer state, gradients, and parameters across data-parallel ranks to cut memory per device. ### Zero-shot instruction **Concept** · Tokenomics > Consumption > Prompt / Context Engineering > Prompt design (few-shot, system prompts) > Zero-shot instruction Relying on instruction alone with no examples, the cheapest prompt shape.