Author: 0xSammy (Khala Research)
Compilation: AIdidiaoJP, Foresight News
The current AI inference market is no longer like a single cloud service market but more like a game of "Risk." Each provider is competing for different territories: hyperscalers control the corporate continent, routers dominate trade routes, and decentralized networks are fighting hard on the open frontier.
The core of the last AI cycle was model training, but it's becoming increasingly clear that the inference stage holds enormous economic value. Many may be hearing the term "inference" for the first time—so what exactly is it?
Training creates AI models, while inference is the process where the model generates answers when someone asks it a question or gives it a task.
Overview of the AI Inference Market
The training stage grabs headlines because it supports those amazing outputs. But in reality, inference currently takes the lion's share of economic benefits—every prompt, agent loop, image generation, transaction execution, tool call, and code edit must run somewhere.
Routers Are the Real Bottleneck
In the game of "Risk," the most valuable territories are often the narrow bottlenecks that determine where armies move next. In the inference market, routers play exactly the same role. They sit between demand and supply, deciding where each request goes and which provider gets paid.
A prime example is OpenRouter, whose protocol processed 4700 trillion tokens last week.
There's no sign of this economic activity slowing down, especially with trillions of agents about to come online.
So, what does a complete inference market need? Core components include:
- Tokens are becoming the unit of account.
- OpenRouter is rapidly becoming the core exchange layer, with token volume used through its LLM market reaching 4700 trillion last week.
- Specialized supply side: Fireworks, Together, Replicate, Baseten, Groq, and the major hyperscalers.
- Crypto AI networks: Projects like Chutes, Akash, io.net, Nosana, Targon, Venice, and NuNet are building the permissionless version of the underlying infrastructure.
Don't think of all these providers as competing in the same market—they simply aren't.
Traditional providers sell reliability, developer experience, and enterprise procurement processes.
Crypto AI networks emphasize cheaper supply, open access, privacy, verifiability, and novel incentive loops.
Anthropic's recent ban on users outside the US from accessing its Mythos model (Fable 5) has made many people reconsider the risks of over-reliance on a single frontier proprietary model.
Interestingly, the two worlds are starting to overlap: privacy, confidential computing, or agent-native payments (Venice and Targon are particularly strong here).
How to View the AI Compute Market
A better perspective is to divide the market into traditional and crypto camps:
The traditional side sells reliability, developer experience, and enterprise procurement.
Crypto networks primarily compete on open access, lower-cost supply, privacy, verifiability, and new incentive mechanisms to coordinate capital seamlessly on a global scale.
Why Inference is the Real AI Market
The model layer is still important, but model quality is compressing faster than expected. Open-source models have reached 90-95% of the quality of frontier models at only 10% of the cost (e.g., Z.ai's GLM-5.2).
Open-source models keep iterating, and Chinese labs keep pushing prices down. Frontier models can still maintain a premium, but below that, token pricing competition is already fierce.
This is exactly why the routing layer becomes crucial: the same open-source model might be offered by five different providers at five different prices. Developers don't want to hardcode a single endpoint forever; they need routers.
Routers can choose based on price, latency, privacy, reliability, and many other factors.
It sits above all providers, turning a chaotic landscape into a clean, unified interface.
This is what OpenRouter got right, and it explains why venture capital invested $113 million in its recent Series B to capture this routing opportunity.
OpenRouter is quickly becoming the market interface: one key to access hundreds of models across multiple providers. The real value isn't in the model list, but in routing the same request to the provider best suited for that task.
This is starting to look like the energy market: users don't care which power plant generated the electricity; they only care if the lights turn on, the price is fair, and the system is stable.
AI users will increasingly think this way—they won't care which GPU cluster served that token, only that the response is fast, cheap, private, and reliable.
Traditional Inference Providers
The traditional side is splitting into four categories:
i) Hyperscalers: AWS, Google, Microsoft
They control the "fortified continents." They win not because they're always the cheapest, but because they already control enterprise procurement, compliance, identity, security, and billing systems. Attacking this front head-on is extremely costly.
They win with enterprise trust. Large companies aren't just buying tokens; they're buying compliance, security, procurement convenience, and someone to hold accountable if things go wrong.
ii) Routing Markets: OpenRouter and Various AI Gateways
Routers sit above model providers, sending each request to the best option. As model leadership changes weekly, hardcoding a single model seems increasingly fragile. AI needs aggregators, just like in crypto.
iii) Optimized Open-Source Model Services: Together, Fireworks, Baseten, Groq
These aren't just cheap APIs; they are performance infrastructure companies focused on speed, batching, scaling, fine-tuning, custom endpoints, and production support.
iv) Model Marketplaces: Replicate and platforms like Hugging Face
Inference is far more than chat. Images, video, speech, embeddings, robotics models, simulations, and multimodal agents all require models to run. Marketplaces make long-tail model needs easily accessible.
Crypto AI Inference Providers
Decentralized networks are the "guerrilla territory."
Crypto inference networks aren't trying to outspend AWS on its main battlefield. They open new fronts: uncensorable models, cheaper GPU supply, private inference, agent-native payments, and workloads that don't need hyperscaler-level reliability.
The crypto side is often lumped together as "decentralized compute," which is too vague. There are at least five different directions:
- Serverless inference networks
- Decentralized GPU marketplaces
- Confidential computing networks
- Private AI applications and gateways
- Orchestration layers
They shouldn't be analyzed as equals.
i) Chutes: Crypto-Native Inference
@chutes_ai is best understood as a decentralized inference platform, not merely a GPU marketplace.
The core idea: Developers don't want to rent GPUs or manage infrastructure; they want a working endpoint. Chutes serves open-source models via a familiar API, using decentralized GPU supply underneath.
The key question is whether top usage can be converted into paid, recurring demand. Cheap tokens are useful, but only if developers trust uptime, latency, and reliability.
Its revenue per trillion tokens continues to rise, showing potential for sustainable profitability/viability.
ii) Akash: GPU Auction Layer
@akashnet is a decentralized cloud marketplace.
Users define the compute they need, providers bid to supply it, and workloads run through leases. It's more of a compute marketplace than a direct inference router.
It's best suited for price-sensitive workloads that can tolerate infrastructure fluctuation and don't need deep AWS/Azure/Google Cloud integration. Fees show some correlation with token price and an upward trend.
iii) io.net: Decentralized GPU Cloud
@ionet is closer to a decentralized GPU cloud provider.
The core selling point is access to distributed GPU supply at lower cost and faster provisioning speed, suitable for AI teams needing compute but not wanting long-term cloud contracts or hyperscaler pricing.
The challenge is execution: hardware verification, reliability, scheduling, support, and consistent performance. Raw GPU access is valuable, but the higher-margin layers are still routing, managed inference, and orchestration.
io.net's performance over the past 30 days stands out, with annualized revenue reaching $12.3 million.
iv) Targon: Confidential Computing
@TargonCompute (built by @manifoldlabs) focuses on confidential computing for AI workloads.
The problem it solves is clear: many users are reluctant to run sensitive prompts, models, or data on infrastructure operated by unknown third parties.
Targon provides protected execution via trusted execution environments, encrypted VMs, remote attestation, and confidential GPU infrastructure. Simply put, it proves workloads run in a secure environment and reduces what the operator can see.
This is particularly relevant for private inference in fields like finance, healthcare, and enterprise AI. Confidential computing isn't magic; it shifts trust to hardware, firmware, and attestation systems.
Last year, the protocol reported $10.4 million in annual revenue and co-authored a research paper with Intel on "decentralized compute on untrusted hardware."
v) Darkbloom: Private Inference on Idle Macs
Darkbloom (built by @eigenlabs) takes a different route.
It doesn't shard large models across random GPUs; it turns idle Apple Silicon Macs into a private inference network. Macs run models locally, and requests are encrypted and routed to verified providers.
The selling point is privacy and cost, not maximizing frontier model performance.
This is useful because "no node holds the full model" doesn't automatically mean prompts are private. Darkbloom more explicitly targets privacy, but still needs to prove supply scale, performance, and developer trust.
The network currently has 300 machines, serving 2 billion tokens and 1 million requests.
vi) Venice: Consumer-Facing Private Inference
@AskVenice occupies a different space than networks like Akash or io.net. It's more like a private AI application and inference gateway than a primary GPU marketplace.
Its gateway throughput has reached 85 billion tokens per day (data from @ErikVoorhees).
Most users want an AI product that respects privacy, provides access to powerful models, and doesn't collect massive amounts of data.
Venice wraps infrastructure ideas into a consumer-facing experience, centered on private prompts, open-source models, uncensored access, API capabilities, and tokenized compute via VVV and DIEM.
The DIEM component is particularly interesting, pointing to the broader idea of an agent economy: providing $1/day access to compute. The market has recently priced this concept quite well.
If agents need continuous inference access, then compute credits start to look like agent-native assets, and entire secondary markets could be built around them.
An agent that can directly hold and spend compute rights is more practical than one relying on humans to swipe credit cards periodically.
This highlights a deeper crypto AI thesis: agents will ultimately need access to capital, identity, memory, and compute, and crypto systems provide a framework for programming these resources.
Venice isn't directly competing with OpenRouter on model breadth, but on privacy, access, and tokenized compute. It's a legitimate niche, but the key question is whether demand for private AI products will be large enough to sustain a token model beyond the current narrative cycle. My judgment is that as AI proliferates, the privacy narrative will only grow stronger.
vii) NuNet: Distributed Compute Orchestration
@nunet_global is often grouped with decentralized compute projects, but a more useful framework is "orchestration."
Orchestration involves matching workloads to the most suitable compute resources and coordinating execution across different machines, environments, and locations.
This becomes increasingly important as AI moves beyond centralized cloud infrastructure.
Future AI systems will likely run across cloud GPUs, edge devices, local servers, robots, phones, sensors, and decentralized provider networks.
A warehouse robot can't wait for a cross-region API response; a drone can't assume perfect connectivity; a field robot needs to perform inference locally when the network is unreliable.
Thus, orchestration is becoming a distinct and meaningful category.
NuNet's challenge is whether it can turn this coordination problem into a functioning economic network with sufficient supply, demand, and developer adoption.
viii) OpenServ: Agent Orchestration, Not Pure Inference
@openservai is best understood as agent infrastructure and an orchestration platform, not a decentralized inference network.
This is important because agents are one of the clearest future sources of inference demand. A regular chatbot might call a model once, while an agent calls it repeatedly: reasoning, using tools, checking output, calling another model, taking action, and then looping.
This creates heavy inference demand, which has already caught crypto's attention.
OpenServ is thus relevant to the inference market from the demand side, not the supply side. If the platform can become a useful place for developers to build, deploy, and coordinate agents, it naturally becomes the layer that routes inference to different providers underneath.
The key question is whether OpenServ can become a true agent execution layer or just another agent marketplace with a token attached.
After multiple conversations with the team, I believe its capabilities go beyond the latter. Its inference framework shows notable benchmark performance, and proprietary models are on the roadmap.
If OpenServ can own the operational workflows for agentification, inference becomes an input to the platform, not its primary product.
In an agentified world, the most valuable layer will be where agents spend significant, ongoing time and resources.
ix) Dolphin AI: Product-Driven Decentralized Inference
@dphnAI is interesting because it started with model demand, not a GPU marketplace.
The Dolphin model family already has a reputation for uncensored open-source models, giving the network a clearer raison d'être.
This is important because many decentralized inference projects are supply-first: "We have GPUs, now who will buy?"
Dolphin does the opposite: start with a collection of models people already want to use, then build a decentralized inference network around that demand.
Its architecture is often called peer-to-pool: GPU owners contribute capacity to a pool for a specific model, rather than each buyer directly renting a specific node. Requests are routed to the pool, and available nodes handle them.
This is a better design for unreliable consumer supply. If someone contributes an idle gaming GPU, they might not stay online forever. A pooled model can absorb this volatility more naturally than a one-to-one rental marketplace.
More interesting is verification. Dolphin is pushing live-weight proofs. Simply put, it checks if the actual model weights loaded during service match what the node claims to be running.
This is important because cheating is one of the hardest problems in decentralized inference. A node might claim to run an expensive model but secretly serve a smaller, cheaper, or quantized version. If the network can't detect this, the whole market loses credibility.
x) c0mpute: Distributed Inference for Agents
@c0mputeAI is worth watching because it attempts to solve one of the hardest problems in decentralized inference: running large models across scattered GPUs on the open internet.
Its Shard engine splits models across multiple machines, rather than requiring one giant server to hold the full model. This is particularly relevant for frontier-scale open-source models that might be too large or restricted to go through conventional hosting routes.
@virtuals_io's link is a key demand-side angle. Virtuals is building an agent economy, and agents are heavy inference users: they plan, call tools, trade, check results, and loop. This creates demand for cheap, open, censorship-resistant inference.
The caveat is that this is still early-stage. c0mpute needs to prove performance under real loads, node reliability, verification, and prompt privacy.
But the direction is important: GPU marketplaces sell compute access; c0mpute is trying to distribute the model itself.
Traditional vs. Crypto Inference
The two will coexist, each with distinct and understandable advantages.
What to Watch For
i) Paid Token Volume
The market should pay less attention to raw token processing stats unless those tokens generate revenue. Free-tier activity and subsidized usage can create impressive numbers but don't prove real product-market fit.
Paid inference demand is the key metric—it's more sustainable and can support long-term viability.
ii) Revenue Per GPU
Decentralized compute networks are only sustainable if GPUs earn more value within the network than outside it. If emissions are the primary reason for provider participation, supply will disappear once incentives drop. GPU providers calculate opportunity costs.
iii) Router Integration: Distribution
Distribution is often more important than the infrastructure itself.
OpenRouter integration, coded agents, wallets, payment endpoints, developer tools, and consumer apps are all potential sources of demand.
Payment endpoints are channels through which software can directly pay for services via API.
iv) Verification
GPU cheating, fake capacity, and unreliable providers remain real risks.
Networks need robust hardware verification, encrypted traffic, reputation systems, and meaningful penalties for bad behavior.
v) Privacy Guarantees
Private inference remains one of crypto AI's strongest opportunities, but guarantees must be real. Marketing privacy is easy; secure execution, local-first architecture, data minimization, and auditable infrastructure are much harder.
vi) Token Value Capture
The strongest token models will tie demand directly to real inference usage. This might involve buybacks, burns, staking requirements, compute rights, or mechanisms linked to revenue.
A broad AI narrative alone likely won't be enough in the long run.
Key Conclusions
i) Endgame is Demand Control
In "Risk," just owning scattered territories isn't enough. You need connected regions, reinforcement routes, and lasting supply lines.
The same applies in the inference market. Winners will control demand, routing, verification, and settlement; just owning GPUs isn't enough.
ii) The inference market is starting to make AI resemble a financial system:
- Every generated token carries a cost,
- Every endpoint carries a margin,
- Every agent loop creates demand,
- Every router acts like a market maker,
- Every GPU network becomes a source of supply...
Traditional providers currently dominate the developer experience and enterprise trust layers.
Crypto AI networks are exploring another frontier: permissionless supply, private inference, verifiable compute, tokenized access, and agent-native (KYC-free) payments.
In the short term, the winners are less likely to be the most decentralized networks, and more likely those that make decentralized inference feel ordinary and reliable—through fast endpoints, strong documentation, reliable uptime, transparent pricing, verified supply, and genuine paid demand.
Chutes remains one of the projects to watch closely, as it's closest to turning Bittensor-backed compute into a functioning inference market, rather than just a GPU narrative. So is Eigen Labs' "Darkbloom."
Akash and io.net represent supply-side challengers, Targon represents the confidential computing thesis, Venice represents the private AI demand layer, and NuNet represents orchestration for a more distributed compute future.
The broader thesis:
"AI models may become increasingly commoditized, but the inference market is unlikely to follow the same path."
The greatest value will accrue to entities that route work, verify work, settle work, and capture demand.
This is precisely where the next crypto AI opportunity may lie... at least until physical AI is competent in society.
