If you pay for AI by the token, you're buying something that's quietly turning into a commodity. That's the view of Erik Bernhardsson, co-founder and CEO of Modal, a cloud company that rents out GPUs and runs AI models for other businesses. He tells Harry Stebbings on 20VC that selling raw tokens is "a race to the bottom", and that the money in inference is in everything wrapped around them. Keep in mind that he sells exactly the bundle he's recommending.

In short: Bernhardsson thinks inference, meaning running trained models for users, is becoming the next big cloud market. Tokens on their own carry no switching costs, so providers bundle them with sandboxes, fine-tuning and batch jobs. Margins are split across a stack of layers, GPUs are scarce but available at a price, and open-source and frontier models can both keep growing. Modal sells inference, so read his case as an interested party's.

1. Why raw tokens race to the bottom

Bernhardsson says Modal sat out open-source model hosting at first because it looked like a commodity business. Then the Chinese lab DeepSeek released its open models, demand spiked, and Modal put about 30 people on large language model inference. He calls Modal a top-three provider, which we couldn't verify.

His key point is about switching costs. If a customer only buys tokens for dollars, they can move to another provider at any moment, and prices fall. If the same customer also uses your sandboxes (isolated environments where AI agents run code), your post-training and reinforcement-learning tools, and your batch processing, leaving gets harder. Sandboxes alone, he says, are about a third of Modal's revenue. So if the bundle protects the price, who keeps the margin?

2. Margins get split up the stack

Stebbings puts typical inference-provider gross margins at 25% to 40%. Bernhardsson thinks Modal can reach 40% to 60%, but not the 80% to 90% of classic software. The reason is structural. A token passes through several businesses on its way to you: the chip maker, the data-centre owner, the GPU cloud (often a "neocloud" such as CoreWeave or Nebius), the inference platform and the app. Each takes a cut.

Modal works with about 25 neoclouds rather than owning data centres, which keeps its capital light but leaves it paying someone else's margin. He expects the market to fragment and specialize rather than settle on one winner.

3. GPUs: scarce, but for sale

He describes Nvidia's GTC conference as "an open-air drug market, but GPUs instead of drugs". Lead times run three to four months, so Modal buys capacity by extrapolating its own growth. You can almost always get GPUs, he says, if you're willing to pay more. Over the long run he expects hardware prices to fall "like gravity", though he admits he has no idea when.

He thinks Nvidia's advantage is less the CUDA software itself than the whole ecosystem around it, and he expects Nvidia to keep winning. He's skeptical that neoclouds can climb into the inference layer, and he says outright, "I'm biased." Which brings up the episode's headline question.

4. Has open-source inference peaked?

The question comes from Stebbings, who cites the investor Jason Calacanis's view that open-source model inference has peaked. Bernhardsson calls that "fixed-pie" thinking. He expects both open and frontier models to grow 5 to 20 times over a 12-month horizon. Some customers, he says, want to own their intelligence by fine-tuning open models they control, but not all do. He'd like to see stronger open models from Europe and the US, and he says Europe struggles to pool capital and talent at the needed scale.

5. A pullback, and the jobs question

He expects a 20% to 30% market pullback at some point and says he'd be fine with it. On engineering jobs he's optimistic in a Jevons-paradox way: cheaper code means more demand for software. Modal, he says, spends about $500,000 a month on coding agents for roughly 150 engineers. He also thinks reinforcement learning hits limits on tasks where feedback takes a long time to arrive.

Weighing it: what holds up

Framing: The episode title asks whether open-source inference has peaked, but that question takes up only a few minutes. Most of the conversation is about Modal's business and inference economics.

Holds up: Modal has announced a $750 million Series D at a $15.75 billion valuation, led by Accel with Kleiner Perkins joining. Accel's announcement describes about 15-fold year-on-year growth, a run rate that passed $1 billion in September 2026, and sandboxes at about $300 million of that, which fits Bernhardsson's "about a third". Stripe agreed in August 2026 to acquire OpenRouter, the model-routing marketplace Bernhardsson says he angel-invested in, which matches his "great outcome".

Partly holds: His figure of 50-fold growth in 12 months is higher than Accel's 15-fold; the two may measure different periods.

Corrections: The captions garble some company names; we used the names as the companies spell them.

Not checked: the top-three claim, his claim that Modal is profitable on paper but cash-flow negative because of GPU spending, the margin ranges, the $500,000 a month on coding agents and the 5-to-20-times forecast. Several big numbers came from Stebbings, not Bernhardsson, including figures for Claude Code revenue, an Anthropic IPO and a 30% fall at Cerebras. Treat them as conversation, not facts.

Disclosures: Bernhardsson has a direct commercial interest: Modal sells the inference, sandboxes and training tools whose value he argues for. He angel-invested in OpenRouter and turbopuffer. He also praises Jev, a model from TypeSafe AI that he says runs on Modal, so he's talking up a customer. We found no sign that Stebbings or 20VC has invested in Modal; Stebbings jokes that he'd "have been a little bit richer" if he had. We worked from YouTube's automatic captions, so quotes may contain small errors.

Bottom line: the bundling logic is sound business reasoning and Modal's growth is real, but the margin and market forecasts come from a seller of the product.

Related reading: Who pays for the AI build-out? · Satya Nadella on owning your AI and open versus closed models

Long Talk Short

• Raw tokens have no switching cost, so their price tends to fall toward cost.
• Inference businesses defend margins by bundling tokens with sandboxes, training tools and batch jobs.
• Every layer of the stack takes a cut, which is why Bernhardsson doesn't expect software-like margins.
• GPU supply is tight but buyable; long-run hardware prices should fall, with no date attached.
• Open and frontier models can both grow; the pie isn't fixed.

The one thing to remember: when a product turns into a commodity, the value moves to whatever makes it hard to leave.

This summary is for general information only and is not investment advice. Do your own research or consult a qualified adviser before making investment decisions.

Source: Harry Stebbings with Erik Bernhardsson, "Has Open-Source LLM Inference Peaked? What Happens to the Cost of Compute," 20VC, on YouTube. This is an independent summary and is not affiliated with 20VC, Harry Stebbings, Erik Bernhardsson or Modal.