AI Gateways Cannot Price Without Margin Attribution

AI gateway and routing companies resell tokens. Their gross margin is the spread between what they pay vendors and what they charge customers. Without per-customer cost tracking, that spread is invisible.

William Min

Technical Product Manager · Lovie

Published June 15, 2026

Creator of TokenOps. Technical PM at Lovie. 12+ years building payment infrastructure and fintech products across Asia and the US.

The Reseller's Dilemma

AI gateway companies — Portkey, OpenRouter, Martian, LiteLLM, Eden AI — share a business model that no other SaaS category has: they resell inference.

A customer routes an LLM request through the gateway. The gateway forwards it to Anthropic, OpenAI, or Google. The customer pays the gateway. The gateway pays the vendor. The difference is gross margin.

This is not a software margin model. It is a reseller margin model. And reseller margins live or die on attribution.

The Blind Spot

An AI gateway knows:

• The total number of requests it processes.
• The total amount it pays vendors.
• The total amount it collects from customers.

What most gateways cannot tell you:

• Which customers are profitable and which are not.
• Which models or vendors drive the best or worst margins.
• Whether a routing decision improved or degraded gross margin.

If a customer routes 80 percent of their traffic through Claude 3.5 Sonnet (expensive) and 20 percent through a cheaper model, the gateway's margin on that customer depends entirely on the pricing spread. Without per-customer, per-model attribution, the gateway is guessing.

The Routing Arbitrage Problem

Gateway companies market "intelligent routing" — automatically switching models to optimize for cost, quality, or latency. This is a compelling feature. But it creates a margin attribution challenge:

When the gateway auto-switches a customer's request from GPT-4o to Claude to Gemini, which cost should be attributed to that customer? The cost of the model they expected, or the cost of the model they received?

The answer matters because:

• If you attribute the expected cost, your margin looks stable but hides the savings (or overage) from routing.
• If you attribute the actual cost, your margin per customer fluctuates and you need to explain the variance.

The Companies in This Segment

Several gateway companies are at different stages of this problem:

• Portkey processed over 500 billion tokens per day and just closed its first institutional round — a $15 million Series A in February 2026. At this stage, margin visibility is the difference between a fundable business and a cost center.
• OpenRouter routes to 400+ models and has approximately $50 million in annualized revenue. They raised $113 million in a Series B led by CapitalG at a $1.3 billion valuation. At that scale, margin by model and by customer is a board-level question.
• Martian focuses on auto-switching models for cost and quality optimization, backed by Accenture Ventures. Their entire value proposition is margin improvement — which they must measure to prove.
• LiteLLM (BerriAI) is an open-source proxy connecting to 100+ LLM APIs, just beginning to monetize.
• Eden AI operates as a pure multi-vendor reseller in Europe, raising a €3 million seed round.

The Margin Formula

For any AI gateway, the per-customer gross margin formula is:

Revenue from customer
  minus (vendor cost for models used by that customer)
  minus (gateway infrastructure cost allocated to that customer)
  equals gross margin for that customer

The first term is easy — you know what you charged. The second term is where the attribution gap lives. Without tagging every routed request with a customer ID, model, and computed cost, the second term is an estimate.

What Gateways Need to Track

Every routed request should carry:

1. Customer ID — who sent this request.
2. Original model requested — what the customer asked for.
3. Actual model used — what the routing engine selected.
4. Vendor cost — what the gateway paid for this request.
5. Customer charge — what the gateway billed for this request.
6. Margin — the spread, computed per request.

With this data, a gateway can answer:

• Which customers generate the best and worst margins.
• Whether the routing engine is actually improving margins or just shifting cost.
• Which vendor relationships are most profitable to maintain.

The Board Conversation

Gateway companies that have raised institutional capital will face this question from investors: "What is your blended gross margin, and what is the margin distribution across your top 20 customers?"

Companies that can answer with data will have a fundraising advantage. Companies that cannot will be asked to build the attribution pipeline before the next round.

TokenOps provides this attribution layer. But for gateway companies, the underlying need — per-request cost and margin tracking — is foundational to the business model itself.