Procuring Intelligence: Finding Value Beyond the Price per Token (Nov 2026 Update)

Spencer Penn

CEO & Co-founder

Every buyer I have worked with learns the same lesson early: the price on the quote is not the price of the part. A casting quoted at $4.10 with a 12% scrap rate costs more than one quoted at $4.40 that comes off the line clean, once you count the rework, the expediting, and the line that sat idle waiting for good parts. Manufacturing has a name for the number that matters -- total landed cost per conforming part -- and it took the industry a generation to stop buying on unit price.

AI is now one of the fastest-growing inputs companies buy, and almost everyone still buys it on unit price: dollars per million tokens. Ramp's card and invoice data shows 56.8% of US businesses paying for AI as of October, and in one recent four-week window the blended price per token fell 14.8% while total spend rose 10.5%, because volume grew 29%. The meter is getting cheaper and the bill is getting bigger, which is roughly what happens when a buyer negotiates the unit price of an input whose consumption nobody has specified.

In July I wrote about renting the model and owning the evals -- the case for treating models as a tiered, multi-sourced commodity behind your own acceptance tests. This is the November update, and it goes a level deeper into the money: what you actually pay per finished piece of work, where to route each kind of task, and whether "price per token" was ever the right unit. It comes with a downloadable total cost of ownership workbook so you can run the numbers on your own work.

Labs vs. Open Weights

There are two purchases hiding inside "which model should we use": whose model, and who runs it. You can buy a frontier model directly from the lab that trained it -- Anthropic, OpenAI, Google, xAI -- or through a cloud that resells it. Or you can buy an open-weight model like Kimi, GLM, DeepSeek, Qwen or gpt-oss from any host that chooses to serve it: an aggregator like OpenRouter, a cloud marketplace like AWS Bedrock, a specialist like Together, Fireworks, Groq or Cerebras, or your own GPUs. (We looked at who owns each layer of the AI stack in October; this is the view from the buyer's end of it.)

The frontier end of the market moved twice in eight days. Anthropic released Claude Fable 5.2 on November 9 at $8 input / $40 output per million tokens, down from $10/$50. OpenAI released GPT-6.2 Astra this morning at the same $8/$40, 20% below the GPT-6 Astra it replaces. Google's Gemini 4 Carbon lists at $2/$10, but that is an introductory price that doubles to $4/$20 on December 1. One tier down, Sonnet 5.5 and GPT-6.2 Sol both sit at $2/$10, and Haiku 5.5 and GPT-6 Luna at $0.10/$0.50.

The open-weight side is cheaper and more fragmented. Kimi K3.1 Pro, the strongest open model on Artificial Analysis's Intelligence Index, lists at $2/$10 from Moonshot and $1.90/$9.50 on DeepInfra. DeepSeek V4.1 Flash runs $0.30/$1.20 on DeepSeek's own API at peak hours and half that off-peak. Menlo Ventures' November report finds open models handling about one in five enterprise tokens but only about one in 25 dollars of model spending, which says both how cheap they are and how much of the serious work still runs on closed models.

Four things about the channel matter more than most buyers expect:

  • The same model is not always the same price. Bedrock's global cross-region pricing for Claude Opus 5.5 matches Anthropic's direct $4/$20 exactly, while in-region inference costs $4.40/$22. Bedrock does not list a batch discount for the current Claude models, so overnight batch work costs about half as much bought direct. For open models the reseller premium can be large: Bedrock prices GLM-5.3 about 20% above Z.ai and offers Kimi K3.1 Pro only as a "Highspeed" tier at twice Moonshot's rate.

  • Aggregators charge at the edges. OpenRouter passes through each host's list price with no inference markup, then charges 5.5% on credit purchases (8% on its Business plan) and 5% on bring-your-own-key usage above $25,000 a month. That is a fair price for one bill, one API and instant switching, and it belongs in the TCO.

  • Host dispersion is wide, and it is not only about price. On OpenRouter's November 17 snapshot, gpt-oss-120b ran at $0.03/$0.15 and 53 output tokens per second on one host and $0.35/$0.75 at 742 tokens per second on Cerebras: five times the output price for fourteen times the speed. For an overnight extraction job the speed buys nothing. For a buyer waiting on a screen, it can be worth more than the tokens. Neither speed is a contractual service level.

  • Data residency eliminates bids before price does. If supplier drawings must stay in one region, a global endpoint is ineligible at any price. Put the region and the permitted fallback hosts on the bid sheet, or a platform team can do exactly what you asked during an outage and route a drawing somewhere your supplier agreement prohibits.

Self-hosting is the make-or-buy decision at the bottom of this list. Lambda lists an eight-GPU H100 node at $4.59 per GPU-hour, about $26,400 for a 720-hour month before operations and failover, and the math only works at high, steady utilization: the same node at 20% utilization costs four times as much per task as it does at 80%. At AI Engineer NYC in October, Revolut's David Abitbol described serving GLM-5.3 to about 2,000 engineers on 56 B200 GPUs at a blended cost near $0.30 per million tokens, around a tenth of the company's previous Anthropic and OpenAI invoice. That is a real result for a high-volume buyer with its own serving team, not a wholesale price available to everyone.


Scatter chart of 25 AI models plotting Artificial Analysis Intelligence Index against cost to run the index, highlighting Claude Fable 5.2 and GPT-6.2 Astra
Fable 5.2 and GPT-6.2 Astra share an $8/$40 list price and score within three points of each other, yet Fable cost $22,191 to run Artificial Analysis's full index to Astra's $4,530 -- Source: Artificial Analysis, Intelligence Index v5.0.1 (Nov 17, 2026)

The chart above is where the price list stops being useful. Fable 5.2 and GPT-6.2 Astra have identical list prices and nearly identical scores on the Intelligence Index (64.6 and 62.1). Running the full index cost $22,191 on Fable at maximum effort and $4,530 on Astra, because Fable generated 275 million output tokens to Astra's 66 million. Sonnet 5.5 at maximum effort, at a quarter of Fable's price, cost $7,981 -- more than Opus 5.5 -- for the same reason. A price per token tells you what one unit costs and nothing about how many units a model will decide to spend on your problem.

What a Token Actually Costs

A token invoice has more lines than a price sheet, and procurement teams should model all of them before comparing bids:

  • Reasoning tokens. Modern models think before they answer, and you pay for the thinking at the output rate. On Artificial Analysis's runs, Fable 5.2's total output ranged from 34.9 million tokens at low effort to 275 million at max, at the same token price -- $3.10 per task against $26.54. Reasoning effort is now about as large a lever as switching model tier, and you can change it without changing suppliers.

  • Tokenizers. Different models count the same text differently. When Anthropic released Opus 4.7 in April, it noted the new tokenizer could produce 1.0 to 1.35 times as many tokens for identical input. An unchanged price can be a 35% increase on your documents.

  • Caching and batch. Cache reads run 90% or more below the input price, and batch APIs typically cut 50%. For agents that re-read the same context on every step, a good cache hit rate is a bigger lever than vendor choice.

  • Failures and retries. A 2026 study of coding agents found that repeated runs of the same task could differ by up to 30 times in token use. Budget for the distribution, not the average.

  • Review and rework. Somebody reads the output. On most business tasks this line is larger than the token bill, and it is the line almost nobody puts in the model comparison.

  • Latency, governance and capacity. Regional data processing carries a 10% premium at both OpenAI and Anthropic, rate limits can bind before budget does, and a person blocked waiting on a slow model is a cost even when the API call is cheap.

The review line deserves the emphasis. OpenAI's GDPval study, which graded model output against experts in 44 occupations (buyers and purchasing agents among them), found GPT-5 roughly 474 times cheaper than a professional on raw inference cost. In the same paper's more realistic workflow -- try the model once, pay an expert to review it, and have the expert redo the work when the model falls short -- the advantage shrank to 1.18 times. Nothing about the model changed between those two numbers; the second one simply includes the expert's time. (It is also why I am skeptical of the louder forecasts about which procurement jobs AI replaces: review time is where the substitution math gets decided.)

To make this concrete, we built a workbook that models ten common tasks across fifteen current models, with review and redo included. Download it free: AI Model TCO Calculator (Excel) or the price and benchmark table (CSV). Prices are checked as of November 17, every formula is live, and the yellow cells are the assumptions you should replace with your own numbers -- above all the success rates, which are illustrative.


Bar chart comparing monthly full cost and API spend for four scenarios: routed by task, all frontier model, all budget model, and no AI
The workbook's example organization: the budget model wins the API bill by a factor of fifty and loses the full cost by a factor of two and a half -- Source: LightSource AI Model TCO Calculator (Nov 2026); illustrative volumes and success rates

For a hypothetical mid-size procurement and engineering organization running about 280,000 AI tasks a month, the example lands like this. Running everything on the cheapest budget model costs $403 a month in API fees and about $309,000 once you count the human review and the redo of everything it gets wrong. Running everything on Fable 5.2 costs $20,500 in API fees and about $128,000 all in. Routing each task to the model with the lowest full cost spends about $9,500 on APIs and lands near $122,000, including $2,000 a month of evaluation overhead. Doing the same work with no AI would cost about $667,000 in labor.

Two things in that table are easy to miss. The budget model could give its inference away free and save this organization $403 a month; the opportunity is in the $309,000 column, where people are checking and finishing the work. And routing cuts about $11,000 from the all-frontier API bill but only about $7,000 from the full cost, because some of the saving comes back as review and redo. That $7,000 is a useful ceiling: it is roughly what this organization could spend on routing software and upkeep before routing stops paying for itself against simply buying the best model.

Route Work by Task

The obvious conclusion from those numbers is to buy the best model for everything, and it is the wrong conclusion. The full-cost curves look very different by task, and the shape is the useful part.


Routing chart showing full cost per completed task for ten business tasks on Claude Haiku 5.5, Sonnet 5.5 and Fable 5.2, with a recommended tier for each
Ten common tasks priced on three tiers of one model family: routine work converges, judgment work splits apart -- Source: LightSource AI Model TCO Calculator (Nov 2026); list prices Nov 17, 2026; success rates illustrative

For bounded work -- classifying a line item, drafting a supplier follow-up, pulling fields off an invoice -- the models converge. Drafting a routine supplier email costs about $0.0005 in API fees on Haiku 5.5 and $0.036 on Fable 5.2, a seventy-fold spread, and on full cost they land within about 20 cents of each other, because the minute a buyer spends reading the draft dwarfs either token bill. When tiers are that close, buy the cheaper one: your estimate of each model's success rate is not precise enough to justify paying more, and the small model is usually faster. For an email, Haiku or Sonnet is plenty.

For judgment and long-horizon work the curves split apart. A should-cost analysis across a stack of quotes costs about $0.02 per attempt on Haiku and $1.42 on Fable, and the full cost runs the other way: about $56 on Haiku against $22 on Fable, because the smaller model's analysis is wrong often enough that a buyer ends up doing the work anyway. A multi-source research report shows the same pattern more sharply, roughly $300 on Haiku against $86 on Fable once redo is counted. Once the denominator is finished work, the expensive model is the cheap option.

A useful way to decide the middle cases is to price the premium in reviewer time. In the workbook, Fable costs $0.91 for a 40-page contract summary against Sonnet's $0.24, a premium of $0.67. At the BLS paralegal median wage of $30.24 an hour, that premium pays for itself if Fable saves about 80 seconds of review, or if it raises first-pass acceptance by about 2.2 percentage points when a rejected summary takes an hour to redo. Those are break-even thresholds, not claims about either model, and they turn a debate about benchmark points into something a paralegal can test with a stopwatch.

Across all ten tasks in the chart, the cheapest tier within 20% of the best full cost works out like this: classifying a line item, drafting a supplier email and extracting an invoice to JSON go to the small tier; support chats, answers from internal documents, 40-page contract summaries and roughly 30-step browser tasks go to the mid tier; should-cost analysis, coding-agent bug fixes and multi-source research reports go to the frontier tier. API cost per attempt runs from about $0.00003 (classifying on Haiku) to $4.23 (a browser task on Fable).

The routing rule that falls out of this is closer to sourcing than to engineering: route on who can check the answer and what a wrong answer costs, not on how hard the prompt sounds.

  • Small tier (Haiku 5.5, GPT-6 Luna, DeepSeek V4.1 Flash, gpt-oss): classification, extraction to a schema, tagging, short drafts, translation -- anything with a built-in check or a low cost of error. Batch it when you can.

  • Mid tier (Sonnet 5.5, GPT-6.2 Sol, GPT-6.1 Terra, Kimi K3.1 Pro): answers from internal documents, contract summaries, support conversations, most coding with tests, and browser tasks where a person can cheaply finish what the agent misses. GPT-6.2 Sol is the standout of the month; OpenAI reports it within half a point of Astra on its FrontierCode benchmark at about a fifth of the cost.

  • Frontier tier (Fable 5.2, GPT-6.2 Astra, Opus 5.5, Gemini 4 Carbon) at high effort: should-cost and negotiation analysis, contract risk review, deep research, and coding agents working across an unfamiliar codebase.

  • The supplier, not a model: when a quote leaves out whether the tooling charge is recurring, a second model will only give you a more polished description of the same gap. Route missing commercial terms to a clarification email and keep the quote open until the supplier answers.

Three cautions keep routing honest. First, the baseline to beat is the best fixed model for each workflow, not "always use the most expensive model." An October audit of commercial routers found one sending 100% of traffic to a single model, and OpenRouter's own November benchmark showed a fixed GLM-5.3 Flash beating both of its routers on a coding suite at about $0.20 per attempt. Second, savings in the lab shrink in production. GitHub's HyDRA router for Copilot cut cost 54% on SWE-bench while staying within 0.3 points of always using the strong model; in a live test with close to a million users per arm, the realized saving was 7 to 20%, because cheaper requests invited more requests. Third, a working router changes the data you judge models on. Once the clean quotes all go to Haiku, Sonnet sees mostly exceptions and its success rate appears to fall. Keep a small fixed sample running through every approved model, or you will demote a model next quarter for a change in the work you gave it.

Not All Tokens Are Equal

Underneath all of this is a question about the unit itself. If one model's token is "smarter" than another's, then comparing their prices per token is like comparing the hourly rate of a senior engineer and an intern without asking what each gets done in the hour.

Imagine a million poodles collaborating on a novel. In aggregate there is a great deal of effort in the room and, arguably, a great deal of raw brainpower. You will still get a worse book than one competent author would write, and a second million poodles will not help. Even if one poodle somewhere produces a readable chapter, somebody has to find it among the million, and the editor paid by the hour would have preferred to hire the author. Some work is not the sum of many small units of intelligence. It needs a single unit of enough intelligence to hold the whole problem at once.


The research on this has gotten much sharper this year. In July I wrote about the "Large Language Monkeys" result, in which a coding model went from solving 15.9% of SWE-bench Lite issues in one attempt to 56% with 250 attempts -- evidence that you can manufacture intelligence from cheap tokens when a test can verify the answer. The limit of that idea is now well documented. Princeton's Benedikt Stroebl, Sayash Kapoor and Arvind Narayanan, in a paper presented at ICLR this year, showed that once the checker is imperfect, "no amount of inference scaling of weaker models can enable them to match the single-sample accuracy of a sufficiently strong model." Their analysis put the best number of attempts at often fewer than ten. For procurement the imperfect checker is familiar: "the line items add up" is a cheap test, and it will happily approve a table in which every price is in the wrong currency.

METR's time-horizon research shows why the gap is nonlinear. Its original study found models succeeding almost 100% of the time on tasks that take a human under four minutes and less than 10% of the time on tasks taking more than four hours. By this spring METR estimated that frontier models could complete about twelve-hour tasks half the time, but only about ninety-minute tasks with 80% reliability. Capability falls off a cliff as tasks get longer, and a slightly weaker model moves the cliff much closer. A token from a model that can hold a twelve-hour task together is a different product from a token that cannot, the same way a journeyman machinist's hour and an apprentice's hour are different products even though both appear on a timesheet.

The economic version of this argument has gone mainstream. OpenAI's CFO Sarah Friar proposed "Useful Intelligence per Dollar" as a scorecard in July, writing that "the lowest price per token does not always produce the lowest cost per outcome." Shelly Palmer put the procurement problem plainly in June: tokens are "easy to meter and easy to price, but hard to value." And the price of a fixed level of intelligence keeps collapsing. Epoch AI's September report estimates that the cost of reaching a given level of performance, reasoning tokens included, has fallen about 47% per quarter -- 13 times a year -- and that o3's January 2025 score on a PhD-level science benchmark, which cost about $0.30 per question then, now costs $0.0004.

That last point cuts both ways, and it is the strongest argument against my own framing. If the cost of yesterday's frontier falls thirteen-fold a year, then for any task yesterday's frontier could do, today's budget model is the right buy, and price per token among the models that clear your bar is exactly the right comparison. The MIT FutureTech group put numbers on both halves: a fixed level of capability gets 5 to 10 times cheaper each year, while running the current frontier gets 3 to 18 times more expensive each year as models think longer. Both are true at once. Intelligence per dollar rises steadily for work you already know how to hand off, and the newest, hardest work keeps getting more expensive because the frontier keeps moving onto it. Epoch's own caveat applies too: its curve assumes a buyer who relentlessly switches to the most economical model, and an enterprise with a security review and an annual commitment lives on a slower one.

I would still keep price per token on the finance dashboard. It is the right unit for auditing whether a provider billed cache reads at the contracted rate, for choosing between hosts serving the same pinned model once quality and latency are shown to match, and for forecasting bounded, repetitive work where the token count per task is stable. It stops working the moment the comparison crosses from one model to another, or from one call to a job with an unknown number of calls and a person at the end.

What the Benchmarks Say

If intelligence has a price, you need a way to measure the quantity, and there is a large body of prior work doing exactly that. The workbook's Benchmarks tab covers sixteen current models across twelve public benchmarks. Three families matter most for a buyer.

The first is composite indexes. Artificial Analysis blends ten evaluations into one score and, usefully, publishes what it cost to run each model through them. Epoch AI's Capabilities Index stitches dozens of benchmarks into a single scale. LMArena ranks models by blind human preference. These are good for building a shortlist and poor for the final call, because each compresses very different skills into one number -- and whoever builds the index decides what a point is worth.

The second is benchmarks that report cost next to score. ARC Prize publishes cost per task for every model on its reasoning puzzles. Princeton's Holistic Agent Leaderboard was built on the argument in Kapoor and colleagues' 2024 paper that "agent evaluations must be cost-controlled," and its headline finding still stands: "Agents can be 100x more expensive while only being 1% better." Vals.ai runs every model on the same harness against business tasks -- research on financial filings, building models in Excel, corporate tax questions -- and reports cost per test. For a procurement audience, Vals is the closest public proxy for real work.


Dumbbell chart comparing Claude Fable 5.2, Claude Haiku 5.5 and DeepSeek V4.1 Flash across eight benchmarks, with frontier lead in points and cost per test
The frontier lead is 28-30 points on long, open-ended tests and single digits on bounded business tasks; on corporate tax research the open model wins -- Sources: Artificial Analysis, Vals.ai, Epoch AI (fetched Nov 17, 2026)

Putting those families side by side shows that the size of the intelligence gap depends on the job. On Artificial Analysis's composite index, DeepSeek V4.1 Flash (32.6) scores half of Fable 5.2 (64.6). On the Vals Index of business tasks it scores 61.3% against Fable's 71.3%, at $0.66 per test against $17.94. On Vals' finance research benchmark, Haiku 5.5 scores 65.4% at $1.64 a test against Fable's 69.5% at $4.83. On Vals' corporate tax benchmark the order reverses: DeepSeek scores 44.4% at $0.46 a test, Fable 32.1% at $3.31. On Vals' Excel modeling benchmark (LBO and DCF builds), Fable leads 77.4% to DeepSeek's 69.2% and Haiku's 65.5%, and on the PhD-level GPQA Diamond science questions all three sit between 90% and 99%. Then on Terminal-Bench, a long agentic coding test, the gap opens back up to 32% and 36% for the small models against 65% for Fable, and on Humanity's Last Exam it is 74% for Fable against 44% and 43%. Bounded, well-specified business work compresses the differences between models, and long, open-ended work stretches them -- which is the poodle problem showing up in the data.

The usual caveats apply and are worth repeating. Benchmarks are not your work. SWE-bench Verified was retired this year after contamination concerns, and Scale's SWE-Bench Pro found one frontier model scoring 17.8 points lower on private code than on public code. "The Leaderboard Illusion" documented how private testing and selective disclosure can skew LMArena. Use benchmarks to cut fifty models to five, and then test the five on your own examples.

Price the Accepted Output

The adjusted metric I would put in front of a CFO is cost per accepted output: everything you spend to get one unit of finished, checked work, divided by the units you actually accept. For a sourcing team the unit might be one supplier quote normalized to the correct revision, currency, unit of measure and price breaks, accepted by someone other than the system that produced it. A model that enters "$48.00 per 100 pieces" as $48 a piece has returned perfectly formatted data in under a second at exactly the token rate you negotiated, and it has not delivered the unit.

There are a few published forms of this metric, and they answer slightly different questions.

Cost-of-pass. Mehmet Hamza Erol and colleagues define it as the cost of one attempt divided by the probability that the attempt is correct -- the expected cost of a correct answer if you retry until you get one. A $0.10 attempt with a 50% pass rate costs an expected $0.20 per pass. Their findings line up with everything above: small models win on basic quantitative work, large models on knowledge-intensive work, reasoning models on complex problems, and tricks like majority voting usually raise the cost-of-pass rather than lowering it. The weakness is that it goes to infinity when a model can never succeed, which is the poodle problem stated as math.

Cost per completed task. GDPval's version assumes the work has to get done: try the model, pay for review, and if it fails, pay a person to finish. With one attempt, that is API cost plus review cost plus the failure rate times the human cost. This is the formula in the workbook, and for business work it is the one I trust, because a wrong quote analysis cannot quietly drop out of the denominator.

Risk-adjusted cost. Add the rate at which a wrong answer slips past review, times what it costs when it does. For a supplier email that term is near zero. For a pricing commitment to a contract manufacturer it can dominate everything else, and for an unauthorized commitment or a prohibited disclosure I would use a hard disqualification rule rather than a dollar allowance.

In practice the scorecard is short: cost per attempted task for budgeting, cost per accepted output for award decisions, and coverage -- the share of work finished without a person stepping in -- reported right next to both. A model that looks cheap per success because it silently skips the hard cases is a supplier with a great unit price and a terrible fill rate. Put the deadline in the definition too: if the team needs 40 quotes for a Thursday award meeting, 38 accepted on Friday is a different service.

What to Put in the Contract

Buying AI well looks like ordinary request for proposal discipline pointed at a strange new commodity. My colleague Aparna Keswani has mapped where AI delivers across the sourcing lifecycle; this is the commercial side of buying the AI itself. Six things I would put in the next RFP:

  1. Specify the deliverable, not the model. Define what an accepted output is for each workflow -- the right supplier, part, currency, unit of measure and quantity breaks, traced to the source document -- and what counts as a material error.

  2. Run the same blind workload on every bidder. Include messy scans, revisions and deliberately ambiguous cases, hold out whole supplier document families, and include a cheap fixed model, a strong fixed model, a routed policy and your current human process as baselines.

  3. Ask for two bid sheets. One is the rate card by model, host, region, token class, effort level and speed tier. The other is cost per accepted output on your replay, with token logs, failure rates and reviewer minutes.

  4. Never lock the unit price. With the cost of constant capability falling thirteen-fold a year, insist on price reopeners, substitution rights when a better model ships, and pass-through of cache and batch discounts. The Gemini 4 Carbon price doubling on December 1 is a reminder that promotions end.

  5. Price the cost of switching. If a new model saves $0.02 per completed job and qualifying it costs $6,000, it takes 300,000 comparable jobs to earn the switch back. A clause that lets you move a commitment across approved models can be worth more than another point off a rate you will not want for long.

  6. Monitor in production. Put total spend, cost per accepted output, coverage and critical errors on one dashboard, and reopen the decision when any of them moves, not when a new model tops a leaderboard. (The same order of operations I argued for in fluency first applies: get people using the cheap tier well before you automate the expensive one, and remember that deployment is where the value gets built.)

This is the discipline our customers already apply to castings, resins and harnesses. LightSource is built for that work in direct materials: normalizing supplier bids line by line, tying every price back to the spec, and showing total cost rather than the number on the quote. The same habits travel well to the newest line item on the P&L.

The models will keep getting better and cheaper, and the top of the leaderboard will keep changing hands. The buyers who come out ahead will be the ones who can say, for each kind of work they hand to a model, what a finished unit costs and what it is worth.

Sources

Frequently Asked Questions

What is the cheapest AI model per task in 2026?

It depends on the task. For classification, extraction and short drafting, small models like Claude Haiku 5.5, GPT-6 Luna or DeepSeek V4.1 Flash cost fractions of a cent per attempt and land close to frontier models on full cost. For long, judgment-heavy work like research reports or should-cost analysis, a frontier model is usually cheaper per finished task, because smaller models fail often enough that a person ends up redoing the work.

Is it cheaper to run open-weight models on OpenRouter or Bedrock than to buy closed models?

Usually on a per-token basis, often by five to twenty times. But the same open model can vary several-fold in price and speed across hosts, cloud marketplaces sometimes charge a premium over the developer's own API, and the best open model still trails the frontier by a wide margin on the hardest tasks. Compare on cost per accepted output for your workload, not on list price.

What is cost-of-pass for AI models?

Cost-of-pass is the expected cost of getting one correct answer from a model: the cost of a single attempt divided by the probability that the attempt is correct. Erol and colleagues formalized it in 2025, and it is useful for comparing models of very different price and capability. It becomes infinite when a model cannot solve a task at all, which is why business teams usually pair it with a version that includes human fallback.

How do reasoning tokens affect AI costs?

Reasoning models generate hidden "thinking" tokens before they answer, and those are billed at the output rate. The effect is large: on Artificial Analysis's benchmark runs, the same model at the same price cost about 8.6 times more per task at maximum reasoning effort than at low effort. Setting effort by task is one of the cheapest optimizations available.

How many examples should an AI acceptance test include?

Start from the error rate you need to detect. If 1% of jobs contain a critical error, a 100-job test still has about a 37% chance of showing no critical errors at all (0.99 to the 100th power). Supplier templates make failures cluster, so supplement a representative sample with targeted tests of the rare, expensive mistakes.

Should procurement own AI model purchasing?

Procurement should at least own the commercial structure: rate cards, price reopeners, substitution rights, data terms and spend monitoring. The technical team should own the evaluations that define an accepted output. The best results come when the two work together the way they do on any engineered component, with a spec, a qualification test and more than one approved supplier.

Every buyer I have worked with learns the same lesson early: the price on the quote is not the price of the part. A casting quoted at $4.10 with a 12% scrap rate costs more than one quoted at $4.40 that comes off the line clean, once you count the rework, the expediting, and the line that sat idle waiting for good parts. Manufacturing has a name for the number that matters -- total landed cost per conforming part -- and it took the industry a generation to stop buying on unit price.

AI is now one of the fastest-growing inputs companies buy, and almost everyone still buys it on unit price: dollars per million tokens. Ramp's card and invoice data shows 56.8% of US businesses paying for AI as of October, and in one recent four-week window the blended price per token fell 14.8% while total spend rose 10.5%, because volume grew 29%. The meter is getting cheaper and the bill is getting bigger, which is roughly what happens when a buyer negotiates the unit price of an input whose consumption nobody has specified.

In July I wrote about renting the model and owning the evals -- the case for treating models as a tiered, multi-sourced commodity behind your own acceptance tests. This is the November update, and it goes a level deeper into the money: what you actually pay per finished piece of work, where to route each kind of task, and whether "price per token" was ever the right unit. It comes with a downloadable total cost of ownership workbook so you can run the numbers on your own work.

Labs vs. Open Weights

There are two purchases hiding inside "which model should we use": whose model, and who runs it. You can buy a frontier model directly from the lab that trained it -- Anthropic, OpenAI, Google, xAI -- or through a cloud that resells it. Or you can buy an open-weight model like Kimi, GLM, DeepSeek, Qwen or gpt-oss from any host that chooses to serve it: an aggregator like OpenRouter, a cloud marketplace like AWS Bedrock, a specialist like Together, Fireworks, Groq or Cerebras, or your own GPUs. (We looked at who owns each layer of the AI stack in October; this is the view from the buyer's end of it.)

The frontier end of the market moved twice in eight days. Anthropic released Claude Fable 5.2 on November 9 at $8 input / $40 output per million tokens, down from $10/$50. OpenAI released GPT-6.2 Astra this morning at the same $8/$40, 20% below the GPT-6 Astra it replaces. Google's Gemini 4 Carbon lists at $2/$10, but that is an introductory price that doubles to $4/$20 on December 1. One tier down, Sonnet 5.5 and GPT-6.2 Sol both sit at $2/$10, and Haiku 5.5 and GPT-6 Luna at $0.10/$0.50.

The open-weight side is cheaper and more fragmented. Kimi K3.1 Pro, the strongest open model on Artificial Analysis's Intelligence Index, lists at $2/$10 from Moonshot and $1.90/$9.50 on DeepInfra. DeepSeek V4.1 Flash runs $0.30/$1.20 on DeepSeek's own API at peak hours and half that off-peak. Menlo Ventures' November report finds open models handling about one in five enterprise tokens but only about one in 25 dollars of model spending, which says both how cheap they are and how much of the serious work still runs on closed models.

Four things about the channel matter more than most buyers expect:

  • The same model is not always the same price. Bedrock's global cross-region pricing for Claude Opus 5.5 matches Anthropic's direct $4/$20 exactly, while in-region inference costs $4.40/$22. Bedrock does not list a batch discount for the current Claude models, so overnight batch work costs about half as much bought direct. For open models the reseller premium can be large: Bedrock prices GLM-5.3 about 20% above Z.ai and offers Kimi K3.1 Pro only as a "Highspeed" tier at twice Moonshot's rate.

  • Aggregators charge at the edges. OpenRouter passes through each host's list price with no inference markup, then charges 5.5% on credit purchases (8% on its Business plan) and 5% on bring-your-own-key usage above $25,000 a month. That is a fair price for one bill, one API and instant switching, and it belongs in the TCO.

  • Host dispersion is wide, and it is not only about price. On OpenRouter's November 17 snapshot, gpt-oss-120b ran at $0.03/$0.15 and 53 output tokens per second on one host and $0.35/$0.75 at 742 tokens per second on Cerebras: five times the output price for fourteen times the speed. For an overnight extraction job the speed buys nothing. For a buyer waiting on a screen, it can be worth more than the tokens. Neither speed is a contractual service level.

  • Data residency eliminates bids before price does. If supplier drawings must stay in one region, a global endpoint is ineligible at any price. Put the region and the permitted fallback hosts on the bid sheet, or a platform team can do exactly what you asked during an outage and route a drawing somewhere your supplier agreement prohibits.

Self-hosting is the make-or-buy decision at the bottom of this list. Lambda lists an eight-GPU H100 node at $4.59 per GPU-hour, about $26,400 for a 720-hour month before operations and failover, and the math only works at high, steady utilization: the same node at 20% utilization costs four times as much per task as it does at 80%. At AI Engineer NYC in October, Revolut's David Abitbol described serving GLM-5.3 to about 2,000 engineers on 56 B200 GPUs at a blended cost near $0.30 per million tokens, around a tenth of the company's previous Anthropic and OpenAI invoice. That is a real result for a high-volume buyer with its own serving team, not a wholesale price available to everyone.


Scatter chart of 25 AI models plotting Artificial Analysis Intelligence Index against cost to run the index, highlighting Claude Fable 5.2 and GPT-6.2 Astra
Fable 5.2 and GPT-6.2 Astra share an $8/$40 list price and score within three points of each other, yet Fable cost $22,191 to run Artificial Analysis's full index to Astra's $4,530 -- Source: Artificial Analysis, Intelligence Index v5.0.1 (Nov 17, 2026)

The chart above is where the price list stops being useful. Fable 5.2 and GPT-6.2 Astra have identical list prices and nearly identical scores on the Intelligence Index (64.6 and 62.1). Running the full index cost $22,191 on Fable at maximum effort and $4,530 on Astra, because Fable generated 275 million output tokens to Astra's 66 million. Sonnet 5.5 at maximum effort, at a quarter of Fable's price, cost $7,981 -- more than Opus 5.5 -- for the same reason. A price per token tells you what one unit costs and nothing about how many units a model will decide to spend on your problem.

What a Token Actually Costs

A token invoice has more lines than a price sheet, and procurement teams should model all of them before comparing bids:

  • Reasoning tokens. Modern models think before they answer, and you pay for the thinking at the output rate. On Artificial Analysis's runs, Fable 5.2's total output ranged from 34.9 million tokens at low effort to 275 million at max, at the same token price -- $3.10 per task against $26.54. Reasoning effort is now about as large a lever as switching model tier, and you can change it without changing suppliers.

  • Tokenizers. Different models count the same text differently. When Anthropic released Opus 4.7 in April, it noted the new tokenizer could produce 1.0 to 1.35 times as many tokens for identical input. An unchanged price can be a 35% increase on your documents.

  • Caching and batch. Cache reads run 90% or more below the input price, and batch APIs typically cut 50%. For agents that re-read the same context on every step, a good cache hit rate is a bigger lever than vendor choice.

  • Failures and retries. A 2026 study of coding agents found that repeated runs of the same task could differ by up to 30 times in token use. Budget for the distribution, not the average.

  • Review and rework. Somebody reads the output. On most business tasks this line is larger than the token bill, and it is the line almost nobody puts in the model comparison.

  • Latency, governance and capacity. Regional data processing carries a 10% premium at both OpenAI and Anthropic, rate limits can bind before budget does, and a person blocked waiting on a slow model is a cost even when the API call is cheap.

The review line deserves the emphasis. OpenAI's GDPval study, which graded model output against experts in 44 occupations (buyers and purchasing agents among them), found GPT-5 roughly 474 times cheaper than a professional on raw inference cost. In the same paper's more realistic workflow -- try the model once, pay an expert to review it, and have the expert redo the work when the model falls short -- the advantage shrank to 1.18 times. Nothing about the model changed between those two numbers; the second one simply includes the expert's time. (It is also why I am skeptical of the louder forecasts about which procurement jobs AI replaces: review time is where the substitution math gets decided.)

To make this concrete, we built a workbook that models ten common tasks across fifteen current models, with review and redo included. Download it free: AI Model TCO Calculator (Excel) or the price and benchmark table (CSV). Prices are checked as of November 17, every formula is live, and the yellow cells are the assumptions you should replace with your own numbers -- above all the success rates, which are illustrative.


Bar chart comparing monthly full cost and API spend for four scenarios: routed by task, all frontier model, all budget model, and no AI
The workbook's example organization: the budget model wins the API bill by a factor of fifty and loses the full cost by a factor of two and a half -- Source: LightSource AI Model TCO Calculator (Nov 2026); illustrative volumes and success rates

For a hypothetical mid-size procurement and engineering organization running about 280,000 AI tasks a month, the example lands like this. Running everything on the cheapest budget model costs $403 a month in API fees and about $309,000 once you count the human review and the redo of everything it gets wrong. Running everything on Fable 5.2 costs $20,500 in API fees and about $128,000 all in. Routing each task to the model with the lowest full cost spends about $9,500 on APIs and lands near $122,000, including $2,000 a month of evaluation overhead. Doing the same work with no AI would cost about $667,000 in labor.

Two things in that table are easy to miss. The budget model could give its inference away free and save this organization $403 a month; the opportunity is in the $309,000 column, where people are checking and finishing the work. And routing cuts about $11,000 from the all-frontier API bill but only about $7,000 from the full cost, because some of the saving comes back as review and redo. That $7,000 is a useful ceiling: it is roughly what this organization could spend on routing software and upkeep before routing stops paying for itself against simply buying the best model.

Route Work by Task

The obvious conclusion from those numbers is to buy the best model for everything, and it is the wrong conclusion. The full-cost curves look very different by task, and the shape is the useful part.


Routing chart showing full cost per completed task for ten business tasks on Claude Haiku 5.5, Sonnet 5.5 and Fable 5.2, with a recommended tier for each
Ten common tasks priced on three tiers of one model family: routine work converges, judgment work splits apart -- Source: LightSource AI Model TCO Calculator (Nov 2026); list prices Nov 17, 2026; success rates illustrative

For bounded work -- classifying a line item, drafting a supplier follow-up, pulling fields off an invoice -- the models converge. Drafting a routine supplier email costs about $0.0005 in API fees on Haiku 5.5 and $0.036 on Fable 5.2, a seventy-fold spread, and on full cost they land within about 20 cents of each other, because the minute a buyer spends reading the draft dwarfs either token bill. When tiers are that close, buy the cheaper one: your estimate of each model's success rate is not precise enough to justify paying more, and the small model is usually faster. For an email, Haiku or Sonnet is plenty.

For judgment and long-horizon work the curves split apart. A should-cost analysis across a stack of quotes costs about $0.02 per attempt on Haiku and $1.42 on Fable, and the full cost runs the other way: about $56 on Haiku against $22 on Fable, because the smaller model's analysis is wrong often enough that a buyer ends up doing the work anyway. A multi-source research report shows the same pattern more sharply, roughly $300 on Haiku against $86 on Fable once redo is counted. Once the denominator is finished work, the expensive model is the cheap option.

A useful way to decide the middle cases is to price the premium in reviewer time. In the workbook, Fable costs $0.91 for a 40-page contract summary against Sonnet's $0.24, a premium of $0.67. At the BLS paralegal median wage of $30.24 an hour, that premium pays for itself if Fable saves about 80 seconds of review, or if it raises first-pass acceptance by about 2.2 percentage points when a rejected summary takes an hour to redo. Those are break-even thresholds, not claims about either model, and they turn a debate about benchmark points into something a paralegal can test with a stopwatch.

Across all ten tasks in the chart, the cheapest tier within 20% of the best full cost works out like this: classifying a line item, drafting a supplier email and extracting an invoice to JSON go to the small tier; support chats, answers from internal documents, 40-page contract summaries and roughly 30-step browser tasks go to the mid tier; should-cost analysis, coding-agent bug fixes and multi-source research reports go to the frontier tier. API cost per attempt runs from about $0.00003 (classifying on Haiku) to $4.23 (a browser task on Fable).

The routing rule that falls out of this is closer to sourcing than to engineering: route on who can check the answer and what a wrong answer costs, not on how hard the prompt sounds.

  • Small tier (Haiku 5.5, GPT-6 Luna, DeepSeek V4.1 Flash, gpt-oss): classification, extraction to a schema, tagging, short drafts, translation -- anything with a built-in check or a low cost of error. Batch it when you can.

  • Mid tier (Sonnet 5.5, GPT-6.2 Sol, GPT-6.1 Terra, Kimi K3.1 Pro): answers from internal documents, contract summaries, support conversations, most coding with tests, and browser tasks where a person can cheaply finish what the agent misses. GPT-6.2 Sol is the standout of the month; OpenAI reports it within half a point of Astra on its FrontierCode benchmark at about a fifth of the cost.

  • Frontier tier (Fable 5.2, GPT-6.2 Astra, Opus 5.5, Gemini 4 Carbon) at high effort: should-cost and negotiation analysis, contract risk review, deep research, and coding agents working across an unfamiliar codebase.

  • The supplier, not a model: when a quote leaves out whether the tooling charge is recurring, a second model will only give you a more polished description of the same gap. Route missing commercial terms to a clarification email and keep the quote open until the supplier answers.

Three cautions keep routing honest. First, the baseline to beat is the best fixed model for each workflow, not "always use the most expensive model." An October audit of commercial routers found one sending 100% of traffic to a single model, and OpenRouter's own November benchmark showed a fixed GLM-5.3 Flash beating both of its routers on a coding suite at about $0.20 per attempt. Second, savings in the lab shrink in production. GitHub's HyDRA router for Copilot cut cost 54% on SWE-bench while staying within 0.3 points of always using the strong model; in a live test with close to a million users per arm, the realized saving was 7 to 20%, because cheaper requests invited more requests. Third, a working router changes the data you judge models on. Once the clean quotes all go to Haiku, Sonnet sees mostly exceptions and its success rate appears to fall. Keep a small fixed sample running through every approved model, or you will demote a model next quarter for a change in the work you gave it.

Not All Tokens Are Equal

Underneath all of this is a question about the unit itself. If one model's token is "smarter" than another's, then comparing their prices per token is like comparing the hourly rate of a senior engineer and an intern without asking what each gets done in the hour.

Imagine a million poodles collaborating on a novel. In aggregate there is a great deal of effort in the room and, arguably, a great deal of raw brainpower. You will still get a worse book than one competent author would write, and a second million poodles will not help. Even if one poodle somewhere produces a readable chapter, somebody has to find it among the million, and the editor paid by the hour would have preferred to hire the author. Some work is not the sum of many small units of intelligence. It needs a single unit of enough intelligence to hold the whole problem at once.


The research on this has gotten much sharper this year. In July I wrote about the "Large Language Monkeys" result, in which a coding model went from solving 15.9% of SWE-bench Lite issues in one attempt to 56% with 250 attempts -- evidence that you can manufacture intelligence from cheap tokens when a test can verify the answer. The limit of that idea is now well documented. Princeton's Benedikt Stroebl, Sayash Kapoor and Arvind Narayanan, in a paper presented at ICLR this year, showed that once the checker is imperfect, "no amount of inference scaling of weaker models can enable them to match the single-sample accuracy of a sufficiently strong model." Their analysis put the best number of attempts at often fewer than ten. For procurement the imperfect checker is familiar: "the line items add up" is a cheap test, and it will happily approve a table in which every price is in the wrong currency.

METR's time-horizon research shows why the gap is nonlinear. Its original study found models succeeding almost 100% of the time on tasks that take a human under four minutes and less than 10% of the time on tasks taking more than four hours. By this spring METR estimated that frontier models could complete about twelve-hour tasks half the time, but only about ninety-minute tasks with 80% reliability. Capability falls off a cliff as tasks get longer, and a slightly weaker model moves the cliff much closer. A token from a model that can hold a twelve-hour task together is a different product from a token that cannot, the same way a journeyman machinist's hour and an apprentice's hour are different products even though both appear on a timesheet.

The economic version of this argument has gone mainstream. OpenAI's CFO Sarah Friar proposed "Useful Intelligence per Dollar" as a scorecard in July, writing that "the lowest price per token does not always produce the lowest cost per outcome." Shelly Palmer put the procurement problem plainly in June: tokens are "easy to meter and easy to price, but hard to value." And the price of a fixed level of intelligence keeps collapsing. Epoch AI's September report estimates that the cost of reaching a given level of performance, reasoning tokens included, has fallen about 47% per quarter -- 13 times a year -- and that o3's January 2025 score on a PhD-level science benchmark, which cost about $0.30 per question then, now costs $0.0004.

That last point cuts both ways, and it is the strongest argument against my own framing. If the cost of yesterday's frontier falls thirteen-fold a year, then for any task yesterday's frontier could do, today's budget model is the right buy, and price per token among the models that clear your bar is exactly the right comparison. The MIT FutureTech group put numbers on both halves: a fixed level of capability gets 5 to 10 times cheaper each year, while running the current frontier gets 3 to 18 times more expensive each year as models think longer. Both are true at once. Intelligence per dollar rises steadily for work you already know how to hand off, and the newest, hardest work keeps getting more expensive because the frontier keeps moving onto it. Epoch's own caveat applies too: its curve assumes a buyer who relentlessly switches to the most economical model, and an enterprise with a security review and an annual commitment lives on a slower one.

I would still keep price per token on the finance dashboard. It is the right unit for auditing whether a provider billed cache reads at the contracted rate, for choosing between hosts serving the same pinned model once quality and latency are shown to match, and for forecasting bounded, repetitive work where the token count per task is stable. It stops working the moment the comparison crosses from one model to another, or from one call to a job with an unknown number of calls and a person at the end.

What the Benchmarks Say

If intelligence has a price, you need a way to measure the quantity, and there is a large body of prior work doing exactly that. The workbook's Benchmarks tab covers sixteen current models across twelve public benchmarks. Three families matter most for a buyer.

The first is composite indexes. Artificial Analysis blends ten evaluations into one score and, usefully, publishes what it cost to run each model through them. Epoch AI's Capabilities Index stitches dozens of benchmarks into a single scale. LMArena ranks models by blind human preference. These are good for building a shortlist and poor for the final call, because each compresses very different skills into one number -- and whoever builds the index decides what a point is worth.

The second is benchmarks that report cost next to score. ARC Prize publishes cost per task for every model on its reasoning puzzles. Princeton's Holistic Agent Leaderboard was built on the argument in Kapoor and colleagues' 2024 paper that "agent evaluations must be cost-controlled," and its headline finding still stands: "Agents can be 100x more expensive while only being 1% better." Vals.ai runs every model on the same harness against business tasks -- research on financial filings, building models in Excel, corporate tax questions -- and reports cost per test. For a procurement audience, Vals is the closest public proxy for real work.


Dumbbell chart comparing Claude Fable 5.2, Claude Haiku 5.5 and DeepSeek V4.1 Flash across eight benchmarks, with frontier lead in points and cost per test
The frontier lead is 28-30 points on long, open-ended tests and single digits on bounded business tasks; on corporate tax research the open model wins -- Sources: Artificial Analysis, Vals.ai, Epoch AI (fetched Nov 17, 2026)

Putting those families side by side shows that the size of the intelligence gap depends on the job. On Artificial Analysis's composite index, DeepSeek V4.1 Flash (32.6) scores half of Fable 5.2 (64.6). On the Vals Index of business tasks it scores 61.3% against Fable's 71.3%, at $0.66 per test against $17.94. On Vals' finance research benchmark, Haiku 5.5 scores 65.4% at $1.64 a test against Fable's 69.5% at $4.83. On Vals' corporate tax benchmark the order reverses: DeepSeek scores 44.4% at $0.46 a test, Fable 32.1% at $3.31. On Vals' Excel modeling benchmark (LBO and DCF builds), Fable leads 77.4% to DeepSeek's 69.2% and Haiku's 65.5%, and on the PhD-level GPQA Diamond science questions all three sit between 90% and 99%. Then on Terminal-Bench, a long agentic coding test, the gap opens back up to 32% and 36% for the small models against 65% for Fable, and on Humanity's Last Exam it is 74% for Fable against 44% and 43%. Bounded, well-specified business work compresses the differences between models, and long, open-ended work stretches them -- which is the poodle problem showing up in the data.

The usual caveats apply and are worth repeating. Benchmarks are not your work. SWE-bench Verified was retired this year after contamination concerns, and Scale's SWE-Bench Pro found one frontier model scoring 17.8 points lower on private code than on public code. "The Leaderboard Illusion" documented how private testing and selective disclosure can skew LMArena. Use benchmarks to cut fifty models to five, and then test the five on your own examples.

Price the Accepted Output

The adjusted metric I would put in front of a CFO is cost per accepted output: everything you spend to get one unit of finished, checked work, divided by the units you actually accept. For a sourcing team the unit might be one supplier quote normalized to the correct revision, currency, unit of measure and price breaks, accepted by someone other than the system that produced it. A model that enters "$48.00 per 100 pieces" as $48 a piece has returned perfectly formatted data in under a second at exactly the token rate you negotiated, and it has not delivered the unit.

There are a few published forms of this metric, and they answer slightly different questions.

Cost-of-pass. Mehmet Hamza Erol and colleagues define it as the cost of one attempt divided by the probability that the attempt is correct -- the expected cost of a correct answer if you retry until you get one. A $0.10 attempt with a 50% pass rate costs an expected $0.20 per pass. Their findings line up with everything above: small models win on basic quantitative work, large models on knowledge-intensive work, reasoning models on complex problems, and tricks like majority voting usually raise the cost-of-pass rather than lowering it. The weakness is that it goes to infinity when a model can never succeed, which is the poodle problem stated as math.

Cost per completed task. GDPval's version assumes the work has to get done: try the model, pay for review, and if it fails, pay a person to finish. With one attempt, that is API cost plus review cost plus the failure rate times the human cost. This is the formula in the workbook, and for business work it is the one I trust, because a wrong quote analysis cannot quietly drop out of the denominator.

Risk-adjusted cost. Add the rate at which a wrong answer slips past review, times what it costs when it does. For a supplier email that term is near zero. For a pricing commitment to a contract manufacturer it can dominate everything else, and for an unauthorized commitment or a prohibited disclosure I would use a hard disqualification rule rather than a dollar allowance.

In practice the scorecard is short: cost per attempted task for budgeting, cost per accepted output for award decisions, and coverage -- the share of work finished without a person stepping in -- reported right next to both. A model that looks cheap per success because it silently skips the hard cases is a supplier with a great unit price and a terrible fill rate. Put the deadline in the definition too: if the team needs 40 quotes for a Thursday award meeting, 38 accepted on Friday is a different service.

What to Put in the Contract

Buying AI well looks like ordinary request for proposal discipline pointed at a strange new commodity. My colleague Aparna Keswani has mapped where AI delivers across the sourcing lifecycle; this is the commercial side of buying the AI itself. Six things I would put in the next RFP:

  1. Specify the deliverable, not the model. Define what an accepted output is for each workflow -- the right supplier, part, currency, unit of measure and quantity breaks, traced to the source document -- and what counts as a material error.

  2. Run the same blind workload on every bidder. Include messy scans, revisions and deliberately ambiguous cases, hold out whole supplier document families, and include a cheap fixed model, a strong fixed model, a routed policy and your current human process as baselines.

  3. Ask for two bid sheets. One is the rate card by model, host, region, token class, effort level and speed tier. The other is cost per accepted output on your replay, with token logs, failure rates and reviewer minutes.

  4. Never lock the unit price. With the cost of constant capability falling thirteen-fold a year, insist on price reopeners, substitution rights when a better model ships, and pass-through of cache and batch discounts. The Gemini 4 Carbon price doubling on December 1 is a reminder that promotions end.

  5. Price the cost of switching. If a new model saves $0.02 per completed job and qualifying it costs $6,000, it takes 300,000 comparable jobs to earn the switch back. A clause that lets you move a commitment across approved models can be worth more than another point off a rate you will not want for long.

  6. Monitor in production. Put total spend, cost per accepted output, coverage and critical errors on one dashboard, and reopen the decision when any of them moves, not when a new model tops a leaderboard. (The same order of operations I argued for in fluency first applies: get people using the cheap tier well before you automate the expensive one, and remember that deployment is where the value gets built.)

This is the discipline our customers already apply to castings, resins and harnesses. LightSource is built for that work in direct materials: normalizing supplier bids line by line, tying every price back to the spec, and showing total cost rather than the number on the quote. The same habits travel well to the newest line item on the P&L.

The models will keep getting better and cheaper, and the top of the leaderboard will keep changing hands. The buyers who come out ahead will be the ones who can say, for each kind of work they hand to a model, what a finished unit costs and what it is worth.

Sources

Frequently Asked Questions

What is the cheapest AI model per task in 2026?

It depends on the task. For classification, extraction and short drafting, small models like Claude Haiku 5.5, GPT-6 Luna or DeepSeek V4.1 Flash cost fractions of a cent per attempt and land close to frontier models on full cost. For long, judgment-heavy work like research reports or should-cost analysis, a frontier model is usually cheaper per finished task, because smaller models fail often enough that a person ends up redoing the work.

Is it cheaper to run open-weight models on OpenRouter or Bedrock than to buy closed models?

Usually on a per-token basis, often by five to twenty times. But the same open model can vary several-fold in price and speed across hosts, cloud marketplaces sometimes charge a premium over the developer's own API, and the best open model still trails the frontier by a wide margin on the hardest tasks. Compare on cost per accepted output for your workload, not on list price.

What is cost-of-pass for AI models?

Cost-of-pass is the expected cost of getting one correct answer from a model: the cost of a single attempt divided by the probability that the attempt is correct. Erol and colleagues formalized it in 2025, and it is useful for comparing models of very different price and capability. It becomes infinite when a model cannot solve a task at all, which is why business teams usually pair it with a version that includes human fallback.

How do reasoning tokens affect AI costs?

Reasoning models generate hidden "thinking" tokens before they answer, and those are billed at the output rate. The effect is large: on Artificial Analysis's benchmark runs, the same model at the same price cost about 8.6 times more per task at maximum reasoning effort than at low effort. Setting effort by task is one of the cheapest optimizations available.

How many examples should an AI acceptance test include?

Start from the error rate you need to detect. If 1% of jobs contain a critical error, a 100-job test still has about a 37% chance of showing no critical errors at all (0.99 to the 100th power). Supplier templates make failures cluster, so supplement a representative sample with targeted tests of the rare, expensive mistakes.

Should procurement own AI model purchasing?

Procurement should at least own the commercial structure: rate cards, price reopeners, substitution rights, data terms and spend monitoring. The technical team should own the evaluations that define an accepted output. The best results come when the two work together the way they do on any engineered component, with a spec, a qualification test and more than one approved supplier.

Every buyer I have worked with learns the same lesson early: the price on the quote is not the price of the part. A casting quoted at $4.10 with a 12% scrap rate costs more than one quoted at $4.40 that comes off the line clean, once you count the rework, the expediting, and the line that sat idle waiting for good parts. Manufacturing has a name for the number that matters -- total landed cost per conforming part -- and it took the industry a generation to stop buying on unit price.

AI is now one of the fastest-growing inputs companies buy, and almost everyone still buys it on unit price: dollars per million tokens. Ramp's card and invoice data shows 56.8% of US businesses paying for AI as of October, and in one recent four-week window the blended price per token fell 14.8% while total spend rose 10.5%, because volume grew 29%. The meter is getting cheaper and the bill is getting bigger, which is roughly what happens when a buyer negotiates the unit price of an input whose consumption nobody has specified.

In July I wrote about renting the model and owning the evals -- the case for treating models as a tiered, multi-sourced commodity behind your own acceptance tests. This is the November update, and it goes a level deeper into the money: what you actually pay per finished piece of work, where to route each kind of task, and whether "price per token" was ever the right unit. It comes with a downloadable total cost of ownership workbook so you can run the numbers on your own work.

Labs vs. Open Weights

There are two purchases hiding inside "which model should we use": whose model, and who runs it. You can buy a frontier model directly from the lab that trained it -- Anthropic, OpenAI, Google, xAI -- or through a cloud that resells it. Or you can buy an open-weight model like Kimi, GLM, DeepSeek, Qwen or gpt-oss from any host that chooses to serve it: an aggregator like OpenRouter, a cloud marketplace like AWS Bedrock, a specialist like Together, Fireworks, Groq or Cerebras, or your own GPUs. (We looked at who owns each layer of the AI stack in October; this is the view from the buyer's end of it.)

The frontier end of the market moved twice in eight days. Anthropic released Claude Fable 5.2 on November 9 at $8 input / $40 output per million tokens, down from $10/$50. OpenAI released GPT-6.2 Astra this morning at the same $8/$40, 20% below the GPT-6 Astra it replaces. Google's Gemini 4 Carbon lists at $2/$10, but that is an introductory price that doubles to $4/$20 on December 1. One tier down, Sonnet 5.5 and GPT-6.2 Sol both sit at $2/$10, and Haiku 5.5 and GPT-6 Luna at $0.10/$0.50.

The open-weight side is cheaper and more fragmented. Kimi K3.1 Pro, the strongest open model on Artificial Analysis's Intelligence Index, lists at $2/$10 from Moonshot and $1.90/$9.50 on DeepInfra. DeepSeek V4.1 Flash runs $0.30/$1.20 on DeepSeek's own API at peak hours and half that off-peak. Menlo Ventures' November report finds open models handling about one in five enterprise tokens but only about one in 25 dollars of model spending, which says both how cheap they are and how much of the serious work still runs on closed models.

Four things about the channel matter more than most buyers expect:

  • The same model is not always the same price. Bedrock's global cross-region pricing for Claude Opus 5.5 matches Anthropic's direct $4/$20 exactly, while in-region inference costs $4.40/$22. Bedrock does not list a batch discount for the current Claude models, so overnight batch work costs about half as much bought direct. For open models the reseller premium can be large: Bedrock prices GLM-5.3 about 20% above Z.ai and offers Kimi K3.1 Pro only as a "Highspeed" tier at twice Moonshot's rate.

  • Aggregators charge at the edges. OpenRouter passes through each host's list price with no inference markup, then charges 5.5% on credit purchases (8% on its Business plan) and 5% on bring-your-own-key usage above $25,000 a month. That is a fair price for one bill, one API and instant switching, and it belongs in the TCO.

  • Host dispersion is wide, and it is not only about price. On OpenRouter's November 17 snapshot, gpt-oss-120b ran at $0.03/$0.15 and 53 output tokens per second on one host and $0.35/$0.75 at 742 tokens per second on Cerebras: five times the output price for fourteen times the speed. For an overnight extraction job the speed buys nothing. For a buyer waiting on a screen, it can be worth more than the tokens. Neither speed is a contractual service level.

  • Data residency eliminates bids before price does. If supplier drawings must stay in one region, a global endpoint is ineligible at any price. Put the region and the permitted fallback hosts on the bid sheet, or a platform team can do exactly what you asked during an outage and route a drawing somewhere your supplier agreement prohibits.

Self-hosting is the make-or-buy decision at the bottom of this list. Lambda lists an eight-GPU H100 node at $4.59 per GPU-hour, about $26,400 for a 720-hour month before operations and failover, and the math only works at high, steady utilization: the same node at 20% utilization costs four times as much per task as it does at 80%. At AI Engineer NYC in October, Revolut's David Abitbol described serving GLM-5.3 to about 2,000 engineers on 56 B200 GPUs at a blended cost near $0.30 per million tokens, around a tenth of the company's previous Anthropic and OpenAI invoice. That is a real result for a high-volume buyer with its own serving team, not a wholesale price available to everyone.


Scatter chart of 25 AI models plotting Artificial Analysis Intelligence Index against cost to run the index, highlighting Claude Fable 5.2 and GPT-6.2 Astra
Fable 5.2 and GPT-6.2 Astra share an $8/$40 list price and score within three points of each other, yet Fable cost $22,191 to run Artificial Analysis's full index to Astra's $4,530 -- Source: Artificial Analysis, Intelligence Index v5.0.1 (Nov 17, 2026)

The chart above is where the price list stops being useful. Fable 5.2 and GPT-6.2 Astra have identical list prices and nearly identical scores on the Intelligence Index (64.6 and 62.1). Running the full index cost $22,191 on Fable at maximum effort and $4,530 on Astra, because Fable generated 275 million output tokens to Astra's 66 million. Sonnet 5.5 at maximum effort, at a quarter of Fable's price, cost $7,981 -- more than Opus 5.5 -- for the same reason. A price per token tells you what one unit costs and nothing about how many units a model will decide to spend on your problem.

What a Token Actually Costs

A token invoice has more lines than a price sheet, and procurement teams should model all of them before comparing bids:

  • Reasoning tokens. Modern models think before they answer, and you pay for the thinking at the output rate. On Artificial Analysis's runs, Fable 5.2's total output ranged from 34.9 million tokens at low effort to 275 million at max, at the same token price -- $3.10 per task against $26.54. Reasoning effort is now about as large a lever as switching model tier, and you can change it without changing suppliers.

  • Tokenizers. Different models count the same text differently. When Anthropic released Opus 4.7 in April, it noted the new tokenizer could produce 1.0 to 1.35 times as many tokens for identical input. An unchanged price can be a 35% increase on your documents.

  • Caching and batch. Cache reads run 90% or more below the input price, and batch APIs typically cut 50%. For agents that re-read the same context on every step, a good cache hit rate is a bigger lever than vendor choice.

  • Failures and retries. A 2026 study of coding agents found that repeated runs of the same task could differ by up to 30 times in token use. Budget for the distribution, not the average.

  • Review and rework. Somebody reads the output. On most business tasks this line is larger than the token bill, and it is the line almost nobody puts in the model comparison.

  • Latency, governance and capacity. Regional data processing carries a 10% premium at both OpenAI and Anthropic, rate limits can bind before budget does, and a person blocked waiting on a slow model is a cost even when the API call is cheap.

The review line deserves the emphasis. OpenAI's GDPval study, which graded model output against experts in 44 occupations (buyers and purchasing agents among them), found GPT-5 roughly 474 times cheaper than a professional on raw inference cost. In the same paper's more realistic workflow -- try the model once, pay an expert to review it, and have the expert redo the work when the model falls short -- the advantage shrank to 1.18 times. Nothing about the model changed between those two numbers; the second one simply includes the expert's time. (It is also why I am skeptical of the louder forecasts about which procurement jobs AI replaces: review time is where the substitution math gets decided.)

To make this concrete, we built a workbook that models ten common tasks across fifteen current models, with review and redo included. Download it free: AI Model TCO Calculator (Excel) or the price and benchmark table (CSV). Prices are checked as of November 17, every formula is live, and the yellow cells are the assumptions you should replace with your own numbers -- above all the success rates, which are illustrative.


Bar chart comparing monthly full cost and API spend for four scenarios: routed by task, all frontier model, all budget model, and no AI
The workbook's example organization: the budget model wins the API bill by a factor of fifty and loses the full cost by a factor of two and a half -- Source: LightSource AI Model TCO Calculator (Nov 2026); illustrative volumes and success rates

For a hypothetical mid-size procurement and engineering organization running about 280,000 AI tasks a month, the example lands like this. Running everything on the cheapest budget model costs $403 a month in API fees and about $309,000 once you count the human review and the redo of everything it gets wrong. Running everything on Fable 5.2 costs $20,500 in API fees and about $128,000 all in. Routing each task to the model with the lowest full cost spends about $9,500 on APIs and lands near $122,000, including $2,000 a month of evaluation overhead. Doing the same work with no AI would cost about $667,000 in labor.

Two things in that table are easy to miss. The budget model could give its inference away free and save this organization $403 a month; the opportunity is in the $309,000 column, where people are checking and finishing the work. And routing cuts about $11,000 from the all-frontier API bill but only about $7,000 from the full cost, because some of the saving comes back as review and redo. That $7,000 is a useful ceiling: it is roughly what this organization could spend on routing software and upkeep before routing stops paying for itself against simply buying the best model.

Route Work by Task

The obvious conclusion from those numbers is to buy the best model for everything, and it is the wrong conclusion. The full-cost curves look very different by task, and the shape is the useful part.


Routing chart showing full cost per completed task for ten business tasks on Claude Haiku 5.5, Sonnet 5.5 and Fable 5.2, with a recommended tier for each
Ten common tasks priced on three tiers of one model family: routine work converges, judgment work splits apart -- Source: LightSource AI Model TCO Calculator (Nov 2026); list prices Nov 17, 2026; success rates illustrative

For bounded work -- classifying a line item, drafting a supplier follow-up, pulling fields off an invoice -- the models converge. Drafting a routine supplier email costs about $0.0005 in API fees on Haiku 5.5 and $0.036 on Fable 5.2, a seventy-fold spread, and on full cost they land within about 20 cents of each other, because the minute a buyer spends reading the draft dwarfs either token bill. When tiers are that close, buy the cheaper one: your estimate of each model's success rate is not precise enough to justify paying more, and the small model is usually faster. For an email, Haiku or Sonnet is plenty.

For judgment and long-horizon work the curves split apart. A should-cost analysis across a stack of quotes costs about $0.02 per attempt on Haiku and $1.42 on Fable, and the full cost runs the other way: about $56 on Haiku against $22 on Fable, because the smaller model's analysis is wrong often enough that a buyer ends up doing the work anyway. A multi-source research report shows the same pattern more sharply, roughly $300 on Haiku against $86 on Fable once redo is counted. Once the denominator is finished work, the expensive model is the cheap option.

A useful way to decide the middle cases is to price the premium in reviewer time. In the workbook, Fable costs $0.91 for a 40-page contract summary against Sonnet's $0.24, a premium of $0.67. At the BLS paralegal median wage of $30.24 an hour, that premium pays for itself if Fable saves about 80 seconds of review, or if it raises first-pass acceptance by about 2.2 percentage points when a rejected summary takes an hour to redo. Those are break-even thresholds, not claims about either model, and they turn a debate about benchmark points into something a paralegal can test with a stopwatch.

Across all ten tasks in the chart, the cheapest tier within 20% of the best full cost works out like this: classifying a line item, drafting a supplier email and extracting an invoice to JSON go to the small tier; support chats, answers from internal documents, 40-page contract summaries and roughly 30-step browser tasks go to the mid tier; should-cost analysis, coding-agent bug fixes and multi-source research reports go to the frontier tier. API cost per attempt runs from about $0.00003 (classifying on Haiku) to $4.23 (a browser task on Fable).

The routing rule that falls out of this is closer to sourcing than to engineering: route on who can check the answer and what a wrong answer costs, not on how hard the prompt sounds.

  • Small tier (Haiku 5.5, GPT-6 Luna, DeepSeek V4.1 Flash, gpt-oss): classification, extraction to a schema, tagging, short drafts, translation -- anything with a built-in check or a low cost of error. Batch it when you can.

  • Mid tier (Sonnet 5.5, GPT-6.2 Sol, GPT-6.1 Terra, Kimi K3.1 Pro): answers from internal documents, contract summaries, support conversations, most coding with tests, and browser tasks where a person can cheaply finish what the agent misses. GPT-6.2 Sol is the standout of the month; OpenAI reports it within half a point of Astra on its FrontierCode benchmark at about a fifth of the cost.

  • Frontier tier (Fable 5.2, GPT-6.2 Astra, Opus 5.5, Gemini 4 Carbon) at high effort: should-cost and negotiation analysis, contract risk review, deep research, and coding agents working across an unfamiliar codebase.

  • The supplier, not a model: when a quote leaves out whether the tooling charge is recurring, a second model will only give you a more polished description of the same gap. Route missing commercial terms to a clarification email and keep the quote open until the supplier answers.

Three cautions keep routing honest. First, the baseline to beat is the best fixed model for each workflow, not "always use the most expensive model." An October audit of commercial routers found one sending 100% of traffic to a single model, and OpenRouter's own November benchmark showed a fixed GLM-5.3 Flash beating both of its routers on a coding suite at about $0.20 per attempt. Second, savings in the lab shrink in production. GitHub's HyDRA router for Copilot cut cost 54% on SWE-bench while staying within 0.3 points of always using the strong model; in a live test with close to a million users per arm, the realized saving was 7 to 20%, because cheaper requests invited more requests. Third, a working router changes the data you judge models on. Once the clean quotes all go to Haiku, Sonnet sees mostly exceptions and its success rate appears to fall. Keep a small fixed sample running through every approved model, or you will demote a model next quarter for a change in the work you gave it.

Not All Tokens Are Equal

Underneath all of this is a question about the unit itself. If one model's token is "smarter" than another's, then comparing their prices per token is like comparing the hourly rate of a senior engineer and an intern without asking what each gets done in the hour.

Imagine a million poodles collaborating on a novel. In aggregate there is a great deal of effort in the room and, arguably, a great deal of raw brainpower. You will still get a worse book than one competent author would write, and a second million poodles will not help. Even if one poodle somewhere produces a readable chapter, somebody has to find it among the million, and the editor paid by the hour would have preferred to hire the author. Some work is not the sum of many small units of intelligence. It needs a single unit of enough intelligence to hold the whole problem at once.


The research on this has gotten much sharper this year. In July I wrote about the "Large Language Monkeys" result, in which a coding model went from solving 15.9% of SWE-bench Lite issues in one attempt to 56% with 250 attempts -- evidence that you can manufacture intelligence from cheap tokens when a test can verify the answer. The limit of that idea is now well documented. Princeton's Benedikt Stroebl, Sayash Kapoor and Arvind Narayanan, in a paper presented at ICLR this year, showed that once the checker is imperfect, "no amount of inference scaling of weaker models can enable them to match the single-sample accuracy of a sufficiently strong model." Their analysis put the best number of attempts at often fewer than ten. For procurement the imperfect checker is familiar: "the line items add up" is a cheap test, and it will happily approve a table in which every price is in the wrong currency.

METR's time-horizon research shows why the gap is nonlinear. Its original study found models succeeding almost 100% of the time on tasks that take a human under four minutes and less than 10% of the time on tasks taking more than four hours. By this spring METR estimated that frontier models could complete about twelve-hour tasks half the time, but only about ninety-minute tasks with 80% reliability. Capability falls off a cliff as tasks get longer, and a slightly weaker model moves the cliff much closer. A token from a model that can hold a twelve-hour task together is a different product from a token that cannot, the same way a journeyman machinist's hour and an apprentice's hour are different products even though both appear on a timesheet.

The economic version of this argument has gone mainstream. OpenAI's CFO Sarah Friar proposed "Useful Intelligence per Dollar" as a scorecard in July, writing that "the lowest price per token does not always produce the lowest cost per outcome." Shelly Palmer put the procurement problem plainly in June: tokens are "easy to meter and easy to price, but hard to value." And the price of a fixed level of intelligence keeps collapsing. Epoch AI's September report estimates that the cost of reaching a given level of performance, reasoning tokens included, has fallen about 47% per quarter -- 13 times a year -- and that o3's January 2025 score on a PhD-level science benchmark, which cost about $0.30 per question then, now costs $0.0004.

That last point cuts both ways, and it is the strongest argument against my own framing. If the cost of yesterday's frontier falls thirteen-fold a year, then for any task yesterday's frontier could do, today's budget model is the right buy, and price per token among the models that clear your bar is exactly the right comparison. The MIT FutureTech group put numbers on both halves: a fixed level of capability gets 5 to 10 times cheaper each year, while running the current frontier gets 3 to 18 times more expensive each year as models think longer. Both are true at once. Intelligence per dollar rises steadily for work you already know how to hand off, and the newest, hardest work keeps getting more expensive because the frontier keeps moving onto it. Epoch's own caveat applies too: its curve assumes a buyer who relentlessly switches to the most economical model, and an enterprise with a security review and an annual commitment lives on a slower one.

I would still keep price per token on the finance dashboard. It is the right unit for auditing whether a provider billed cache reads at the contracted rate, for choosing between hosts serving the same pinned model once quality and latency are shown to match, and for forecasting bounded, repetitive work where the token count per task is stable. It stops working the moment the comparison crosses from one model to another, or from one call to a job with an unknown number of calls and a person at the end.

What the Benchmarks Say

If intelligence has a price, you need a way to measure the quantity, and there is a large body of prior work doing exactly that. The workbook's Benchmarks tab covers sixteen current models across twelve public benchmarks. Three families matter most for a buyer.

The first is composite indexes. Artificial Analysis blends ten evaluations into one score and, usefully, publishes what it cost to run each model through them. Epoch AI's Capabilities Index stitches dozens of benchmarks into a single scale. LMArena ranks models by blind human preference. These are good for building a shortlist and poor for the final call, because each compresses very different skills into one number -- and whoever builds the index decides what a point is worth.

The second is benchmarks that report cost next to score. ARC Prize publishes cost per task for every model on its reasoning puzzles. Princeton's Holistic Agent Leaderboard was built on the argument in Kapoor and colleagues' 2024 paper that "agent evaluations must be cost-controlled," and its headline finding still stands: "Agents can be 100x more expensive while only being 1% better." Vals.ai runs every model on the same harness against business tasks -- research on financial filings, building models in Excel, corporate tax questions -- and reports cost per test. For a procurement audience, Vals is the closest public proxy for real work.


Dumbbell chart comparing Claude Fable 5.2, Claude Haiku 5.5 and DeepSeek V4.1 Flash across eight benchmarks, with frontier lead in points and cost per test
The frontier lead is 28-30 points on long, open-ended tests and single digits on bounded business tasks; on corporate tax research the open model wins -- Sources: Artificial Analysis, Vals.ai, Epoch AI (fetched Nov 17, 2026)

Putting those families side by side shows that the size of the intelligence gap depends on the job. On Artificial Analysis's composite index, DeepSeek V4.1 Flash (32.6) scores half of Fable 5.2 (64.6). On the Vals Index of business tasks it scores 61.3% against Fable's 71.3%, at $0.66 per test against $17.94. On Vals' finance research benchmark, Haiku 5.5 scores 65.4% at $1.64 a test against Fable's 69.5% at $4.83. On Vals' corporate tax benchmark the order reverses: DeepSeek scores 44.4% at $0.46 a test, Fable 32.1% at $3.31. On Vals' Excel modeling benchmark (LBO and DCF builds), Fable leads 77.4% to DeepSeek's 69.2% and Haiku's 65.5%, and on the PhD-level GPQA Diamond science questions all three sit between 90% and 99%. Then on Terminal-Bench, a long agentic coding test, the gap opens back up to 32% and 36% for the small models against 65% for Fable, and on Humanity's Last Exam it is 74% for Fable against 44% and 43%. Bounded, well-specified business work compresses the differences between models, and long, open-ended work stretches them -- which is the poodle problem showing up in the data.

The usual caveats apply and are worth repeating. Benchmarks are not your work. SWE-bench Verified was retired this year after contamination concerns, and Scale's SWE-Bench Pro found one frontier model scoring 17.8 points lower on private code than on public code. "The Leaderboard Illusion" documented how private testing and selective disclosure can skew LMArena. Use benchmarks to cut fifty models to five, and then test the five on your own examples.

Price the Accepted Output

The adjusted metric I would put in front of a CFO is cost per accepted output: everything you spend to get one unit of finished, checked work, divided by the units you actually accept. For a sourcing team the unit might be one supplier quote normalized to the correct revision, currency, unit of measure and price breaks, accepted by someone other than the system that produced it. A model that enters "$48.00 per 100 pieces" as $48 a piece has returned perfectly formatted data in under a second at exactly the token rate you negotiated, and it has not delivered the unit.

There are a few published forms of this metric, and they answer slightly different questions.

Cost-of-pass. Mehmet Hamza Erol and colleagues define it as the cost of one attempt divided by the probability that the attempt is correct -- the expected cost of a correct answer if you retry until you get one. A $0.10 attempt with a 50% pass rate costs an expected $0.20 per pass. Their findings line up with everything above: small models win on basic quantitative work, large models on knowledge-intensive work, reasoning models on complex problems, and tricks like majority voting usually raise the cost-of-pass rather than lowering it. The weakness is that it goes to infinity when a model can never succeed, which is the poodle problem stated as math.

Cost per completed task. GDPval's version assumes the work has to get done: try the model, pay for review, and if it fails, pay a person to finish. With one attempt, that is API cost plus review cost plus the failure rate times the human cost. This is the formula in the workbook, and for business work it is the one I trust, because a wrong quote analysis cannot quietly drop out of the denominator.

Risk-adjusted cost. Add the rate at which a wrong answer slips past review, times what it costs when it does. For a supplier email that term is near zero. For a pricing commitment to a contract manufacturer it can dominate everything else, and for an unauthorized commitment or a prohibited disclosure I would use a hard disqualification rule rather than a dollar allowance.

In practice the scorecard is short: cost per attempted task for budgeting, cost per accepted output for award decisions, and coverage -- the share of work finished without a person stepping in -- reported right next to both. A model that looks cheap per success because it silently skips the hard cases is a supplier with a great unit price and a terrible fill rate. Put the deadline in the definition too: if the team needs 40 quotes for a Thursday award meeting, 38 accepted on Friday is a different service.

What to Put in the Contract

Buying AI well looks like ordinary request for proposal discipline pointed at a strange new commodity. My colleague Aparna Keswani has mapped where AI delivers across the sourcing lifecycle; this is the commercial side of buying the AI itself. Six things I would put in the next RFP:

  1. Specify the deliverable, not the model. Define what an accepted output is for each workflow -- the right supplier, part, currency, unit of measure and quantity breaks, traced to the source document -- and what counts as a material error.

  2. Run the same blind workload on every bidder. Include messy scans, revisions and deliberately ambiguous cases, hold out whole supplier document families, and include a cheap fixed model, a strong fixed model, a routed policy and your current human process as baselines.

  3. Ask for two bid sheets. One is the rate card by model, host, region, token class, effort level and speed tier. The other is cost per accepted output on your replay, with token logs, failure rates and reviewer minutes.

  4. Never lock the unit price. With the cost of constant capability falling thirteen-fold a year, insist on price reopeners, substitution rights when a better model ships, and pass-through of cache and batch discounts. The Gemini 4 Carbon price doubling on December 1 is a reminder that promotions end.

  5. Price the cost of switching. If a new model saves $0.02 per completed job and qualifying it costs $6,000, it takes 300,000 comparable jobs to earn the switch back. A clause that lets you move a commitment across approved models can be worth more than another point off a rate you will not want for long.

  6. Monitor in production. Put total spend, cost per accepted output, coverage and critical errors on one dashboard, and reopen the decision when any of them moves, not when a new model tops a leaderboard. (The same order of operations I argued for in fluency first applies: get people using the cheap tier well before you automate the expensive one, and remember that deployment is where the value gets built.)

This is the discipline our customers already apply to castings, resins and harnesses. LightSource is built for that work in direct materials: normalizing supplier bids line by line, tying every price back to the spec, and showing total cost rather than the number on the quote. The same habits travel well to the newest line item on the P&L.

The models will keep getting better and cheaper, and the top of the leaderboard will keep changing hands. The buyers who come out ahead will be the ones who can say, for each kind of work they hand to a model, what a finished unit costs and what it is worth.

Sources

Frequently Asked Questions

What is the cheapest AI model per task in 2026?

It depends on the task. For classification, extraction and short drafting, small models like Claude Haiku 5.5, GPT-6 Luna or DeepSeek V4.1 Flash cost fractions of a cent per attempt and land close to frontier models on full cost. For long, judgment-heavy work like research reports or should-cost analysis, a frontier model is usually cheaper per finished task, because smaller models fail often enough that a person ends up redoing the work.

Is it cheaper to run open-weight models on OpenRouter or Bedrock than to buy closed models?

Usually on a per-token basis, often by five to twenty times. But the same open model can vary several-fold in price and speed across hosts, cloud marketplaces sometimes charge a premium over the developer's own API, and the best open model still trails the frontier by a wide margin on the hardest tasks. Compare on cost per accepted output for your workload, not on list price.

What is cost-of-pass for AI models?

Cost-of-pass is the expected cost of getting one correct answer from a model: the cost of a single attempt divided by the probability that the attempt is correct. Erol and colleagues formalized it in 2025, and it is useful for comparing models of very different price and capability. It becomes infinite when a model cannot solve a task at all, which is why business teams usually pair it with a version that includes human fallback.

How do reasoning tokens affect AI costs?

Reasoning models generate hidden "thinking" tokens before they answer, and those are billed at the output rate. The effect is large: on Artificial Analysis's benchmark runs, the same model at the same price cost about 8.6 times more per task at maximum reasoning effort than at low effort. Setting effort by task is one of the cheapest optimizations available.

How many examples should an AI acceptance test include?

Start from the error rate you need to detect. If 1% of jobs contain a critical error, a 100-job test still has about a 37% chance of showing no critical errors at all (0.99 to the 100th power). Supplier templates make failures cluster, so supplement a representative sample with targeted tests of the rare, expensive mistakes.

Should procurement own AI model purchasing?

Procurement should at least own the commercial structure: rate cards, price reopeners, substitution rights, data terms and spend monitoring. The technical team should own the evaluations that define an accepted output. The best results come when the two work together the way they do on any engineered component, with a spec, a qualification test and more than one approved supplier.

Faster sourcing,
lower cost, less chaos.

See how LightSource connects engineering, procurement, and suppliers in one operating system to help you launch faster at lower cost.

DPW Award Winner

SOC 2

Hackett Group 2026 Validated

Kearney #1

Gartner Cool Vendor

Procuretech 100

G2 Top Rated

Faster sourcing,
lower cost, less chaos.

See how LightSource connects engineering, procurement, and suppliers in one operating system to help you launch faster at lower cost.

DPW Award Winner

SOC 2

Hackett Group 2026 Validated

Kearney #1

Gartner Cool Vendor

Procuretech 100

G2 Top Rated

Faster sourcing,
lower cost, less chaos.

See how LightSource connects engineering, procurement, and suppliers in one operating system to help you launch faster at lower cost.

DPW Award Winner

SOC 2

Hackett Group 2026 Validated

Kearney #1

Gartner Cool Vendor

Procuretech 100

G2 Top Rated

Trusted by:

  • Procuring Intelligence: Finding Value Beyond the Price per Token (Nov 2026 Update) — LightSource Blog
  • Procuring Intelligence: Finding Value Beyond the Price per Token (Nov 2026 Update) — LightSource Blog
  • Procuring Intelligence: Finding Value Beyond the Price per Token (Nov 2026 Update) — LightSource Blog
  • Procuring Intelligence: Finding Value Beyond the Price per Token (Nov 2026 Update) — LightSource Blog
  • Procuring Intelligence: Finding Value Beyond the Price per Token (Nov 2026 Update) — LightSource Blog
  • Procuring Intelligence: Finding Value Beyond the Price per Token (Nov 2026 Update) — LightSource Blog
  • Procuring Intelligence: Finding Value Beyond the Price per Token (Nov 2026 Update) — LightSource Blog
  • Procuring Intelligence: Finding Value Beyond the Price per Token (Nov 2026 Update) — LightSource Blog
  • Procuring Intelligence: Finding Value Beyond the Price per Token (Nov 2026 Update) — LightSource Blog
  • Procuring Intelligence: Finding Value Beyond the Price per Token (Nov 2026 Update) — LightSource Blog
  • Procuring Intelligence: Finding Value Beyond the Price per Token (Nov 2026 Update) — LightSource Blog
  • Procuring Intelligence: Finding Value Beyond the Price per Token (Nov 2026 Update) — LightSource Blog
  • Procuring Intelligence: Finding Value Beyond the Price per Token (Nov 2026 Update) — LightSource Blog

Trusted by:

  • Procuring Intelligence: Finding Value Beyond the Price per Token (Nov 2026 Update) — LightSource Blog
  • Procuring Intelligence: Finding Value Beyond the Price per Token (Nov 2026 Update) — LightSource Blog
  • Procuring Intelligence: Finding Value Beyond the Price per Token (Nov 2026 Update) — LightSource Blog
  • Procuring Intelligence: Finding Value Beyond the Price per Token (Nov 2026 Update) — LightSource Blog
  • Procuring Intelligence: Finding Value Beyond the Price per Token (Nov 2026 Update) — LightSource Blog
  • Procuring Intelligence: Finding Value Beyond the Price per Token (Nov 2026 Update) — LightSource Blog
  • Procuring Intelligence: Finding Value Beyond the Price per Token (Nov 2026 Update) — LightSource Blog
  • Procuring Intelligence: Finding Value Beyond the Price per Token (Nov 2026 Update) — LightSource Blog
  • Procuring Intelligence: Finding Value Beyond the Price per Token (Nov 2026 Update) — LightSource Blog
  • Procuring Intelligence: Finding Value Beyond the Price per Token (Nov 2026 Update) — LightSource Blog
  • Procuring Intelligence: Finding Value Beyond the Price per Token (Nov 2026 Update) — LightSource Blog
  • Procuring Intelligence: Finding Value Beyond the Price per Token (Nov 2026 Update) — LightSource Blog
  • Procuring Intelligence: Finding Value Beyond the Price per Token (Nov 2026 Update) — LightSource Blog

Trusted by:

  • Procuring Intelligence: Finding Value Beyond the Price per Token (Nov 2026 Update) — LightSource Blog
  • Procuring Intelligence: Finding Value Beyond the Price per Token (Nov 2026 Update) — LightSource Blog
  • Procuring Intelligence: Finding Value Beyond the Price per Token (Nov 2026 Update) — LightSource Blog
  • Procuring Intelligence: Finding Value Beyond the Price per Token (Nov 2026 Update) — LightSource Blog
  • Procuring Intelligence: Finding Value Beyond the Price per Token (Nov 2026 Update) — LightSource Blog
  • Procuring Intelligence: Finding Value Beyond the Price per Token (Nov 2026 Update) — LightSource Blog
  • Procuring Intelligence: Finding Value Beyond the Price per Token (Nov 2026 Update) — LightSource Blog
  • Procuring Intelligence: Finding Value Beyond the Price per Token (Nov 2026 Update) — LightSource Blog
  • Procuring Intelligence: Finding Value Beyond the Price per Token (Nov 2026 Update) — LightSource Blog
  • Procuring Intelligence: Finding Value Beyond the Price per Token (Nov 2026 Update) — LightSource Blog
  • Procuring Intelligence: Finding Value Beyond the Price per Token (Nov 2026 Update) — LightSource Blog
  • Procuring Intelligence: Finding Value Beyond the Price per Token (Nov 2026 Update) — LightSource Blog
  • Procuring Intelligence: Finding Value Beyond the Price per Token (Nov 2026 Update) — LightSource Blog