---
title: "How to reduce OpenAI API costs: five levers, ranked | Zeroic"
description: "How to reduce OpenAI API costs on GPT-6: cache the prefix, route easy calls to Luna, batch what can wait. Priced on a 300,000-request month, in ship order."
url: https://zeroic.in/blog/reduce-openai-api-costs
---

[The Zeroic Journal](https://zeroic.in/blog)

[AI development](https://zeroic.in/blog/topic/ai-development) October 19, 2026 7 min read

# How to reduce OpenAI API costs: five levers, ranked

Five ways to cut an OpenAI bill, priced on the same 300,000-request month and ordered by savings per day of work.

By Prashant Abbi Partner @ Zeroic

On this page

1. [The month we priced](#the-month-we-priced)
2. [How to reduce OpenAI API costs, ranked](#how-to-reduce-openai-api-costs-ranked)
3. [1. Fix the prompt order first](#1-fix-the-prompt-order-first)
4. [2. Route easy requests to a cheaper model](#2-route-easy-requests-to-a-cheaper-model)
5. [3. Batch or Flex for work nobody is waiting on](#3-batch-or-flex-for-work-nobody-is-waiting-on)
6. [4. Shrink the output](#4-shrink-the-output)
7. [5. Check long prompts and old fine-tunes](#5-check-long-prompts-and-old-fine-tunes)
8. [Which lever for which product](#which-lever-for-which-product)
9. [How FormulaBot kept inference cheap at 1.5M+ users](#how-formulabot-kept-inference-cheap-at-15m-users)

On this page 9 sections

1. [The month we priced](#the-month-we-priced)
2. [How to reduce OpenAI API costs, ranked](#how-to-reduce-openai-api-costs-ranked)
3. [1. Fix the prompt order first](#1-fix-the-prompt-order-first)
4. [2. Route easy requests to a cheaper model](#2-route-easy-requests-to-a-cheaper-model)
5. [3. Batch or Flex for work nobody is waiting on](#3-batch-or-flex-for-work-nobody-is-waiting-on)
6. [4. Shrink the output](#4-shrink-the-output)
7. [5. Check long prompts and old fine-tunes](#5-check-long-prompts-and-old-fine-tunes)
8. [Which lever for which product](#which-lever-for-which-product)
9. [How FormulaBot kept inference cheap at 1.5M+ users](#how-formulabot-kept-inference-cheap-at-15m-users)

How to reduce OpenAI API costs comes down to three changes: put stable text first so prompt caching bills it at about 10% of the input rate, route simple requests to GPT-6 Luna, 100 times cheaper than Astra per token, and move work that can wait to Batch at half price. On a 300,000-request month, the first two cut 85%.

Nothing below needs fine-tuning, and that matters now: OpenAI is retiring self-serve fine-tuning, so the old “fine-tune a small model” answer is closed to many teams. Every price here is from OpenAI’s [pricing page](https://developers.openai.com/api/docs/pricing) as of October 19, 2026, for requests of 272,000 input tokens or less.

## The month we priced

One workload makes the levers comparable. Say your product sends 300,000 requests a month. Each has 3,000 input tokens, of which 2,500 are a stable prefix (system prompt, schema, few-shot examples, tool definitions) and 500 are the user’s turn, and each returns 300 output tokens.

On GPT-6 Astra, at $10 per million input tokens and $50 per million output tokens, that is $9,000 of input and $4,500 of output: **$13,500 a month**. Output is the smaller volume and the bigger rate. Every GPT-6 model prices output at five times input, so a long answer costs more than a long prompt.

Here is what each lever does to that bill.

## How to reduce OpenAI API costs, ranked

LeverMonthly billSavingWork

Baseline, all Astra$13,500--

1. Cache the 2,500-token prefix$6,75050%Reorder prompts, log hits

2. Route 70% of requests to Luna$4,14569%Router and an eval set

1 + 2 together$2,07285%Both

3. Batch or Flex for work that can waitHalf rate on that shareUp to 50% of that shareA queue

Each saving is computed on the baseline alone, except the combined row. The order is by saving per day of work. Start at the top.

## 1. Fix the prompt order first

Prompt caching needs no new infrastructure and no quality test, so it comes first. OpenAI’s [caching guide](https://developers.openai.com/api/docs/guides/prompt-caching) says a hit needs the whole rendered prefix to match up to a breakpoint. Put developer instructions, shared reference material and tool definitions first. Put timestamps and user text last.

On Astra, cached input costs $1.00 per million against $10.00. In our month, the 2,500-token prefix drops from $7,500 to $750, which takes the bill to $6,750. Two costs sit on top. Models from GPT-5.6 on charge 1.25 times the input rate to write the cache, and the cache lasts 30 minutes, refreshed on each reuse. At roughly seven requests a minute the prefix stays warm, and writes add at most $45 a month.

Caching fails quietly. A timestamp at the top of the system prompt, tools sent in a different order per request, or a summarization step that rewrites earlier turns each reset the prefix. The guide says so directly. Log `cached_tokens` on every response and alert when the ratio drops after a deploy, because nothing else will tell you.

Prompts under 1,024 tokens don’t cache on GPT-5.6 and later, so short prompts get nothing from this lever. For them, skip to routing.

## 2. Route easy requests to a cheaper model

The price gap inside the lineup is wider than most teams use. GPT-6 Luna costs $0.10 input and $0.50 output per million tokens. GPT-6 Sol costs $2 and $10. Astra costs $10 and $50. Luna is 100 times cheaper than Astra per token, and Sol is 5 times cheaper.

If 70% of requests are classification, extraction or short replies that Luna handles, and 30% need Astra, the baseline falls from $13,500 to about $4,145 before caching and to $2,072 with it. A full move to Luna would cost $135 a month, which is why the argument for “just use the small model” never dies. It dies on quality, not on price.

The router is an eval problem. [Hamel Husain’s cascade notes](https://hamel.dev/notes/llm/ai-product-engineering/evals-model-cascades.html) give the method: label about 500 sampled records with the large model, treat each confidence value from the small model as a candidate threshold, and pick the cheapest one that meets your accuracy target. The caveat in the same notes applies to you: the target measures agreement with the large model, and it only works if the small model’s confidence tracks the large model’s answers. If it doesn’t, route by task type instead of by confidence.

We covered how the tiers differ in [our GPT vs Claude vs Llama guide](https://zeroic.in/blog/gpt-vs-claude-vs-llama-production), and the same tiering logic on the Anthropic side in [our Claude Haiku 5.5 cost breakdown](https://zeroic.in/blog/claude-haiku-5-5-cost).

## 3. Batch or Flex for work nobody is waiting on

The [Batch API](https://developers.openai.com/api/docs/guides/batch) takes 50% off the standard rate in exchange for a 24-hour completion window. A batch holds up to 50,000 requests or 200 MB, and it has its own rate-limit pool, so it doesn’t compete with live traffic. Nightly enrichment, evals, backfills and bulk classification belong here.

[Flex processing](https://developers.openai.com/api/docs/guides/flex-processing) bills at Batch rates but runs synchronously: set `service_tier` to `"flex"`. It is in beta, covers only some models, and returns a 429 “resource unavailable” error, uncharged, when capacity is short. Retry with backoff or fall back to the standard tier, and raise your client timeout, because the SDK default is 10 minutes and Flex is slower. OpenAI’s [cost guide](https://developers.openai.com/api/docs/guides/cost-optimization) names evaluations and data enrichment as the fit.

Neither belongs on a request a user is watching. Fast mode, which replaced Priority processing in July, goes the other way and costs about twice the standard rate.

## 4. Shrink the output

Output is billed at five times input, so a shorter answer is the cheapest token you can remove. Structured outputs help twice. With `strict: true`, the model [must follow your JSON Schema](https://developers.openai.com/api/docs/guides/structured-outputs), so an extraction call returns the fields you need and nothing around them. It also removes the retry loop that “return valid JSON” prompts need.

Cut requests too. A pipeline that makes three sequential calls where one would do pays the prefix and the latency three times.

## 5. Check long prompts and old fine-tunes

Past 272,000 input tokens, Astra’s input rate doubles to $20 per million. Cap retrieval so a runaway context doesn’t cross the line.

If you run fine-tuned models, check the [deprecations page](https://developers.openai.com/api/docs/deprecations). Several fine-tuned snapshots are removed on October 23, 2026, and fine-tuned GPT-4o and GPT-4.1 variants shut down January 14, 2027. OpenAI recommends moving them to base models such as GPT-6 Luna, which pairs naturally with lever 2.

## Which lever for which product

- **Your prompts exceed 1,000 tokens and repeat a prefix.** Fix the order and log cache hits this week.
- **More than half your requests classify, extract or route.** Build a 500-example eval set and test Luna on it before anything else.
- **You run nightly jobs or evals on the live tier.** Move them to Batch. It’s a one-day change for half the cost.
- **Your answers are long and structured.** Add a strict schema and cap the output.
- **Your bill is small.** On a bill of a few hundred dollars, skip routing. The engineering time costs more than it saves.
- **You rely on a fine-tune.** Plan the migration before the removal dates.

## How FormulaBot kept inference cheap at 1.5M+ users

[Better Analyst (formerly FormulaBot)](https://zeroic.in/formulabot) reaches 1.5M+ people on an LLM pipeline that two Zeroic engineers built with the founder. The cost levers above are the ones we wrote up in [our English-to-SQL breakdown](https://zeroic.in/blog/english-to-sql-at-1m-users): a stable prefix first, simple queries to a cheaper model, and only the relevant schema in the prompt.

We split AI inference into its own service there. That keeps the model ID, prompt order and routing rules in one place, so a lever like these is a change to one service. If your model calls are scattered through the codebase, that refactor comes before any of the five.

If you’d rather have someone find the biggest saving for you, our [AI Audit](https://zeroic.in/services/ai-audit) ranks every model call in your product by cost and by how safely it can move down a tier.

## Frequently asked questions

**How much does OpenAI prompt caching save?**

Cached input tokens bill at about 10% of the normal input rate on GPT-6 Astra, GPT-6 Sol and GPT-6 Luna, and at 5% on GPT-6.1 Sol. The saving applies only to the part of the prompt that matches a recent request exactly, starting from the first token. On GPT-6 and GPT-5.6 models, writing the cache costs 1.25 times the input rate.

**Is OpenAI's Batch API worth using?**

Yes, for any work that can wait up to 24 hours, such as nightly enrichment, evals, backfills and bulk classification. Batch takes 50% off the standard rate and has its own rate-limit pool. It is the wrong tool for anything a user is waiting on, because results can take up to the full 24-hour window to return.

**What is OpenAI Flex processing and when does it fail?**

Flex is a cheaper service tier, billed at Batch rates, that you request with service_tier set to flex on the Responses or Chat Completions API. It is in beta and covers only some models. When capacity runs out it returns a 429 error without charging you, so retry with backoff or fall back to the standard tier.

**Should I still fine-tune a model to cut costs?**

Not as a first move, and for most teams it is no longer available. OpenAI is retiring self-serve fine-tuning: organizations that never fine-tuned cannot start, and active customers lose new job creation on January 6, 2027. Few-shot examples in a cached prefix, plus routing to a smaller model, get most of the same saving.

**Which GPT-6 model is cheapest for simple tasks?**

GPT-6 Luna is the cheapest GPT-6 model at $0.10 per million input tokens and $0.50 per million output tokens. That is one hundredth of GPT-6 Astra's price per token. It suits classification, extraction and routing. Test it against your own examples first, because a cheaper model that fails 10% of requests costs more in retries than it saves.

## References

1. [OpenAI API pricing](https://developers.openai.com/api/docs/pricing)
2. [OpenAI Docs - Prompt caching](https://developers.openai.com/api/docs/guides/prompt-caching)
3. [OpenAI Docs - Batch API](https://developers.openai.com/api/docs/guides/batch)
4. [OpenAI Docs - Flex processing](https://developers.openai.com/api/docs/guides/flex-processing)
5. [OpenAI Docs - Cost optimization](https://developers.openai.com/api/docs/guides/cost-optimization)
6. [OpenAI Docs - Structured outputs](https://developers.openai.com/api/docs/guides/structured-outputs)
7. [OpenAI Docs - Deprecations](https://developers.openai.com/api/docs/deprecations)
8. [Hamel Husain - Cut classification costs with a model cascade](https://hamel.dev/notes/llm/ai-product-engineering/evals-model-cascades.html)

#openai#llm-cost#prompt-caching#model-routing#batch-api

More from the journal

[ AI development

## [Claude Haiku 5.5 cost: what to move and what breaks](https://zeroic.in/blog/claude-haiku-5-5-cost)

Anthropic cut its small model's price tenfold on October 7. Here's what that does to your product's AI bill, and what to fix before you switch.

Prashant Abbi Oct 15, 2026 · 8 min read

[ AI development

## [GPT vs Claude vs Llama in production: choose by task type](https://zeroic.in/blog/gpt-vs-claude-vs-llama-production)

Four kinds of LLM work, the model tier that wins each one, and the volume at which running open weights yourself starts to make sense.

Prashant Abbi Sep 29, 2026 · 8 min read

[ AI development

## [Custom AI receptionist for clinics: build or buy](https://zeroic.in/blog/custom-ai-receptionist-clinics)

Most clinics should buy. Here are the three cases where building your own wins, what each option costs, and what a build involves.

Prashant Abbi Oct 12, 2026 · 6 min read

## Hear from Prashant within 24 hours.

**Prashant Abbi** Partner @ Zeroic

Want to know where your OpenAI spend goes? An AI audit maps every model call in your product to its cost, then ranks the cache, routing and batch changes by saving, so you ship the biggest first.

[See AI audits](https://zeroic.in/services/ai-audit), or [book a 30-minute call](https://zeroic.in/book)

Pages [Services](https://zeroic.in/services) [Work](https://zeroic.in/work) [Blog](https://zeroic.in/blog) [About](https://zeroic.in/about) [Guide for AI](https://zeroic.in/llms.txt)

Services [Audits](https://zeroic.in/services/ai-audit) [MVP Sprint](https://zeroic.in/services/mvp-development) [Retainer](https://zeroic.in/services/senior-developers) [Code Rescue](https://zeroic.in/services/code-rescue) [Bubble to code](https://zeroic.in/bubble-to-code) [AI development](https://zeroic.in/ai-development-company)

Industries [Marketplaces](https://zeroic.in/marketplace-app-development) [B2B SaaS](https://zeroic.in/saas-mvp-development) [FinTech](https://zeroic.in/fintech-app-development) [Telehealth](https://zeroic.in/telehealth-app-development) [EdTech](https://zeroic.in/edtech-app-development) [All industries →](https://zeroic.in/industries)

Developers [Bubble](https://zeroic.in/hire-bubble-developers) [React](https://zeroic.in/hire-nextjs-developers) [Python](https://zeroic.in/hire-python-developers) [AI & LLMs](https://zeroic.in/hire-ai-developers) [API integrations](https://zeroic.in/api-integration-services) [All developers →](https://zeroic.in/developers)

Contact [Email](mailto:zeroic.in@gmail.com) [WhatsApp](https://wa.me/919873927225) [LinkedIn](https://www.linkedin.com/company/zeroic/) [X (Twitter)](https://x.com/PrashantAbbi) [Book a call](https://zeroic.in/book)

© 2026 Zeroic [Privacy](https://zeroic.in/privacy) [Terms](https://zeroic.in/terms)

Built in India · Shipping globally

SINCE