What an AI Cost Optimization Tool Actually Sees

Every AI cost optimization tool makes the same promise: connect it, and it finds the waste. The question buried under that promise is what does it read to do that? — and the honest answer is "more than most teams check before they connect it."

Because these tools operate on prompts, they touch the single most sensitive stream of data your engineers generate. This is a checklist to run before you wire one in — ideally with FinOps and security in the same room.

The four things a cost tool reads

1. Prompts

To route a task to the right model or to compress it, a tool has to read the prompt. Prompts are not benign. They routinely contain:

• Source code and diffs
• Internal service names, hostnames, and architecture details
• Secrets that slip through despite everyone's best intentions
• Occasionally, customer data pulled in as context

If the tool routes traffic, assume it sees all of this, on every request.

2. Surrounding context

Modern coding agents don't send a lone question — they pull in files, repository structure, and diffs. A cost tool sitting in that path sees the context window too, which is often larger and more revealing than the prompt itself.

3. Invocation and token logs

Who called which model, when, how many input and output tokens, how many retries. This is lower-sensitivity than prompt text but still maps your team's internal activity — useful to you, and useful to anyone who shouldn't have it.

4. Provider billing

The real dollar amounts, net of prompt caching. Needed to prove savings honestly. On its own it's the least sensitive item on the list — but it's usually bundled with the rest.

The routing-vs-reporting split

Here's the distinction that changes everything, and that most vendor pages blur:

• A routing tool decides which model handles each task. It must see prompt contents. Maximum data exposure.
• A reporting tool only measures spend after the fact. It can often work from token logs and billing alone — no prompt text required. Much smaller exposure.

Ask which one you're buying. "AI cost optimization" covers both, and they carry wildly different risk. If you only need visibility, don't hand over prompts to get it.

The exposure checklist

Run these before connecting anything:

1. Does it need prompt contents, or just logs and billing? Buy the minimum.
2. Where does the analysis run — your account or the vendor's?
3. Retention — how long are prompts and logs kept, and can you set it to zero?
4. Training — are your prompts used to improve the vendor's models or heuristics?
5. Sub-processors — who else touches the data on the way?
6. Blast radius — if the vendor is breached, what of yours is in it?
7. Deletion — can you get everything purged, and get proof?

A tool that answers "it all runs in your environment" collapses most of this list to "n/a." That's the point of self-hosting.

Why this is a security review, not a budget one

FinOps owns the bill, so FinOps usually buys the cost tool. But the data this tool reads — production prompts and code — puts it squarely in security's lane. The cheapest way to cut AI spend is worthless if it opens a new data-egress path nobody reviewed. We argued the broader case for not shipping your logs to a vendor separately; this is the concrete data inventory behind it.

Where Tokor fits

Tokor is KorPro's AI cost optimization product, built self-hosted first precisely so this checklist stays short. It runs the router and measurement layer inside your own environment, in front of your team's AI coding tools (starting with Claude Code) on Bedrock, Azure AI Foundry, or Vertex. Prompts and code are read where they already live and never leave your cloud, and the savings are measured in net-of-cache dollars you can audit yourself.

Tokor is in early access. If you're evaluating AI cost tools and want the data-exposure answer to be "nothing leaves," apply to the design partner program.

KorPro helps teams find and recover wasted spend across Kubernetes, cloud, and now AI — without adding a new place for your data to leak.