August 25, 2026 · 6 min read

Charge before inference, refund on failure

How a metered API keeps its books correct when a request is replayed, a stream dies halfway, or the cache it depends on stops answering.

A metered inference API has two ways to bill. Charge after the work succeeds, or charge before it starts and refund when it does not.

The first sounds safer. It is the harder of the two to get right. By the time the output exists the money is already spent, and the only remaining question is whether the customer's balance covers it. If it does not, three bad options are left. Refuse to deliver work you have already done, deliver it and carry the debt, or let the balance go negative and reconcile later.

We charge first. Every request's cost is a deterministic function of its input, characters for speech and translation and pages for a document. The price is therefore knowable before anything runs. That buys one clean invariant and three problems worth writing down, each of them a place where a simpler design quietly loses money.

The invariant is enforced by the database#

Balances live in the database, and never in a number a service holds in memory. The ledger is append-only, the balance always equals its sum, and a CHECK constraint refuses to let it go negative. A per-key spending cap is a second constraint on the same row.

That placement is the whole design. No amount of concurrency, retry, or process death can produce an overdraft, because the guarantee lives below every code path that could have violated it. An application-level check states an intention. A constraint states a fact.

one charge, one statement

BEGIN (implicit)
  UPDATE credit_balance   -- balance lock
     SET ...              -- CHECK forbids < 0
   WHERE user_id = $1
  UPDATE api_key          -- key lock, same order
  INSERT credit_ledger
  INSERT usage_event
COMMIT

one guarded CTE in autocommit: a single
statement is its own transaction, so a
charge is ONE round trip, not three

The refund takes the same two locks in the same order. Lock ordering is the entire reason a charge and a refund arriving at once do not deadlock. It is also the kind of property that survives only by being written down next to both of them.

The stream that dies halfway#

Streaming audio puts bytes on the wire before the generation is known to have succeeded. When the upstream fails after the first chunk, the customer holds part of a file and we hold their credits.

A partial file is not a delivery. The response is truncated and the charge is refunded, whether the failure arrived on the second chunk or the last. The timed variant returns line-delimited JSON and closes with an in-band error line, which lets a client tell a failure from a short generation. The refund runs either way.

The live socket is the same rule applied per leg. Each synthesis leg charges the delta between the cumulative character count and what the connection has already paid. The per-request floor therefore applies once across a whole conversation. A failed leg refunds its delta and rolls the cumulative counters back, which is what stops a client re-sending the same text from paying twice.

Note

The socket keeps its own refund and never calls the shared helper. It has to know whether the refund landed before it rewinds the counters, and the shared helper deliberately swallows that outcome. Swallowing it is right for a request and wrong for a connection.

The same request, sent twice#

A client that retries on a timeout may be retrying a request that succeeded. Idempotency-Key exists for that, and serving the stored result without charging again is the easy half.

The hard half is proving the replay is the same request. The usage row carries a fingerprint, an HMAC over the request's fields keyed by the caller's API key. The fingerprint is computed only for requests carrying an Idempotency-Key. If the same key returns with a different fingerprint, the API returns a conflict.

Situation	Answer	Why
Two requests, one key, concurrently	409 `idempotency_in_flight`	A Redis lock; the second cannot serve what the first has not finished
Replay of a refunded request	409 `idempotency_refunded`	The charge it refers to no longer stands
Replay with a different payload	422 `idempotency_mismatch`	Verified against the stored fingerprint, quantity, and model
Replay after a successful charge	200, no charge	The stored event is served again
Refund landing during a replay	Rechecked after synthesis	A refund must not race an uncharged serve

That last row is the subtle one. A replay checks the refund state before synthesizing, and a refund can land while inference is running. It is therefore checked a second time afterwards. Without that recheck there is a window in which a request is served free against a charge that has already been reversed.

The cache that stops answering#

Rate limiting, concurrency leases, and key verification all read Redis. Redis is a cache. Treating a cache as a source of truth is how an outage in a performance optimization becomes an outage in the product.

Every one of those paths fails open. When Redis is unreachable the rate limiter admits, the lease is granted, and the key cache misses through to the database. The charge then re-checks everything that matters. Key state, spending cap, and balance are all verified again at charge time, against Postgres. A cache that is down, stale, or lying can make the service more permissive for a few seconds. It cannot let a request spend money that is not there.

Failing open has an edge that is easy to miss. A client with no command timeout does not fail when the server wedges. It hangs, and a hung read inside a request path is worse than an error. Every Redis command therefore runs under an explicit budget, 250 milliseconds on the request path and two seconds on the heartbeat. A request-path expiry silently returns the fallback. Only the heartbeat is allowed to conclude that the connection is dead, and it proves that by closing a connection it has just demonstrated is not carrying commands.

what each layer is allowed to decide

Redis          advisory. may be wrong, may be gone
  rate limit   fails open
  lease        fails open
  key cache    fails open (miss to Postgres)

Postgres       authoritative
  key state    re-read at charge
  spend cap    CHECK constraint
  balance      CHECK constraint

nothing above the line can spend money

What is left over#

One case survives all of it, a process that dies between the charge and the refund. Nothing in a single-database design makes those two writes atomic across a crash. Pretending otherwise would mean inventing a distributed transaction for a problem that happens rarely.

So the failure is made loud instead. The 5xx log line carries the usage event id, and recovery is a manual grantCredits adjustment against that id. It is rare, it is visible, and it is recorded here where a customer never has to discover it. That is the most that can honestly be claimed for it.

That is the pattern under all four of these. Decide where the truth lives, put the guarantee there, and make every layer above it explicitly advisory. The parts that cannot be guaranteed get written down.