How to cut a prompt's tokens without changing what it asks for

What a token is, and why it takes up room

A language model does not read letters or words: it reads tokens, chunks of text averaging about four characters in English. «Optimizer» is several tokens; «the» is one. Punctuation, spaces and line breaks count too.

That matters for two different reasons, and they are worth keeping apart. If you pay for an API, every input token is billed on each and every call: a system prompt you send ten thousand times a day is a fixed cost repeated ten thousand times. If you use a free tier, you pay nothing, but every token still eats context window and brings forward the point where the conversation gets cut or the model starts forgetting the beginning.

Either way, a token is budget. You pay in money or you pay in room.

Careful: characters are not tokens

This is the most common mistake when trimming by hand. Replacing «in order to» with «to» saves nine characters, but the real saving is in tokens, and the two do not move together: a long common word can be one token while a short rare one is three.

That is why a character count is an approximation, not a measurement. The only reliable figure is the one the tokenizer of the model you are actually using gives you, and it varies between models.

What must never be touched

Trimming a prompt has a hard limit, and crossing it does not save money: it breaks the call. Four kinds of content have to come out of any optimization untouched.

• ·Code blocks and inline code. A comment inside a function is not a pleasantry: it is part of what you are asking the model to interpret.
• ·URLs. Removing a word from an address makes it a different address, or none at all.
• ·Template variables such as {{user_name}}. If your system substitutes them later, altering them breaks the replacement silently.
• ·Numbers, units and identifiers. There is no such thing as a redundant digit.

By hand or automated?

By hand works once. If you have a fixed system prompt, you go over it, shorten it and leave it written: you need no tool for that, and in fact a tool would be overkill.

The problem shows up when the text changes on every call: what the user types, the documents you pull from a store, the history you drag along. There you cannot anticipate what will arrive, so the only option is applying the rules at the moment.

And there is a third case: when there are several of you. One person shortens their way and another theirs, and three months later every prompt on the team reads differently. A shared dictionary is what keeps the criterion from depending on who wrote it.

Why rules and not a model

The obvious way to shorten a prompt is to ask a model to do it. It is a bad idea except in one specific case.

Spending a model's tokens to save a model's tokens only pays off if the result is reused a great many times. Compressing a prompt with an API call costs tokens now to save tokens later, so it makes sense for a system prompt you will send thousands of times, and makes none for the message you are about to send once.

A rules engine does not have that problem. It costs nothing to run, answers instantly, and, what matters most when the result goes to production, it is deterministic: the same input gives the same output every time. A model gives you a slightly different version each run, which is exactly what you do not want in something that runs unattended.