Guides: writing prompts, counting tokens and what they cost

What a token is, and why it takes up room

A language model does not read letters or words: it reads tokens, chunks of text averaging about four characters in English. «Optimizer» is several tokens; «the» is one. Punctuation, spaces and line breaks count too.

That matters for two different reasons, and they are worth keeping apart. If you pay for an API, every input token is billed on each and every call: a system prompt you send ten thousand times a day is a fixed cost repeated ten thousand times. If you use a free tier, you pay nothing, but every token still eats context window and brings forward the point where the conversation gets cut or the model starts forgetting the beginning.

Either way, a token is budget. You pay in money or you pay in room.

Careful: characters are not tokens

This is the most common mistake when trimming by hand. Replacing «in order to» with «to» saves nine characters, but the real saving is in tokens, and the two do not move together: a long common word can be one token while a short rare one is three.

That is why a character count is an approximation, not a measurement. The only reliable figure is the one the tokenizer of the model you are actually using gives you, and it varies between models.

What must never be touched

Trimming a prompt has a hard limit, and crossing it does not save money: it breaks the call. Four kinds of content have to come out of any optimization untouched.

• ·Code blocks and inline code. A comment inside a function is not a pleasantry: it is part of what you are asking the model to interpret.
• ·URLs. Removing a word from an address makes it a different address, or none at all.
• ·Template variables such as {{user_name}}. If your system substitutes them later, altering them breaks the replacement silently.
• ·Numbers, units and identifiers. There is no such thing as a redundant digit.

By hand or automated?

By hand works once. If you have a fixed system prompt, you go over it, shorten it and leave it written: you need no tool for that, and in fact a tool would be overkill.

The problem shows up when the text changes on every call: what the user types, the documents you pull from a store, the history you drag along. There you cannot anticipate what will arrive, so the only option is applying the rules at the moment.

And there is a third case: when there are several of you. One person shortens their way and another theirs, and three months later every prompt on the team reads differently. A shared dictionary is what keeps the criterion from depending on who wrote it.

When it is worth putting the prompt through a model

Cutting the obvious waste by hand costs nothing and always pays off: if the only goal is a smaller prompt, you do not need a model for that.

The sums change when the problem is not size but ambiguity. No rule can fix that: working out that «summarize this» needs to know for whom, at what length, and what to do about the missing pieces is exactly the kind of judgement only a model makes. Spending a tenth of a cent on it pays off if it saves three messages correcting the answer.

What you must demand of that model is that it does not fill the gaps on its own. If the original said nothing about length and the result comes back with «max 200 words», the tool decided for you and you did not even notice. The right behaviour is to leave the gap marked so you fill it in.

And keep this in mind: a model gives a different version each run. For something running unattended in production that is a real drawback, and it is worth fixing the prompt once and reusing it rather than re-optimizing it on every run.