Claude Code: understanding and mastering your tokens

What?! Already?!

I've lost count of how many times I've said it (more or less out loud) watching my 5-hour Claude Code quota get blown through in 2-3 prompts.

If sometimes the size of the codebase could partly explain it, along with the lovely "2x consumption" during peak hours, at times I'd blow through the quota early in the morning on small codebases. I'm sure it's happened to everyone at least once.

And behind those blown quotas, the same problem: tokens. Input token / output token, why my prompt suddenly jumps from 500 tokens to 20k all at once.

So what exactly is a token? What generates the most of them? How do you cut consumption so you can get more out of your quota?

Under the hood of Claude Code

Token? What's that?

A token is a unit used by LLMs to read and generate text. Tokens aren't specific to Claude Code — GPT and the others use this unit too.

When you write text, or have Claude read text/code, it splits the input into tokens. OpenAI lets you see how they split your prompts into tokens from their tokenizer.

Watch out though: the way text is split into tokens differs from one model to another; so Claude doesn't tokenize the same way OpenAI does, for example. But on average, a word is estimated at 1.3 tokens in English, and about 2 tokens for a French word.

That's for a classic prompt when you're chatting naturally with your favorite LLM, but with code, token density is much higher. Modern tokenizers group frequent sequences into a single token — a common English word often becomes 1 token. But code fragments much more: operators, short variable names, indentation, and symbolic sequences (=>, !=, ::) each generate their own token where natural language would compress them. Result: for the same number of characters, a code file consistently consumes more tokens than a sentence of prose.

For a code snippet with fewer characters than the previous text, you can clearly see that more tokens were used.

Input / Output Tokens

Claude and the other LLMs split tokens into 2 categories:

• Input tokens: the tokens used to read your prompt or the analyzed code, for example
• Output tokens: the tokens used to generate the response (the code, for example)

These 2 categories generally have different prices: input tokens cost less than output tokens per million tokens. On current Claude 4 models (Sonnet, Opus), the ratio is about 1:5 — an output token costs five times more than an input token. This ratio can vary across models, but the order of magnitude stays stable. That's because producing text or code takes far more resources than reading it.

To cut this cost, Claude uses a prompt caching mechanism: the portions of context that don't change between two requests (system prompt, CLAUDE.md, tool definitions...) are cached. The silent injections at startup benefit a lot from this cache — they aren't recomputed on every prompt.

Context window

We've seen what tokens are and what they're for, now we'll look at how Claude and the other LLMs manage their "memory", and that memory is the context window. The context window is the set of tokens Claude can keep in memory. For most paying users, this limit is 200K tokens. Some plans give access to larger windows (500K on Enterprise, 1M on specific models with usage credits enabled), but 200K is the reference you'll hit in most Claude Code sessions. This limit covers both input tokens AND output tokens.

And this limit can be reached very quickly: the more Claude has to read your codebase, use tools, and the more you reuse the same session, the more your context fills up. Even more so because on every iteration, Claude sends back the previous context.

For example, you write a 200-token prompt, Claude uses 2000 tokens to reply (using a tool + generating the response), your context contains 2200 tokens. You're not satisfied, you write a new 800-token prompt → Claude sends back the previous 2200 + the tokens for the new response, and so on.

And this context window isn't expandable. Once the limit is reached, Claude will drop the oldest tokens and therefore lose context, and that's when the AI can start losing accuracy, or even hallucinate.

A context window is limited to the current session. As soon as you quit Claude and restart it, this context window goes back to zero and Claude doesn't remember anything. One partial exception: if you use memory files (like CLAUDE.md), their content is automatically reinjected at the start of every session. That's not memory in the strict sense because Claude doesn't "remember" anything, it's simply persistent context that will be loaded from the first prompt.

What Claude injects silently

We've got tokens, the context window. We've seen that the context window can grow very quickly depending on prompt size, codebase, etc. But behind your prompt, Claude uses a whole bunch of tokens without showing you. And you can see those tokens with the /context command. Give it a try, type the command in a fresh Claude session. Example from my setup:

You can see that in my case, before I've even written a prompt, 9% of the context is already filled.

But why did Claude consume 9% of the context when I haven't asked for anything? Because from the start, Claude injects quite a few things:

• Your CLAUDE.md and other memory files
• The definitions of the configured MCP tools (each MCP server adds its definitions to every session)
• The plugins
• The custom agents defined in your project
• Claude Code's own system prompt

And all of that can fill your context window fast before you've even started working. And since we said every prompt sends back the previous context, your first prompt will already consume at least 18k tokens (in my example).

Then, on every request, Claude will also inject the files it needs, consume extra tokens if it has to load other tools, use MCPs, run commands, etc.

What you'll also see in /context is an Autocompact buffer line — reserved space (~16% of the context window) that Claude Code sets aside exclusively for the compaction process. It isn't usable for your work. In other words, on a 200K-token context window, your actual workspace already starts reduced by about 33K tokens, before you've even written a first prompt.

To limit the damage, Claude has a tool.

Autocompact

Claude Code doesn't wait to hit the limit before acting. Autocompact triggers by default around 83% of the context window, keeping a reserve of about 13,000 tokens so it can finish the current response before compacting. That's why it can be surprising to see compaction trigger "early" — Claude is anticipating.

In practice, Claude compresses the context and generates a session summary, more compact, that uses fewer tokens. And it's this summary that gets sent with every prompt instead of the full previous context.

That does reduce the tokens sent each time, but it's still a summary generated by Claude, with the problems that implies: not everything is in there, and that's when Claude starts going off in all directions. It can forget it already handled certain things, ask a question you've already answered, contradict itself, etc.

There's also a /compact command you can trigger manually to do this compaction.

And to deal with this growth problem, Claude has another tool.

Sub-agents

Rather than stuffing itself with context, Claude can use sub-agents. These are sub-sessions specialized in a task with their own context. When you request a feature, Claude can typically create different specialized sub-agents, each using the necessary tools, rather than doing everything in the main session:

⏺ 6 Agents finished (ctrl+o to expand)
   ├─ Explore backend codebase · 23 tool uses · 104.5k tokens
   │  ⎿ Done
   ├─ Explore frontend codebase · 28 tool uses · 108.0k tokens
   │  ⎿ Done
   ├─ Write backend part · 4 tool uses · 20.0k tokens
   │  ⎿ Done
   ├─ Write frontend part · 1 tool uses · 18.6k tokens
   │  ⎿ Done
   ├─ Write unit tests · 7 tool uses · 64.0k tokens
   │  ⎿ Done
   └─ Write documentation · 24 tool uses · 52.7k tokens
      ⎿ Done

The upside is that no matter how many tokens each sub-agent consumes, your main session only receives a summary. If your agents all consume 150K tokens, they won't fill your main session's context and will only send a summary of a few tokens.

The downside... is that it can blow through your quota very quickly.

Saving your tokens

We've seen how easy it is to blow through your token consumption, and therefore your quota. Even if you'll have less of this issue on MAX plans, we don't all have the means to pay for those plans, so we have to watch our tokens.

CLAUDE.md: your project's memory

CLAUDE.md is THE most important file in your project. It's the one that gives Claude the information it needs to generate code the way you want it, that gives it your project architecture, your naming/git branching strategy, how to write and run tests, etc. All of that will save you a lot of input tokens (Claude will need to "guess" less about what it should do by analyzing your codebase) and output tokens (since it'll be guided by CLAUDE.md, there will be fewer round trips).

Watch out though: every token in your CLAUDE.md will be injected and reread at the start of every session. A 5,000-token CLAUDE.md is 5,000 input tokens consumed before you've even written a prompt. AND that multiplies by the number of sessions in your day. It needs to stay short and precise: don't turn it into complete documentation of your project.

Managing the size of your sessions

Claude systematically sends back the previous context on every turn. Two commands let you manage that, with very different behaviors:

• /compact :  compresses the history into a summary. The session thread is kept, Claude knows what was done. Use it when you want to keep going on the same topic but free up some space. You can pass it an instruction to guide the summary: /compact retain the architecture decisions.
• /clear : resets the context to zero. Claude doesn't remember anything. Use it when you switch features.

Simple rule: /compact to continue, /clear to start over.

One last command: /cost. It shows token consumption and the estimated cost of the current session. Handy for seeing in real time how much a session that's a bit too chatty has cost you.

Guiding Claude

Claude Code doesn't read your entire codebase on every request. It uses search tools to find the files it needs. Those tool calls are what consume tokens: locating a file, reading its contents, looking for another... On a large codebase, this exploration can represent several thousand tokens, sometimes just to find a file.

With @path/to/file, you skip that exploration: Claude receives the files directly without having to search for them. During a feature or a bugfix, identify the files to modify and the ones that will serve as reference ahead of time.

.claudeignore: exclude what's useless

Like a .gitignore, you can create a .claudeignore file at the root of your project to exclude files and folders. The syntax is identical. For example, it's recommended to exclude node_modules from your front-end projects, .nuxt, or vendor and var from your Symfony projects.

node_modules/
/var
.nuxt

Plan your development

Very useful for your large features: planning. Have Claude analyze the code before developing, plan the development steps, leave no room for doubt. The goal is to clearly detail the path to building the feature, what must be done, what must not be done, the important files to read, etc. A plan will often break down like this:

• Discovery phase
  • The important files Claude needs to know about
  • The anti-patterns
  • The external APIs to use
• Phases 1 to X
  • Splitting development into phases (phase 1 back, 2 front, 3 tests) with the implementation details for each

And luckily, Claude Code has a native tool for this: plan mode, activated with the /plan command (or Shift+Tab twice). It's a read-only mode where Claude can read your files, analyze your codebase, ask questions, but it can neither modify a file nor run a command. It produces a plan that you review, refine, then approve. Only after your approval does it leave plan mode and execute.

Even if generating a big plan can seem counterintuitive, the token savings are real: a failed implementation costs the tokens of the initial exploration, those of the incorrect implementation, those to detect the error, and those to fix it. Plan mode lets you anticipate all of that.

Mix the models

It's tempting to let the best model do everything. You switch to Opus, and you do everything with it. But it's useless in 90% of cases and above all it consumes a huge amount.

A logic you can adopt: Opus to plan, Sonnet to execute. You let the most advanced model plan the task, and Sonnet will be more than enough to execute the plan.

The price difference is significant: Sonnet 4.6 is 40% cheaper per input token and 40% cheaper per output token than Opus. And since most of the tokens in a development session are consumed during execution, the savings are substantial compared to a full session on Opus.

Claude Code bakes this pattern in natively with the /model command and the opus-plan option:

/model opus-plan

With this option enabled, Claude Code automatically routes planning tasks to Opus and execution tasks to Sonnet in the same session. You don't have to switch models manually or manage two separate contexts.

In practice, it looks like this:

1. You describe your feature → Opus analyzes, reasons, produces the plan
2. You approve the plan → Sonnet takes over for the implementation

A plan costs a few hundred tokens. An incorrect 400-line diff that you undo and regenerate costs thousands, twice, plus the turns to explain what was wrong.

Stop chatting

It's not the wildest trick to save tokens, but stop talking to Claude Code like it's your coworker! Be less verbose in your prompts, I still see plenty of colleagues writing their prompt the way you'd write a bug ticket.

It's an AI on the other side, no need to be that verbose. You can be more direct.

The difference is pretty significant. Here it's a bit forced for the example and it's a small prompt, but imagine on a large plan how many tokens get wasted...

A few practical tools

Claude-mem, a memory for Claude

We said above that a context window resets between two sessions, and that only CLAUDE.md persisted. That's true by default. But there's a plugin that changes that: claude-mem.

claude-mem is a Claude Code plugin that automatically captures what Claude does during your coding sessions, compresses those observations, and reinjects them as context in later sessions. All stored locally in a SQLite database.

Basically: you work two hours on an authentication bug, you close Claude Code, you come back the next day. The next session already starts knowing what you were working on. The bug you fixed the day before? Claude remembers it. It won't reread files already analyzed, or fall back on the same bad hypotheses you already ruled out.

What makes claude-mem interesting compared to a simple injection into CLAUDE.md is the compression: the raw transcript of a long session can represent tens of thousands of tokens. claude-mem distills it into short, typed observations (bugfix, decision, discovery), so the recall costs very little.

Installation:

# Depuis une session Claude Code

/plugin marketplace add thedotmack/claude-mem

/plugin install claude-mem

Caveman, back to the stone age

If you've ever asked Claude Code to write a small function and it answered with three introductory paragraphs before writing a line of code — you see the problem. Those tokens of politeness, contextualization, and rephrasing are pure waste.

That's the idea behind caveman: a Claude Code skill that makes the agent talk like a caveman — strips the filler, keeps the substance, uses fragments. It cuts about 75% of output tokens while keeping technical accuracy intact.

The logic is simple: caveman only affects output tokens — reasoning tokens aren't touched. Caveman doesn't make the brain smaller. It makes the mouth smaller.

There are several intensity levels:

- lite : just strips politeness formulas and filler

- full : switches to fragmented style

- ultra : maximum compression, useful when every token counts, less readable for debugging a complex error. A real caveman, grrrr

And as a bonus: Caveman Compress is a separate feature that rewrites your CLAUDE.md in caveman-speak, which reduces input tokens rather than output tokens. Also works on any other file you pass it, like a saved plan for example.

Installation:

/plugin marketplace add JuliusBrussee/caveman

/plugin install caveman@caveman

Then in a session:

/caveman        # mode par défaut

/caveman lite   # supprime juste le remplissage

/caveman full   # réponses en fragments

/caveman ultra  # compression maximale

To be honest, it'll save you a few hundred tokens, so it's not a game changer. No. The real advantage of caveman is more the reading speed and cutting down the noise.

Tokscale, visualize your ROI

You can't optimize what you don't measure. /cost gives you the cost of the current session, but nothing about your trends over the week or the month.

tokscale fills that gap. It's a CLI tool with an interactive interface that aggregates your token consumption across all your Claude Code sessions, and presents it from several angles:

- Overview : total consumption, estimated cost, trend

- Daily / Hourly : identify the days or time slots where you over-consume

- Models : breakdown by model (useful if you mix Sonnet and Opus)

- Agents : detail by session and by agent

What's handy: tokscale doesn't connect to any external service, it reads Claude Code's local logs directly. And if you use other tools (Cursor, Gemini CLI, Codex...), it aggregates everything in one place.

Installation:

npm install -g tokscale

Then:

tokscale               # vue interactive complète

tokscale --week        # consommation sur 7 jours

tokscale --client claude  # filtrer sur Claude Code uniquement

tokscale models        # détail par modèle

CodeBurn, the precise breakdown of your consumption

A bit more complete than tokscale. CodeBurn lets you visualize more precisely where your tokens go, down to the task level.

It reads session transcripts stored locally in ~/.claude/projects/ and classifies each exchange into 13 categories based on tool usage patterns.

The categories: coding, debugging, exploration, brainstorming — and especially conversation, the exchanges where Claude replies without using any tool. That's where the surprise comes: for the tool's creator, who was spending $200/day on Claude Code with no visibility, more than half the budget was going to pure conversation, not code generation.

CodeBurn is open source and runs entirely locally. It offers an interactive TUI dashboard with macOS menu bar support via SwiftBar, CSV/JSON export, and breakdown by project, model, tool, and MCP server.

npx codeburn

RTK (Rust Token Killer)

RTK takes the problem at a place nobody had addressed: the CLI outputs that Claude injects into the context.

The idea is simple: most CLI outputs sent to the LLM are noise: passing tests, verbose logs, progress bars, etc. RTK is a CLI proxy that filters and compresses those outputs before they reach the context window.

The numbers published by the author are pretty impressive: cargo test goes from 155 lines to 3 lines (98% reduction), git status from 119 to 28 characters (76%). On heavy usage it pays for itself very quickly.

RTK installs outside Claude and hooks into Claude to intercept requests. It supports more than 100 commands: git, pytest, cargo, docker, kubectl, eslint, tsc…

curl -fsSL https://raw.githubusercontent.com/rtk-ai/rtk/master/install.sh | sh
rtk init -g  # installe le hook Claude Code
rtk gain     # affiche les tokens économisés

The same init command works for Cursor, Codex, Windsurf, Gemini CLI — consistent with the article's philosophy: the principles apply to all LLM-based tools.

To wrap up

There you go, you now know more about how Claude works, what tokens are, where they go, and how to save them properly.

Of course, if you're on MAX plans, token consumption probably isn't your priority; and even then, with the new models, even MAX plans are starting to show their limits. But on lower plans, you now have a few tools to get more out of your quota.

Feel free to check the sources in the appendix to go further, Claude Code is a lot more complex when you really want to dig into the details!