How to Save 30%–95% on LLM Tokens: A Practical Guide to Context Compression with Headroom

How to Save 30%–95% on LLM Tokens: A Practical Guide to Context Compression with Headroom

Last week, I ran a RAG pipeline at work, and a single long conversation consumed over 50,000 tokens. When the monthly bill arrived, Agent interaction costs dominated—and most of the billed content was just repetitive logs, full JSON arrays, and code snippets I had already reviewed.

If you've built AI Agents or RAG applications, you've likely faced the same pain point: LLMs charge by the token, but often less than half of the context we feed them is actually critical.

Today, I'll walk you through Headroom, an open-source project that recently hit #1 on GitHub Trending. Its purpose is straightforward: automatically compress tool outputs, logs, RAG chunks, file contents, and conversation history before they reach the LLM. The result? The same accurate responses, with drastically fewer tokens. Official benchmarks show 20%–95% savings, and everything runs locally—your code or logs never leave your machine.

By the end of this tutorial, you will:

1. Install Headroom and understand its three integration modes (Library / Proxy / Agent Wrap)
2. Use Python Library mode to compress a realistic log/JSON context and measure token savings
3. Launch a local Proxy to enable zero-code, automatic compression for any OpenAI-compatible request
4. Navigate common pitfalls and best practices

Let's get started.

Prerequisites

• Python 3.10+ (3.13 recommended) or Node.js environment
• pip, uv, or npm installed
• A working LLM API key (OpenAI / Anthropic) to verify results before and after compression
• Basic familiarity with calling the OpenAI SDK in Python

Note: If you work in Java, Go, or other languages, the Proxy mode requires zero code changes and works seamlessly across all languages.

Installation

Headroom provides both Python and TypeScript SDKs. The CLI is bundled with the Python package. I recommend a full installation via pip:

pip install "headroom-ai[all]"

If you prefer uv for cleaner environment isolation:

uv tool install --python 3.13 "headroom-ai[all]"

Once installed, run a health check to ensure everything is ready:

headroom doctor

This command checks routes, compressors, and network connectivity. A fully green output means success. Why run doctor first? It proactively identifies potential issues like missing ONNX Runtime dependencies or unsupported AVX2 instruction sets, saving you debugging time later.

Tip: On corporate networks with SSL interception, you might face certificate issues. In that case, use pip install --only-binary headroom-ai headroom-ai to skip Rust compilation, and set the HEADROOM_CA_BUNDLE environment variable as outlined in the README's enterprise network guide.

Quick Start: Three Integration Modes

Headroom offers three usage modes tailored to different workflows:

We'll focus on Library and Proxy modes, as they are most practical for general development.

Hands-on: Compressing a Real Conversation Context (Library Mode)

Here's a minimal, fully runnable example. Assume you have a message list containing tool outputs, log snippets, and JSON. Let's see the compression in action:

from headroom import compress
from openai import OpenAI

## Simulate a real Agent conversation context (tool output + JSON result)
messages = [
    {"role": "system", "content": "You are a helpful debugging assistant."},
    {"role": "user", "content": "Here are the service logs, help me analyze the FATAL error:\n" +
     "[2026-10-10 09:00:01] INFO  Starting service...\n" * 50 +
     "[2026-10-10 09:00:02] ERROR Connection timeout\n" * 30 +
     "[2026-10-10 09:00:03] FATAL NullPointerException at UserService.java:42\n"},
    {"role": "assistant", "content": "{\"files\": [\"UserService.java\", \"config.yml\"], \"status\": \"failed\", \"retry_count\": 3}"}
]

## Compress — Headroom automatically detects content types and selects the appropriate compressor
result = compress(messages, model="gpt-4o")

print(f"Before compression: {result.original_tokens} tokens")
print(f"After compression: {result.compressed_tokens} tokens")
print(f"Saved: {result.tokens_saved} tokens ({result.compression_ratio:.0%})")

## Send compressed messages to the LLM
client = OpenAI()
response = client.chat.completions.create(
    model="gpt-4o",
    messages=result.messages
)
print(f"\nModel response: {response.choices[0].message.content}")

This script does three things:

1. Constructs messages with repetitive logs and JSON: This represents the typical "bloat" in Agent workflows.
2. Calls compress(): Headroom's ContentRouter identifies content types automatically—logs are compressed using the Kompress-v2-base model, while JSON is handled by SmartCrusher (preserving key fields and anomalies while collapsing redundant array items).
3. Sends a request with compressed messages: result.messages maintains the exact same structure as the original, just with compressed content. It integrates seamlessly with the OpenAI SDK.

On my machine, this example achieves a 60%–70% compression rate, thanks to the heavy repetition of INFO/ERROR lines being flagged as compressible. Try swapping in your own long conversations to see real-world results.

How it works: Why does the model still answer correctly after compression? Headroom uses intelligent retention—critical lines like FATAL logs and key JSON status fields are marked as "important" and preserved, while repetitive or statistically invariant data is collapsed. Combined with the CCR (Compressible Context Retrieval) mechanism, if the LLM ever finds information missing, it can fall back to the built-in headroom_retrieve tool to fetch the original context.

Zero-Code Mode: The "Set-and-Forget" Proxy

If you don't want to manually call compress() in every request, Headroom's Proxy mode is ideal:

headroom proxy --port 8787

Once running, simply update your LLM request's base_url to http://localhost:8787. Here's how it looks with the OpenAI SDK:

from openai import OpenAI

## Point to the local Headroom Proxy
client = OpenAI(
    api_key="sk-your-api-key",  # Your actual API key
    base_url="http://localhost:8787/v1"
)

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Analyze these logs..."}]  # Compression is completely transparent
)

How it works: The Proxy intercepts all OpenAI-compatible requests, compresses them before forwarding to the real LLM, and returns the response. From a developer perspective, you just change one base_url and leave the rest of your code untouched. Java, Go, or any HTTP-based LLM client benefits immediately.

You can monitor real-time token savings at any time:

headroom dashboard

This opens a local web UI showing per-request compression rates and cumulative savings. After running it for a day, my dashboard showed a total input token reduction of 34.2%.

Wrapping AI Coding Agents (Claude Code / Codex)

For daily users of Claude Code, Codex, Cursor, or similar tools, Headroom offers a one-command setup:

headroom wrap claude

This command starts a local Proxy, automatically installs Serena (the semantic code navigation MCP server), and configures the agent to route traffic through Headroom. The next time you run claude, compression is already active. Revert anytime with headroom unwrap claude.

Supported agents include: claude, codex, grok, copilot, cursor, aider, cline, continue, goose, openhands, and over ten others. Run headroom wrap without arguments to see the full list.

Troubleshooting & Common Pitfalls

Q1: Does compression degrade model response quality?
Headroom runs benchmarks on GSM8K, TruthfulQA, and SQuAD v2 across 100 samples. Accuracy variance stays within statistical margins (±0.03). Simply put: Headroom never drops critical info. Repetitive content is compressed, while vital data is preserved.

Q2: Installation fails on x86 CPUs?
If your x86/x86_64 machine lacks AVX2 support (common on older cloud instances), ONNX Runtime will fail to load. Headroom automatically degrades to BM25 heuristic compression without throwing errors. For full ONNX functionality, switch to an AVX2-supported machine or use the official Docker image.

Q3: Installation fails on Intel Macs?
Intel macOS lacks precompiled ONNX Runtime binaries. Install it via Homebrew first, then use environment variables to point to it:

ORT_STRATEGY=system \
ORT_LIB_LOCATION="$(brew --prefix onnxruntime)/lib" \
ORT_PREFER_DYNAMIC_LINK=1 \
  pip install "headroom-ai[all]"

Q4: My context is short. Do I still need compression?
No. Headroom transparently passes short requests (below the min_input_words threshold) without modification. It shines in long logs, massive JSON arrays, RAG retrievals returning dozens of chunks, or Agents with extensive conversation histories.

Q5: Proxy environment variable changes aren't taking effect?
The Proxy snapshots environment variables on startup. If you export HEADROOM_OUTPUT_SHAPER=1 in another terminal, it won't affect the running Proxy. Restart the Proxy to apply changes, or use headroom wrap which dynamically pushes configs via a loopback POST /admin/runtime-env call.

Conclusion

Let's recap what we covered:

1. Installed Headroom with a single pip install
2. Used Library Mode to compress context in Python, achieving 60%+ savings
3. Deployed Proxy Mode for zero-code, cross-language token optimization
4. Wrapped AI Agents with a single command for seamless integration

Headroom delivers the most value in long conversations, heavy tool outputs, and RAG workflows. If your daily work involves AI Agents, this tool deserves a spot in your infrastructure stack.

Next steps: Try headroom dashboard for live monitoring, or run headroom perf for baseline performance tests. If you're interested in headroom learn (which auto-optimizes strategies from failed sessions), the official docs are highly detailed.

Feel free to leave questions or feedback in the comments, and give the repo a Star if you found it useful. In a future article, I'll cover integrating Headroom into a Java Spring Boot LLM pipeline—the Proxy mode is particularly Java-friendly. Stay tuned!