Avoiding Claude Code’s Billing Tax! The “claude-thermos” Tool That Keeps Your Cache Alive is Amazing

What Happened? News Overview

• It has been revealed that Claude Code’s prompt cache expires in 5 minutes (TTL), and reconstructing it incurs a hefty cost due to writing (1.25x) rather than reading (0.1x).
• While the main agent waits for sub-agents that take longer than 5 minutes, the cache disappears, leading to approximately 20% to 22% of the billing amount being wasted on this re-encoding cost.
• The new tool “claude-thermos” automatically avoids this unnecessary billing by “warming” the cache in the background.

Why Is This Important? Key Takeaways

• Hidden Cost Reduction: Results from around 185 session measurements showed that over one-fifth of the billing amount stemmed from re-writing costs due to cache expiration.
• Smart “Warming” Feature: Before 5 minutes elapse, it resends the last prompt with “max_tokens: 1,” resetting the cache’s expiration. This allows the 1.25x writing cost to be replaced with a 0.1x reading cost.
• Seamless Integration: By simply using uvx claude-thermos, it leverages existing Claude CLI arguments while acting as a local reverse proxy to monitor communications and optimize automatically.

🦈 Shark’s Eye (Curator’s Perspective)

The implementation of this tool is incredibly sensible, my friend! It’s not just about communicating periodically; it’s smartly monitoring the “lineage” as a reverse proxy. It identifies when the communication between the main agent and sub-agent is taking place, and precisely targets warming requests only when the main cache is at risk of being idle for 5 minutes. This “warming” is sent directly to the API, ensuring it doesn’t interfere with live streaming communications—definitely a pro move! In today’s development scene, where massive contexts of 200K to 500K tokens can be processed, preventing just a few rewrites can save you several dollars, making this a huge win! Truly a “guardian shark protecting the developer’s wallet”!

What’s Next?

If other LLM providers have similar cache expiration logic, such “cache maintenance tools” are likely to become a standard proxy layer. This technique will be essential for agent development, cutting costs without sacrificing development efficiency.

A Word from Haru-Same

I’ll gobble up all those unnecessary charges! With the saved cash, I’m treating myself to some delicious karupasu! 🦈🔥

Terminology Explanation

• Prompt Cache: A technique that retains previously sent input on the model’s side to reuse it, thereby minimizing computational costs and fees.

• TTL (Time To Live): The duration for which data is kept valid. Currently, Claude’s cache disappears 5 minutes after the last use.

• Reverse Proxy: A system that intermediates communication between the server and the client. Here, it operates locally to monitor Claude Code’s communications.

• Source: Claude-thermos – keeps your Claude session warm for you