Fake Caching and the Heartbeat Tax: How to Burn Your Money¶

Evidence Level: F (directly proven by source code)
Analysis Baseline: 4f843556

TL;DR¶

OpenAI's prompt caching is explicitly disabled by a single line of code: supports_cache_control = normalized_provider == "qwen". Only Qwen benefits. Meanwhile, a heartbeat fires every 4 hours regardless of idle/running state, triggering a full LLM session with zero concurrency control.

1. Fake Caching¶

# client.py:2599
supports_cache_control = normalized_provider == "qwen"

Caching Strategy by Provider¶

Provider	Caching	Actual Effect
OpenAI	Disabled	Full prompt retransmitted every round
Anthropic	Static prefix only	Dynamic content always resent
Qwen	Enabled	The sole beneficiary

Fixed Overhead¶

System prompt:  ~6,000 tokens
Tools:          ~5,000 tokens
─────────────────────────
Per-round fixed: ~11,000 tokens (zero cache hits)

2. The Heartbeat Tax¶

# heartbeat.py:473,512
if state in ("idle", "running"):  # no distinction
    create_task(heartbeat_session())  # no concurrency control

No lock, semaphore, rate limiting, jitter, or backpressure
Fixed 4-hour interval

Not a Ping-Pong—A Full Dress Rehearsal¶

A heartbeat is supposed to be "are you alive?"—a lightweight health check. Here's what actually happens:

Assembles the full context: soul.md + memory.md + skills/ + tools + enterprise_info + focus + triggers → all concatenated into a single system message, sent to the LLM
The LLM responds in earnest: the model receives ~20K tokens of full context and generates a thoughtful response
The response is thrown away: nobody reads it, nothing is done with it

This is not a heartbeat. This is making the LLM read the Agent's entire "personality + memory + capabilities" every 4 hours, write a book report, and then toss it in the trash.

What a Real Heartbeat Should Look Like¶

A proper heartbeat:
  echo pang → "pang"
  2 tokens

The actual heartbeat:
  → assemble soul + memory + skills + tools + enterprise_info + focus + triggers
  → send to LLM
  ← LLM generates a thoughtful response
  → response is thrown away
  ~20,000 tokens

From 2 to 20,000—a 10,000× minimum. This isn't poor optimization. It's never having asked whether a heartbeat should touch the LLM at all.

Cost Estimate¶

1 Agent × 6/day × ~20K tokens = ~120K tokens/day
100 Agents × 6/day × ~20K tokens = ~12M tokens/day

Heartbeat only. Excludes actual user usage.

What This Costs¶

Input:  12M tokens/day
Output: ~0.2M tokens/day

                   Input Price    Output Price    Per Day    Per Month
Claude Sonnet 5    $3/1M         $15/1M          ~$39       ~$1,170
Claude Opus 4.8    $5/1M         $25/1M          ~$65       ~$1,950

100 agents, doing nothing useful. Sonnet burns over a thousand dollars a month. Opus doubles it.

Put another way: at ~$11.70/agent/month on Sonnet, two agents' heartbeats alone cost as much as a Claude Pro subscription ($20/month). On Opus, one agent is enough.

A proper heartbeat needs only echo pang—2 tokens.

3. Compounding Effects¶

Fake caching + heartbeat: every 4 hours, every Agent retransmits ~20K tokens of full context—exactly what caching should have prevented.

Heartbeat + no concurrency control: all Agents restart simultaneously → all heartbeats fire simultaneously → API self-DDoS.

Heartbeat + no jitter: fixed interval, all Agents expire at the same time.

Heartbeat vs Dashboard: Two Disasters, Perfectly Misaligned¶

The heartbeat burns 20K tokens every 4 hours, but never updates agent.status.

Meanwhile, the Dashboard auto-refreshes every 15 seconds—43,600 HTTP requests per day. But the database field it reads is one the heartbeat never writes to.

Heartbeat:  6×/day × 20K tokens = 120K tokens/day → never updates status
Dashboard:  2,880 refreshes/day × 42 requests/refresh = 43,600 requests/day → reads data the heartbeat never wrote

One burns money producing nothing. The other refreshes furiously seeing nothing.
Both working hard, in completely wrong directions.

4. Relationship to AI Coding¶

== "qwen" cannot be intentional design: it's the minimal implementation—enable caching only for the provider you tested, "leave the rest for later"
Heartbeat logic was task-by-task accretion: first "trigger every 4 hours," then "check state," and nobody ever added concurrency control
AI won't proactively optimize costs: heartbeat, caching, concurrency—these are cross-task system properties; AI only sees local requirements in each task

Key Code¶

File	Lines	Content
`client.py`	2599	`supports_cache_control = normalized_provider == "qwen"`
`heartbeat.py`	473	`if state in ("idle", "running"):`
`heartbeat.py`	512	`create_task(...)` with no concurrency control
`heartbeat.py`	282	`_build_heartbeat_instruction()` → builds heartbeat instruction
`heartbeat_runtime.py`	81	`enqueue_heartbeat_runtime()` → enqueues as runtime run
`worker_service.py`	224	`RuntimeModelStepService(prompt_builder=build_agent_context)` → defaults to full context builder
`model_step_service.py`	1208	`self._prompt_builder(...)` → calls `build_agent_context`, assembling soul + memory + skills + enterprise_info + focus + triggers